Data analysis method, device, equipment, medium and product
By obtaining, cleaning and analyzing the behavioral data of practitioners, and using association rules and clustering analysis algorithms to identify abnormal behaviors, the problem of inability to identify potential risks in the existing technology is solved, and timely monitoring of the behavior of institutions and practitioners is achieved. Early warning ensures the safety and compliance of the system.
Patent Information
- Application Number
- CN202411290690.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-13
AI Technical Summary
The existing technology cannot promptly and effectively identify and discover potential risks of the behavior of institutions and practitioners and provide early warning information, making it difficult to achieve comprehensive monitoring and early warning of the information of many practitioners.
By obtaining the behavioral data of practitioners, performing data cleaning and format conversion, using association rules mining algorithm to mine the correlation between behavioral data, and clustering analysis of behavioral data through clustering analysis algorithm to generate behavioral analysis results and output early warning Message to indicate abnormal behavior.
It realizes timely discovery, accurate identification and rapid response to the behavior of institutions and practitioners, and ensures the operational safety and compliance of the system.
Smart Images

Figure CN119128765A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of network security technology, and in particular, relates to a data analysis method, device, equipment, medium and product. Background Art
[0002] With the continuous development of various industries, each industry needs to comply with certain industry standards. We can strengthen supervision of the behavior of relevant institutions and practitioners in various industries to prevent them from engaging in illegal and irregular activities. Therefore, how to effectively control the behavior of institutions and practitioners and prevent the occurrence of risks has become an important issue that needs to be resolved urgently.
[0003] Related technologies mostly rely on manual supervision by regulatory agency staff. This method is not only inefficient and prone to errors, but also makes it difficult to achieve comprehensive monitoring and early warning of the information of numerous practitioners. It is also impossible to timely and effectively identify and discover the potential risks of institutions and practitioners' behavior and provide early warning information.
[0004] Therefore, there is an urgent need for a method that can timely analyze, monitor and warn the behavior of many practitioners and institutions. Summary of the invention
[0005] The embodiments of the present application provide a data analysis method, apparatus, device, storage medium and computer program product to solve the technical problem that the related technology cannot timely and effectively identify and discover the potential risks of the behavior of institutions and practitioners and provide early warning information.
[0006] In a first aspect, an embodiment of the present application provides a data analysis method, the method comprising:
[0007] Obtaining behavioral data of employees of each institution reported by each institution, wherein the behavioral data is used to perform abnormal analysis on historical behaviors of employees and institutions corresponding to the behavioral data;
[0008] After data cleaning and data format conversion of the behavior data, the behavior data is stored in a blank data set to obtain a behavior data set, wherein the behavior data set includes a plurality of the behavior data;
[0009] Using an association rule mining algorithm to mine the associations between the behavior data in the behavior data set to obtain target association rules;
[0010] Performing cluster analysis on a plurality of the behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result;
[0011] Performing association analysis on the cluster analysis results according to the target association rule to generate behavior analysis results;
[0012] When the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, an early warning message is output, where the early warning message is used to indicate that there is abnormal behavior in the institution or the practitioner.
[0013] In a second aspect, an embodiment of the present application provides a data analysis device, the device comprising:
[0014] An acquisition module, used to acquire behavior data of employees of each institution reported by each institution, wherein the behavior data is used to perform abnormal analysis on historical behaviors of employees and institutions corresponding to the behavior data;
[0015] A preprocessing module, used for performing data cleaning and data format conversion on the behavior data and storing the data in a blank data set to obtain a behavior data set, wherein the behavior data set includes a plurality of the behavior data;
[0016] A mining module, used for mining the associations between the behavior data in the behavior data set by using an association rule mining algorithm to obtain target association rules;
[0017] A clustering module, used for performing cluster analysis on a plurality of the behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result;
[0018] A generating module, used for performing association analysis on the clustering analysis results according to the target association rule to generate behavior analysis results;
[0019] The output module is used to output warning information when the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, and the warning information is used to indicate that the institution or the practitioner has abnormal behavior.
[0020] In a third aspect, an embodiment of the present application provides a data analysis system, the system comprising:
[0021] A data collection module, used to obtain behavioral data of employees of each institution reported by each institution, wherein the behavioral data is used to perform abnormal analysis on historical behaviors of employees and institutions corresponding to the behavioral data;
[0022] A data processing module, used for cleaning and converting the behavior data into a data format, and storing the data in a blank data set to obtain a behavior data set, wherein the behavior data set includes a plurality of the behavior data;
[0023] The data analysis module uses an association rule mining algorithm to mine the associations between the behavior data in the behavior data set to obtain a target association rule; performs cluster analysis on a number of the behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result; performs association analysis on the cluster analysis result according to the target association rule to generate a behavior analysis result;
[0024] The early warning module outputs early warning information when the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, wherein the early warning information is used to indicate that the institution or the practitioner has abnormal behavior.
[0025] In a fourth aspect, an embodiment of the present application provides an electronic device, the device comprising:
[0026] A processor and a memory storing programs or instructions;
[0027] The above method is implemented when the processor executes the program or instruction.
[0028] In a fifth aspect, an embodiment of the present application provides a machine-readable storage medium, on which a program or instruction is stored, and the program or instruction implements the above method when executed by a processor.
[0029] In a sixth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the above method.
[0030] In the present application, firstly, by acquiring a large amount of behavioral data of practitioners from institutions, and performing data cleaning and data processing on the data, a large amount of high-quality behavioral data is provided for subsequent data analysis to obtain behavioral analysis results, so that the obtained behavioral analysis results are more accurate and reliable; secondly, by using an association rule mining algorithm to mine the correlation between behavioral data, and by using a cluster analysis algorithm to perform cluster analysis on behavioral data, the potential information between behavioral data can be deeply mined, thereby identifying whether institutions and practitioners have abnormal behaviors; finally, by outputting early warning information when the behavioral analysis results indicate the existence of institutions or practitioners with abnormal behaviors, it is ensured that the system can achieve timely discovery, accurate identification and rapid response to abnormal behaviors, thereby effectively ensuring the operational safety and compliance of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0032] Figure 1 is a flow chart of a data analysis method according to an embodiment of the present invention;
[0033] Figure 2 is a schematic diagram of the overall process of a data analysis method according to an embodiment of the present invention;
[0034] Figure 3 is a structural block diagram of a data analysis device according to an embodiment of the present invention;
[0035] Figure 4 1 is a block diagram showing a structure of a data analysis system according to an embodiment of the present invention;
[0036] Figure 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0038] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0039] In addition, it should be noted that the acquisition, storage, use, and processing of data in the embodiments of the present application comply with the relevant provisions of national laws and regulations.
[0040] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0041] It should be noted that the acquisition, storage, use and processing of data in the embodiments of the present application are in compliance with the relevant provisions of national laws and regulations.
[0042] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0043] In order to solve the technical problem that the existing technology cannot timely and effectively identify and discover the potential risks of the behavior of institutions and practitioners and provide early warning information, the embodiment of the present application provides a data analysis method, which can be applied in a data analysis system to perform abnormal analysis on the practitioners of each institution and the historical behavior of the institution. Figure 1 is a flow chart of a data analysis method according to an embodiment of the present invention. Figure 1 As shown, the data analysis method may include the following steps:
[0044] S110, obtaining the behavior data of the practitioners of each institution reported by each institution, the behavior data is used to perform abnormal analysis on the historical behaviors of the practitioners and institutions corresponding to the behavior data.
[0045] In this embodiment, the behavior data may include various types of information of practitioners, for example, the work records, violation information, and identity information of the practitioners.
[0046] In this way, through the extensive collection of various types of information on practitioners, the abnormal analysis results of practitioners and institutions can be made more accurate.
[0047] S120, after data cleaning and data format conversion of the behavior data, the behavior data is stored in a blank data set to obtain a behavior data set, which includes a plurality of behavior data.
[0048] In this embodiment, in order to improve the quality of behavioral data, errors, outliers, missing values, etc. in the behavioral data can be corrected or deleted through data cleaning; the method of data cleaning is not limited here; because the collected behavioral data may come from multiple different data sources (institutions), the data format and encoding method may not be unified, therefore, the format of the behavioral data can be unified through data format conversion; the method of data format conversion is not limited here.
[0049] In this way, by cleaning the behavioral data and converting the data format, the accuracy and consistency of the behavioral data are ensured, and the accuracy and credibility of subsequent data analysis results and generated warning information are improved.
[0050] S130, using an association rule mining algorithm to mine the associations between the behavior data in the behavior data set to obtain target association rules.
[0051] In this embodiment, association rule mining algorithms such as Apriori algorithm and FP-Growth algorithm can be used to mine the behavior data set, which can find association rules based on the generation of frequent item sets (ie, behavior data that often appear together).
[0052] For example, the association rule mining algorithm can output multiple association rules, each of which represents the association between behavioral data. Association rules are usually expressed in the form of "if...then...", for example, "if practitioner A performs behavior X, then the practitioner is likely to perform behavior Y."
[0053] After the association rule mining algorithm outputs multiple association rules, the target association rules can be obtained by screening, and there is no limitation on this.
[0054] S140, performing cluster analysis on a plurality of behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result.
[0055] In this embodiment, a suitable clustering analysis algorithm can be selected to perform clustering analysis on several behavior data in the behavior data set. The clustering analysis algorithm can be selected based on the characteristics of the behavior data and the analysis requirements. For example, the K-means algorithm is suitable for large data sets and has a good clustering effect on spherical clusters, while the DBSCAN algorithm can identify clusters of any shape and does not require the number of clusters to be specified in advance, and there is no limitation on this.
[0056] In this embodiment, the characteristics of the behavior data may include: identity information of the practitioners, transaction records, communication records, login behaviors and operation behaviors, etc.
[0057] S150, performing association analysis on the cluster analysis results according to the target association rule to generate behavior analysis results.
[0058] In this embodiment, association analysis is performed on the clustering analysis results according to the target association rules to generate behavior analysis results, which may include two types of association analysis: performing association analysis between any multiple behavior feature clusters in the clustering analysis results according to the target association rules to obtain the behavior analysis results, and performing internal association analysis on any one behavior feature cluster in the clustering analysis results according to the target association rules to obtain the behavior analysis results.
[0059] In this way, it is possible to perform correlation analysis on the behavioral data between clusters and within clusters, making the behavioral analysis results more multi-dimensional and multi-layered, and able to more comprehensively reveal the behavioral patterns, potential trends, and possible anomalies and risk points of practitioners and institutions.
[0060] S160, when the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, output warning information, where the warning information is used to indicate that there is abnormal behavior in the institution or practitioner.
[0061] In this embodiment, the output warning information may include the specific content of the abnormal behavior, the institutions or practitioners involved, and the possible potential risks. The specific content of the abnormal behavior may include a description of the abnormal behavior, the time of occurrence, the scope of impact, the possible consequences, and the recommended response measures or action guidelines. The output warning information can be output to relevant departments and personnel through output channels. For example, the output channels may include text messages, emails, application push, internal management systems, etc.
[0062] In this way, by outputting early warning information, the system can achieve timely discovery, accurate identification and rapid response to abnormal behavior, thereby effectively ensuring the operational safety and compliance of the system.
[0063] In the embodiments of the present application, firstly, by acquiring the behavioral data of a large number of practitioners of an institution and performing data cleaning and data processing on the data, a large amount of high-quality behavioral data is provided for subsequent data analysis to obtain behavioral analysis results, so that the obtained behavioral analysis results are more accurate and reliable; secondly, by using an association rule mining algorithm to mine the correlation between the behavioral data, and by using a cluster analysis algorithm to perform cluster analysis on the behavioral data, the potential information between the behavioral data can be deeply mined, thereby identifying whether there are abnormal behaviors of the institutions and practitioners; finally, by outputting early warning information when the behavioral analysis results indicate the existence of institutions or practitioners with abnormal behaviors, it is ensured that the system can achieve timely discovery, accurate identification and rapid response to abnormal behaviors, thereby effectively ensuring the operational safety and compliance of the system.
[0064] In some embodiments, an association rule mining algorithm is used to mine the associations between behavior data in a behavior data set to obtain a target association rule, including:
[0065] The association rule mining algorithm is used to mine the associations between the behavior data in the behavior data set and obtain several association rules;
[0066] Determine the association rule with the largest confidence and support among several association rules as the target association rule.
[0067] In this embodiment, for each generated association rule, its confidence and support can be calculated:
[0068] Confidence can represent the probability that the result item (such as behavior Y) will also occur under the condition that the premise item (such as behavior X) occurs. For example, the calculation formula can be: Confidence (X→Y) = P (Y|X).
[0069] Support represents the probability of the antecedent item and the consequence item occurring at the same time. For example, the calculation formula may be: Support(X→Y)=P(X,Y).
[0070] In this embodiment, when there is no association rule with the highest confidence and support among several association rules, the target association rule can be comprehensively determined based on indicators such as lift, leverage, and certainty.
[0071] In this way, by determining the target association rules, the system can help identify potential risk behavior associations between practitioners, and provide important basis for subsequent monitoring, early warning and intervention measures, thereby providing more accurate and effective support for the regulatory side.
[0072] In some embodiments, cluster analysis is performed on several behavior data in the behavior data set according to a cluster analysis algorithm to obtain cluster analysis results, including:
[0073] Extracting behavioral features of several behavioral data in the behavioral data set;
[0074] Based on the behavioral characteristics of the behavioral data, a cluster analysis algorithm is used to group several behavioral data in the behavioral data set to obtain multiple behavioral characteristic clusters;
[0075] Multiple behavioral feature clusters are determined as cluster analysis results.
[0076] In this embodiment, the behavioral characteristics of the behavioral data may include: identity information of practitioners, transaction records, communication records, login behaviors and operation behaviors, etc.
[0077] In this embodiment, after the cluster analysis algorithm is determined, the extracted behavior features can be used as input, and the selected cluster analysis algorithm can be applied to perform cluster analysis. The clustering process can divide the behavior data into different groups according to the similarity or difference between the behavior features. The behavior data within each group has a high similarity in features, while the behavior data between different groups has a large difference. In this way, multiple behavior feature clusters can be obtained.
[0078] After obtaining multiple behavioral feature clusters, you can evaluate the multiple behavioral feature clusters to confirm whether the clustering effect meets expectations. The evaluation method may include internal indicators (such as silhouette coefficient) and external indicators (such as matching degree with known labels), as well as subjective judgment through the professional knowledge and experience of management-side business personnel to determine the final cluster analysis results.
[0079] Thus, through the above steps, behavioral features can be systematically extracted from the behavioral data set, cluster analysis can be performed, and cluster groups with specific behavioral features can be identified, providing strong support for subsequent behavioral supervision and early warning of practitioners.
[0080] In some embodiments, performing association analysis on the cluster analysis results according to the target association rule to generate behavior analysis results includes:
[0081] Merge any multiple behavioral feature clusters in the cluster analysis results to obtain a cross-dataset;
[0082] The cross-datasets are mined according to the target association rules to obtain the behavior analysis results.
[0083] In this embodiment, association analysis can be performed on any multiple behavior feature clusters in the cluster analysis result according to the target association rule to obtain the behavior analysis result.
[0084] For example, if the system finds that there may be some correlation between the two clusters of "high-frequency trading behavior" and "large-amount trading behavior", it may choose to merge the two clusters.
[0085] The system found that after the clustering of "high-frequency trading behavior" and "large-value trading behavior" was merged, there was a significant correlation with "illegal operation records". Then, the system can generate the following behavioral analysis results: "Some practitioners exhibit both high-frequency trading and large-value trading behaviors, and a high proportion of these practitioners have illegal operation records." This suggests that institutions need to pay special attention to these practitioners to prevent potential violations.
[0086] In this way, by merging different behavioral feature clusters into a cross-dataset, cross-cluster correlation analysis can be achieved, making the resulting behavioral analysis results more multi-dimensional and multi-layered, and able to more comprehensively reveal the behavioral patterns, potential trends, and possible anomalies and risk points of practitioners and institutions.
[0087] In some embodiments, performing association analysis on the cluster analysis results according to the target association rule to generate behavior analysis results includes:
[0088] For any behavior feature cluster in the cluster analysis result, extract all behavior data of the behavior feature cluster;
[0089] According to the target association rules, the correlation between all behavior data is mined to obtain the behavior analysis results.
[0090] In this embodiment, internal association analysis can be performed on any behavior feature cluster in the cluster analysis result according to the target association rule to obtain the behavior analysis result.
[0091] For example, the behavior feature cluster is a high-frequency small-amount transaction cluster. This cluster can include all the data of practitioners who show high-frequency and small-amount transaction behaviors. If the system finds that multiple practitioners have conducted high-frequency, small-amount transactions with various transaction objects on consecutive working days, then these behaviors will be regarded as abnormal behaviors and corresponding warning information will be generated.
[0092] In this way, by mining the internal associations of a single behavioral feature cluster, the association analysis within the cluster can be achieved.
[0093] In some embodiments, when the behavior analysis results indicate that there are institutions or practitioners with abnormal behavior, outputting warning information includes:
[0094] In the event that the behavioral analysis results indicate the presence of an institution or practitioner with abnormal behavior, generating early warning information based on the behavioral analysis results;
[0095] According to the warning information, determine the target warning strategy corresponding to the warning information from the preset corresponding relationship table;
[0096] According to the target warning strategy, the warning information is sent to the management end.
[0097] In this embodiment, the preset correspondence table may include the correspondence between multiple warning levels and warning strategies. The warning level can be determined by the abnormality degree of the abnormal behavior. The warning strategy can be used to indicate the warning channel and / or warning object. The warning channel may include internal systems, emails, text messages, etc., and the warning objects may include management terminals, relevant agencies, etc.
[0098] In this embodiment, in order to enable the warning object to understand the warning information more clearly and accurately, the warning information can be visualized and sent to the management end. The visualization method is not limited here.
[0099] In this embodiment, the warning level can be determined by the degree of abnormality of the abnormal behavior, including: when the degree of abnormality of the abnormal behavior is greater than the first abnormal threshold, a first-level warning information is generated; when the degree of abnormality of the abnormal behavior is less than or equal to the first abnormal threshold and greater than the second abnormal threshold, a second-level warning information is generated, and the first abnormal threshold is greater than the second abnormal threshold; when the degree of abnormality of the abnormal behavior is less than or equal to the second abnormal threshold, a third-level warning information is generated.
[0100] In this embodiment, the early warning strategy may include: a first-level early warning strategy, sending a first-level early warning information to the management end through all early warning channels; a second-level early warning strategy, sending a second-level early warning information to the management end through some early warning channels; a third-level early warning strategy, sending a third-level early warning information to the management end through any early warning channel.
[0101] In this way, the system can quickly generate warning information when abnormal behavior is discovered, and deliver it to management personnel in a timely manner according to the target warning strategy, thereby achieving early detection, early warning, and early disposal of risks, and improving the system's risk prevention capabilities.
[0102] Figure 2 FIG. 1 shows an overall flow chart of a data analysis method provided by another embodiment of the present application. Figure 2 As shown, the data analysis method may include the following steps:
[0103] S201, each institution shall report behavioral data.
[0104] In this embodiment, the behavior data of employees of each institution reported by each institution is first obtained.
[0105] S202, determine whether the behavior data is repeated, if so, execute S205, otherwise execute S203.
[0106] In this embodiment, by judging whether the behavior data reported by each institution are repeated, since the behavior data can also include the information of the practitioners, it is possible to judge whether the practitioners of each institution have abnormal part-time behavior by judging whether the behavior data are repeated. If abnormal part-time behavior of the practitioners is found, S205 is directly executed to output the warning information, otherwise the risk analysis continues.
[0107] S203, data processing.
[0108] In this embodiment, the behavior data of employees of each institution reported by each institution may be first processed. Examples of data processing may include: data classification, data storage, data cleaning and format conversion.
[0109] S204, data analysis.
[0110] In this embodiment, an association rule mining algorithm and a cluster analysis algorithm may be used to perform cluster analysis on the behavior data;
[0111] S205, output warning information.
[0112] In this embodiment, warning information may be output based on the behavior analysis result obtained by data analysis or the result of determining whether the behavior data is repeated.
[0113] Figure 3 A schematic diagram of the structure of a data analysis device provided in another embodiment of the present application is shown. For the sake of ease of explanation, only the parts related to the embodiment of the present application are shown.
[0114] Reference Figure 3 , the data analysis device 300 may include:
[0115] The acquisition module 301 is used to acquire the behavior data of the practitioners of each institution reported by each institution, and the behavior data is used to perform abnormal analysis on the historical behavior of the practitioners and institutions corresponding to the behavior data;
[0116] The preprocessing module 302 is used to clean the behavior data and convert the data format, and then store the data in a blank data set to obtain a behavior data set, which includes a plurality of behavior data;
[0117] A mining module 303 is used to mine the associations between the behavior data in the behavior data set using an association rule mining algorithm to obtain a target association rule;
[0118] The clustering module 304 is used to perform cluster analysis on a plurality of behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result;
[0119] A generating module 305 is used to perform association analysis on the clustering analysis results according to the target association rule to generate behavior analysis results;
[0120] The output module 306 is used to output warning information when the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, and the warning information is used to indicate that the institution or practitioner has abnormal behavior.
[0121] In some embodiments, the mining module 303 may include the following units:
[0122] A mining unit, used for mining the associations between the behavior data in the behavior data set by using an association rule mining algorithm to obtain a number of association rules;
[0123] The determination unit is used to determine the association rule with the largest confidence and support among several association rules as the target association rule.
[0124] In some embodiments, the clustering module 304 may include the following units:
[0125] An extraction unit, used for extracting behavior features of a plurality of behavior data in the behavior data set;
[0126] A grouping unit, used to group a number of behavior data in the behavior data set by using a clustering analysis algorithm based on the behavior characteristics of the behavior data to obtain a plurality of behavior characteristic clusters;
[0127] The clustering unit is used to determine the clustering of multiple behavior features as a cluster analysis result.
[0128] In some embodiments, the generation module 305 may include the following units:
[0129] A merging unit, used to cluster and merge any multiple behavioral features in the cluster analysis results to obtain a cross data set;
[0130] The mining unit is used to mine cross-data sets according to target association rules to obtain behavior analysis results.
[0131] In some embodiments, the generation module 305 may include the following units:
[0132] An extraction unit, used for extracting all behavior data of any behavior feature cluster in the cluster analysis result;
[0133] The mining unit is used to mine the associations among all behavior data according to the target association rules to obtain the behavior analysis results.
[0134] In some embodiments, the output module 306 may include the following units:
[0135] a generating unit, for generating warning information according to the behavior analysis results, when the behavior analysis results indicate the presence of an institution or practitioner with abnormal behavior;
[0136] A determination unit, configured to determine, according to the warning information, a target warning strategy corresponding to the warning information from a preset correspondence table, wherein the correspondence table includes a correspondence between a plurality of warning levels and warning strategies, wherein the warning level is determined by the abnormal degree of the abnormal behavior, wherein the warning strategy is used to indicate a warning path and / or a warning object;
[0137] The sending unit is used to send the warning information to the management end according to the target warning strategy.
[0138] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application, and are devices corresponding to the above-mentioned data analysis method. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device. Its specific functions and technical effects brought about can be found in the method embodiment part, which will not be repeated here.
[0139] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0140] Figure 4 A structural schematic diagram of a data analysis system provided by another embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.
[0141] Reference Figure 4 , the data analysis system 400 may include:
[0142] The data collection module 401 is used to obtain the behavior data of the practitioners of each institution reported by each institution, and the behavior data is used to perform abnormal analysis on the historical behavior of the practitioners and institutions corresponding to the behavior data;
[0143] The data processing module 402 is used to clean the behavior data and convert the data format, and then store the data in a blank data set to obtain a behavior data set, which includes a plurality of behavior data;
[0144] The data analysis module 403 uses an association rule mining algorithm to mine the associations between the behavior data in the behavior data set to obtain a target association rule; performs cluster analysis on a number of behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result; performs association analysis on the cluster analysis result according to the target association rule to generate a behavior analysis result;
[0145] The early warning module 404 outputs early warning information when the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior. The early warning information is used to indicate that there is abnormal behavior in the institution or practitioner.
[0146] Figure 5 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 5 As shown:
[0147] The device may include a processor 501 and a memory 502 storing programs or instructions.
[0148] When the processor 501 executes the program, the steps in any of the above method embodiments are implemented.
[0149] Exemplarily, the program may be divided into one or more modules / units, one or more modules / units are stored in the memory 502, and executed by the processor 501 to complete the present application. One or more modules / units may be a series of program instruction segments capable of completing a specific function, and the instruction segments are used to describe the execution process of the program in the device.
[0150] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0151] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.
[0152] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) machine-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.
[0153] The processor 501 implements any one of the methods in the above embodiments by reading and executing the program or instruction stored in the memory 502 .
[0154] In one example, the electronic device may further include a communication interface 503 and a bus 504. The processor 501, the memory 502, and the communication interface 503 are connected via the bus 504 and communicate with each other.
[0155] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0156] Bus 504 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 504 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.
[0157] In addition, in combination with the method in the above embodiment, the embodiment of the present application may provide a machine-readable storage medium for implementation. The machine-readable storage medium stores a program or instruction; when the program or instruction is executed by a processor, any one of the methods in the above embodiment is implemented. The machine-readable storage medium can be read by a machine such as a computer.
[0158] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0159] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0160] An embodiment of the present application provides a computer program product, which is stored in a machine-readable storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0161] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0162] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware or their combination. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.
[0163] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0164] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by a computer program or instruction. These programs or instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0165] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A data analysis method, characterized in that: The method comprises: Obtaining behavioral data of employees of each institution reported by each institution, wherein the behavioral data is used to perform abnormal analysis on historical behaviors of employees and institutions corresponding to the behavioral data; After data cleaning and data format conversion of the behavior data, the behavior data is stored in a blank data set to obtain a behavior data set, wherein the behavior data set includes a plurality of the behavior data; Using an association rule mining algorithm to mine the associations between the behavior data in the behavior data set to obtain a target association rule; Performing cluster analysis on a plurality of the behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result; Performing association analysis on the cluster analysis results according to the target association rule to generate behavior analysis results; When the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, an early warning message is output, where the early warning message is used to indicate that there is abnormal behavior in the institution or the practitioner.
2. The method according to claim 1, characterized in that The adopting of the association rule mining algorithm to mine the associations between the behavior data in the behavior data set to obtain the target association rules includes: Using an association rule mining algorithm to mine the associations between the behavior data in the behavior data set to obtain a number of association rules; An association rule with the greatest confidence and support among the plurality of association rules is determined as the target association rule.
3. The method according to claim 1, characterized in that: The clustering analysis is performed on a plurality of the behavior data in the behavior data set according to the clustering analysis algorithm to obtain a clustering analysis result, including: Extracting behavior features of a plurality of behavior data from the behavior data set; Based on the behavior characteristics of the behavior data, using the cluster analysis algorithm to group a number of the behavior data in the behavior data set to obtain a plurality of behavior characteristic clusters; Determining the plurality of behavior feature clusters as the cluster analysis result.
4. The method according to claim 1, characterized in that: The step of performing association analysis on the cluster analysis results according to the target association rule to generate behavior analysis results includes: Clustering and merging any multiple behavioral features in the cluster analysis results to obtain a cross-dataset; The cross data set is mined according to the target association rule to obtain the behavior analysis result.
5. The method according to claim 1, characterized in that The step of performing association analysis on the cluster analysis results according to the target association rule to generate behavior analysis results includes: For any behavior feature cluster in the cluster analysis result, extract all behavior data of the behavior feature cluster; The associations among all the behavior data are mined according to the target association rules to obtain the behavior analysis results.
6. The method according to claim 1, characterized in that When the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, outputting warning information includes: In the case where the behavior analysis result indicates the presence of an institution or practitioner with abnormal behavior, generating early warning information based on the behavior analysis result; According to the warning information, determining a target warning strategy corresponding to the warning information from a preset correspondence table, wherein the correspondence table includes a plurality of correspondences between warning levels and warning strategies, wherein the warning level is determined by the abnormal degree of the abnormal behavior, wherein the warning strategy is used to indicate a warning path and / or a warning object; According to the target early warning strategy, the early warning information is sent to the management end.
7. A data analysis device, characterized in that: The device comprises: An acquisition module, used to acquire behavior data of employees of each institution reported by each institution, wherein the behavior data is used to perform abnormal analysis on historical behaviors of employees and institutions corresponding to the behavior data; A preprocessing module, used for performing data cleaning and data format conversion on the behavior data and storing the data in a blank data set to obtain a behavior data set, wherein the behavior data set includes a plurality of the behavior data; A mining module, used for mining the associations between the behavior data in the behavior data set by using an association rule mining algorithm to obtain target association rules; A clustering module, used for performing cluster analysis on a plurality of the behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result; A generating module, used for performing association analysis on the clustering analysis results according to the target association rule to generate behavior analysis results; The output module is used to output warning information when the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, and the warning information is used to indicate that the institution or the practitioner has abnormal behavior.
8. A data analysis system, characterized in that: The system comprises: A data collection module, used to obtain behavioral data of employees of each institution reported by each institution, wherein the behavioral data is used to perform abnormal analysis on historical behaviors of employees and institutions corresponding to the behavioral data; A data processing module, used for cleaning and converting the behavior data into a data format, and storing the data in a blank data set to obtain a behavior data set, wherein the behavior data set includes a plurality of the behavior data; The data analysis module uses an association rule mining algorithm to mine the associations between the behavior data in the behavior data set to obtain a target association rule; performs cluster analysis on a number of the behavior data in the behavior data set according to a cluster analysis algorithm to obtain a cluster analysis result; performs association analysis on the cluster analysis result according to the target association rule to generate a behavior analysis result; The early warning module outputs early warning information when the behavior analysis result indicates that there is an institution or practitioner with abnormal behavior, wherein the early warning information is used to indicate that the institution or the practitioner has abnormal behavior.
9. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method according to any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 6.
11. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Abnormal user detection method and device, equipment and storage medium
CN114416916A
Abnormality detection method and device, equipment and storage medium
CN115913710A
Data processing method and device, computer equipment and storage medium
CN116861272A
Log association analysis method and device, electronic equipment, chip and medium
CN117573410A
Audit data analysis method and system based on big data
CN117786223A
Cited By
Data association analysis method and system for network security
CN119720187A
