A risk rule extraction method and a risk rule extraction device
By processing sample data through automatic encoding and association rule algorithms, multivariate risk rules are constructed, which solves the problem that univariate rules in existing technologies cannot adapt to diverse risk control businesses, and achieves more efficient risk detection and interpretation support.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-11-11
- Publication Date
- 2026-03-24
AI Technical Summary
Existing technologies can only extract risk rules from single variables, which cannot adapt to diverse risk control business scenarios, and their filtering capabilities are insufficient to meet complex risk detection needs.
The sample data is processed using autocoding and association rule algorithms to identify multiple high-risk and highly correlated variables, construct multivariate risk rules, filter important variables using autocoding algorithms and identify multiple strongly correlated variables using association rule algorithms to generate multivariate risk rules.
It can extract multivariate risk rules covering various risk scenarios, improve the filtering capability of risk detection, meet the complex risk control business needs, and provide the basis for interpreting risk detection results.
Smart Images

Figure CN112785112B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] Embodiments of the present application relate to the field of computer data mining, and in particular, to a risk rule extraction method and device. BACKGROUND
[0002] At present, risk detection needs to be performed in many business scenarios, such as financial expense exception detection, network intrusion detection, credit card fraud detection, health risk detection, product defect detection, and the like. Specifically, after inputting sample data, the known rules are used to identify the abnormality of the input sample data, and output the abnormal sample data that does not conform to the known rules. In addition, the abnormal sample data can be learned to generate a single-variable risk rule, that is, a rule caused by a variable.
[0003] In the prior art, the known rules of a certain business are used to identify risks, which can only face a single business scenario and cannot be flexibly applied to other business scenarios. In addition, the prior art can only extract single-variable risk rules, and the filtering capability of the single-variable rules is low, which cannot adapt to the diversity of scenarios and cannot meet the increasingly complex risk control business requirements. SUMMARY
[0004] Embodiments of the present application provide a risk rule extraction method and device, which can output multi-variable risk rules and meet the requirements of various risk control businesses.
[0005] To achieve the above object, embodiments of the present application adopt the following technical solutions:
[0006] In a first aspect, a risk rule extraction method is provided, which includes: first, obtaining N groups of sample data; any one of the N groups of sample data includes attribute values corresponding to M variables; N is an integer greater than or equal to 2, and M is an integer greater than or equal to 2; an automatic encoding algorithm and an association rule algorithm can also be used to process the N groups of sample data to determine X variables; the risk coefficient of the sample data corresponding to the X variables is greater than a first threshold value, and the association degree of the X variables is greater than a second threshold value; the X variables belong to the M variables, and X is an integer less than or equal to M; finally, a risk rule can be determined according to the X variables, and the risk rule is used to indicate conditions met by the X variables to cause a risk event.
[0007] The embodiment of the present application provides a risk rule extraction method, first, N groups of sample data are acquired; wherein one group of sample data comprises attribute values corresponding to M variables. Subsequently, the N groups of sample data can also be processed by using an automatic encoding (AE) algorithm and an association rule algorithm, X variables with high risk and high correlation are determined, and finally, a risk rule is determined according to the X variables, wherein the risk rule is used to indicate conditions that the X variables meet to cause a risk event. In the method provided by the embodiment of the present application, the sample data with high risk (i.e., a risk coefficient greater than a first threshold) can be determined by learning the sample data according to the automatic encoding algorithm, and the multiple variables with strong correlation can also be determined by learning the sample data with high risk by using the association rule algorithm, and the variables with strong correlation determined according to the sample data with high risk can cause a risk, so that finally, a multivariate risk rule composed of the multiple variables with strong correlation can be output. By extracting the risk rule, a basis can be provided for the explanation of a risk detection result. Meanwhile, the embodiment of the present application can extract a multivariate risk rule, can cover various risk scenarios, has higher filtering capability compared with a single variable risk rule, and can meet increasingly complex risk control business requirements.
[0008] With reference to the first aspect, the N groups of sample data are processed by using the association rule algorithm and the automatic encoding algorithm to determine the X variables, comprising:
[0009] First, the importance coefficients of the M variables can be determined according to the weight coefficients of the neurons of the AE model, and T variables with importance coefficients greater than a third threshold are screened out, i.e., important variables; the S groups of sample data with risk coefficients greater than the first threshold in the N groups of sample data can be determined by using the automatic encoding algorithm, and the S groups of sample data only comprise attribute values corresponding to the T important variables; the S groups of sample data are processed by using the association rule algorithm to determine X variables with correlation greater than a second threshold in the T variables.
[0010] In the method provided by the embodiment of the present application, the input sample data can be learned by using the AE model to obtain sample data with high risk, and non-important variables in the sample data can be removed. In addition, the output of the AE model can be processed by using the association rule algorithm to determine multiple strongly correlated high-risk variables, so that a multivariate risk rule can be created.
[0011] With reference to the first possible implementation manner of the first aspect, in the second possible implementation manner of the first aspect, the S groups of sample data with risk coefficients greater than the first threshold in the N groups of sample data are determined by using the automatic encoding algorithm, comprising:
[0012] For each of the N groups of sample data, the sample data is encoded to obtain a first vector, the first vector including K components, K being an integer less than or equal to M; the first vector is decoded to obtain reconstructed sample data; an error score between the sample data and the reconstructed sample data is calculated; S groups of sample data with error scores greater than a first threshold are filtered, only retaining attribute values corresponding to T variables, to obtain the S groups of sample data.
[0013] In the embodiments of the present application, the sample data can be encoded and then decoded to obtain reconstructed sample data. The greater the error between the reconstructed sample data and the initial sample data, the higher the risk of the sample data.
[0014] In combination with the first or second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the S groups of sample data are processed using an association rule algorithm to determine X variables of the T variables with an association degree greater than a third threshold, including: a first matrix is determined according to the S groups of sample data; the first matrix has S rows and T columns, an element in the i th row and the j th column of the first matrix is a first value or a second value, the first value indicating that an attribute value corresponding to the j th variable in the i th group of sample data is abnormal, the second value indicating that the attribute value corresponding to the j th variable in the i th group of sample data is normal, the i th group of sample data being the i th group of sample data in the S groups of sample data, the j th variable being the j th variable of the T variables, i∈[1, 2, …, S], j∈[1, 2, …, T]; the first matrix is further processed using the association rule algorithm to determine the X variables with the association degree greater than the second threshold.
[0015] In the embodiments of the present application, after determining the high-risk sample data, the bar value detection method can be used to detect the high-risk sample data to determine whether each variable in each sample data is abnormal, to obtain a Boolean matrix. Thus, the association rule algorithm can be used to process the Boolean matrix (for example, the first matrix described in the embodiments of the present application) to obtain a plurality of high-risk variables with strong association.
[0016] In combination with the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the X variables are obtained by processing the first matrix using the association rule algorithm, including:
[0017] at least one candidate item set is determined according to the first matrix; the attribute of the candidate item set is any one or more of the T variables, and the value of the candidate item set is the first value or the second value; the X variables with the association degree greater than the second threshold are determined according to the at least one candidate item set.
[0018] In the embodiments of the present application, a possible implementation of determining strong correlation high-risk variables according to association rules is provided. The higher the support degree (i.e., the association degree) of the candidate item set is, the higher the probability of the variables in the candidate item set appearing simultaneously is. Since the Boolean matrix embodies the distribution of high-risk important variables, processing the first matrix by using the association rule algorithm can determine the probability of multiple high-risk variables appearing simultaneously. The greater the probability of multiple high-risk variables appearing simultaneously is, the higher the association degree between the high-risk variables is, and thus the risk rule of multiple variables can be determined.
[0019] With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, determining X variables with the association degree greater than the second threshold according to the at least one candidate item set comprises:
[0020] The X variables satisfy:
[0021] wherein Q represents the number of target rows in the first matrix, the elements on the X target columns in the target rows are all the first numerical value, the X target columns correspond to the X variables, p1 is the second threshold.
[0022] The embodiments of the present application provide an implementation of determining multiple strong correlation high-risk variables by using the association rule algorithm without weight.
[0023] With reference to the fourth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, determining X variables with the association degree greater than the second threshold according to the at least one candidate item set comprises:
[0024] The X variables satisfy:
[0025] wherein Q represents the number of target rows in the first matrix, the elements on the X target columns in the target rows are all the first numerical value, the X target columns correspond to the X variables, p2 is the second threshold, y t represents the error score of the sample data corresponding to the tth target row in the Q target rows, and Z represents the sum of the error scores of the S groups of sample data.
[0026] In a second aspect, an apparatus is disclosed, comprising:
[0027] a data acquisition module configured to acquire N groups of sample data; any one of the N groups of sample data comprises attribute values corresponding to M variables; N is an integer greater than or equal to 2, and M is an integer greater than or equal to 2;
[0028] The risk rule extraction module is configured to determine X variables by using an auto-encoding algorithm and an association rule algorithm to process N groups of sample data, wherein a risk coefficient of the X variables corresponding to the sample data is greater than a first threshold value, and an association degree of the X variables is greater than a second threshold value; the X variables belong to M variables, and X is an integer less than or equal to M.
[0029] The risk rule extraction module is further configured to determine a risk rule according to the X variables, wherein the risk rule is used to indicate a condition that is satisfied by the X variables to cause a risk event.
[0030] With reference to the second aspect, in a first possible implementation manner of the second aspect, the risk rule extraction module is specifically configured to:
[0031] determine T variables with an importance coefficient greater than a third threshold value in the M variables; determine S groups of sample data with a risk coefficient greater than the first threshold value in the N groups of sample data by using the auto-encoding algorithm, and wherein the S groups of sample data only include attribute values corresponding to the T important variables; and determine X variables with an association degree greater than the second threshold value in the T variables by using the association rule algorithm to process the S groups of sample data.
[0032] With reference to the first possible implementation manner of the second aspect, in a second possible implementation manner of the second aspect, the risk rule extraction module is specifically configured to:
[0033] encode the sample data to obtain a first vector for each group of sample data in the N groups of sample data, wherein the first vector includes K components, and K is an integer less than or equal to M; decode the first vector to obtain reconstructed sample data; calculate an error score between the sample data and the reconstructed sample data; and filter S groups of sample data with an error score greater than a first threshold value, and only retain attribute values corresponding to the T variables to obtain the S groups of sample data.
[0034] With reference to the first or second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the risk rule extraction module is specifically configured to determine a first matrix according to the S groups of sample data, wherein a row number of the first matrix is S, a column number of the first matrix is T, an element in an i th row and a j th column of the first matrix is a first numerical value or a second numerical value, the first numerical value indicates that an attribute value corresponding to a j th variable in an i th group of sample data is abnormal, the second numerical value indicates that the attribute value corresponding to the j th variable in the i th group of sample data is normal, the i th group of sample data is an i th group of sample data in the S groups of sample data, the j th variable is a j th variable in the T variables, i ∈ [1, 2, …, S], and j ∈ [1, 2, …, T]; and determine X variables with an association degree greater than a second threshold value by using the association rule algorithm to process the first matrix.
[0035] In conjunction with the third possible implementation of the second aspect, in the fourth possible implementation of the second aspect, the risk rule extraction module is specifically used to determine at least one candidate set based on the first matrix; the attribute of the candidate set is any one or more of T variables, and the value of the candidate set is a first value or a second value;
[0036] Based on at least one candidate set, identify X variables whose correlation is greater than a second threshold.
[0037] Combining the fourth possible implementation of the second aspect, in the fifth possible implementation of the second aspect,
[0038] X variables satisfy:
[0039] Where Q represents the number of target rows in the first matrix, the elements in the X target columns of the target row are all the first values, the X target columns correspond to X variables, and p1 is the second threshold.
[0040] Combining the fourth possible implementation of the second aspect, in the sixth possible implementation of the second aspect,
[0041] X variables satisfy:
[0042] Where Q represents the number of target rows in the first matrix, the elements in the X target columns of the target row are all the first values, the X target columns correspond to X variables, p2 is the second threshold, and y t Z represents the error score of the sample data corresponding to the t-th target row among the Q target rows, and Z represents the sum of the error scores of the S groups of sample data.
[0043] Thirdly, this application provides an apparatus comprising at least one processor and a memory, the at least one processor being coupled to the memory; the memory being for storing a computer program; and at least one processor being for executing the computer program stored in the memory, such that the apparatus performs the method as described in the first aspect or any possible implementation thereof.
[0044] The device can be a terminal device or a server, etc. Terminal devices include, but are not limited to, smartphones, in-vehicle devices (e.g., autonomous driving equipment), personal computers, artificial intelligence devices, tablets, personal digital assistants, smart wearable devices (e.g., smartwatches or bracelets, smart glasses), smart voice devices (e.g., smart speakers), virtual reality / mixed reality / augmented display devices, or network access devices (e.g., gateways), etc. Servers can include storage servers or computing servers, etc.
[0045] Fourthly, this application discloses a computer-readable storage medium, comprising: instructions stored in the computer-readable storage medium; and, when the computer-readable storage medium is operated on the apparatus described in the second aspect and any implementation thereof, or the third aspect, causing the apparatus to perform the method described in the first aspect and any implementation thereof.
[0046] Fifthly, this application provides a chip including an interface and a processor, the processor being configured to acquire a computer program through the interface and implement the method described in the first aspect or any possible implementation thereof.
[0047] Sixthly, this application provides a chip including a plurality of circuit modules, the plurality of circuit modules being used to implement the method described in the first aspect or any possible implementation of the first aspect. In some implementations, the plurality of circuit modules, together with a software program, implement the method described in the first aspect or any possible implementation of the first aspect. Attached Figure Description
[0048] Figure 1 Structural block diagram of the device provided in the embodiments of this application;
[0049] Figure 2 This is a schematic diagram of the functional modules provided in the embodiments of this application;
[0050] Figure 3 A flowchart illustrating the risk rule extraction method provided in this application embodiment;
[0051] Figure 4 A schematic diagram of the bar value detection method provided in the embodiments of this application;
[0052] Figure 5 Another flowchart illustrating the risk rule extraction method provided in this application embodiment;
[0053] Figure 6 This is a schematic diagram of the data processing flow provided in the embodiments of this application;
[0054] Figure 7 A simulation diagram of a Boolean matrix provided in an embodiment of this application;
[0055] Figure 8 Another structural block diagram of the device provided in the embodiments of this application;
[0056] Figure 9 Another structural block diagram of the device provided in the embodiments of this application. Detailed Implementation
[0057] The technical solutions in this application will now be described with reference to the accompanying drawings.
[0058] First, the terminology used in the embodiments of this application will be explained:
[0059] (1) Risk detection
[0060] Risk detection refers to identifying "outliers" in sample data through data mining techniques. Outliers are data that deviates significantly from other data in the sample data set and can also be called outliers.
[0061] (2) Risk Rules
[0062] Risk rules are used to indicate the conditions that certain variables must meet to cause a risk event. For example, a risk rule might be: the "amount" in an expense report is greater than 3000; that is, the condition for the "amount" to cause an expense report to be abnormal is that it must be greater than 3000. Typically, risk rules can be obtained by learning from "abnormal sample data."
[0063] Univariate risk rules are the conditions that a risk event must be satisfied for a single variable to occur. Multivariate risk rules are the conditions that a risk event must be satisfied for multiple variables to occur.
[0064] (3) Unsupervised learning
[0065] Unsupervised learning refers to machine learning tasks that learn from unlabeled sample data. For example, learning directly from raw data samples.
[0066] (4) Supervised learning
[0067] Supervised learning refers to machine learning tasks that learn from labeled sample data. The labels on the sample data can be added manually.
[0068] (5) Autoencoder (AE) algorithm
[0069] An AE (Advanced Assessment) model is an unsupervised learning model that can be used for risk detection. Specifically, sample data is input into the AE model, which learns from the input data. The output of the AE model is the deviation between the learned sample data and the input sample data. Based on this deviation, it can be determined whether the sample data is high-risk, thus achieving the purpose of risk detection.
[0070] (6) Association rule algorithm
[0071] Association rule algorithms can uncover the relationships between variables. For example, there is a 10% chance that a customer who buys shoes will also buy socks; there is a 60% chance that a customer who buys bread will also buy milk.
[0072] Specifically, association rules can be measured using support. Support represents the probability that multiple variables appear simultaneously in a set of sample data; a higher support indicates a stronger association between the variables. For example, consider the probability that a customer buys both product A and product B in all transaction records. If out of 5 transaction records, 2 records show a customer buying both product A and product B, then the support for this rule ("buying both product A and product B") is 2 / 5 = 0.4.
[0073] Furthermore, a set containing one or more variables can be called an itemset. A frequent itemset is an itemset whose support is greater than a preset threshold. For example, an itemset may include {product A, product B}. Assuming the preset threshold is 0.3, and the support of {product A, product B} is 0.4, then {product A, product B} is a frequent itemset.
[0074] (7) Itemset
[0075] When calculating association rules, the calculation is performed at the itemset level. An itemset is a collection that includes "attributes and values," where "attributes" can be the variables described in the embodiments of this application, and "values" can be the attribute values of the variables.
[0076] First, candidate itemsets are determined based on the sample data. Then, itemsets with support greater than a preset threshold are selected, i.e., frequent itemsets. Frequent itemsets can also be used to identify strongly correlated variables; that is, multiple variables in a frequent itemset are strongly correlated, and the support between strongly correlated variables is greater than a preset threshold.
[0077] (8) Boolean matrix
[0078] A Boolean matrix is a matrix whose elements are either 0 or 1; it can also be called a 0-1 matrix.
[0079] Existing technologies can only extract univariate risk rules, not multivariate risk rules, and cannot adapt to the diversity of data samples, resulting in low efficiency.
[0080] This application provides a risk rule extraction method. First, N sets of sample data are acquired; each set of sample data includes attribute values corresponding to M variables. Then, an autoencoding (AE) algorithm and an association rule algorithm are used to process the N sets of sample data to determine X high-risk variables with strong correlation. Finally, risk rules are determined based on these X variables, indicating the conditions that the X variables must meet to cause a risk event. In the method provided by this application, the autoencoding algorithm can be used to learn from the sample data to determine high-risk sample data (i.e., risk coefficients greater than a first threshold). The association rule algorithm can also be used to learn from the high-risk sample data to determine the aforementioned strongly correlated variables. Since the strongly correlated variables determined from the high-risk sample data may lead to risk, a multivariate risk rule composed of strongly correlated variables can be output. By extracting risk rules, a basis for interpreting risk detection results can be provided. Furthermore, this application can extract multivariate risk rules, covering various risk scenarios. Compared to univariate risk rules, it has higher filtering capabilities and can meet the increasingly complex risk control business needs.
[0081] The method provided in this application embodiment is applicable to general-purpose computers. The aforementioned computer can... Figure 1 The computer 10 in the middle is used to implement this. Figure 1 The diagram shown is a hardware structure schematic of a computer 10 provided in an embodiment of this application. The computer 10 includes a processor 101, a memory 102, and at least one network interface. Figure 1 (This is merely an example illustration using network interface 103 as an example only.) The processor 101, memory 102, and network interface 103 are interconnected.
[0082] The processor 101 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.
[0083] Network interface 103 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.
[0084] The memory 102 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently or be connected to the processor. The memory may also be integrated with the processor.
[0085] The memory 102 stores computer execution instructions for implementing the scheme of this application, and the processor 101 controls the execution. The processor 101 executes the computer execution instructions stored in the memory 102 to implement the intent processing method provided in the following embodiments of this application.
[0086] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.
[0087] In a specific implementation, as one embodiment, the processor 101 may include one or more CPUs, for example... Figure 1 CPU0 and CPU1 in the CPU.
[0088] In a specific implementation, as one example, computer 10 may include multiple processors, for example... Figure 1 Processors 101 and 106 are included. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).
[0089] In a specific implementation, as one embodiment, the computer 10 may further include an output device 104 and an input device 105. The output device 104 communicates with the processor 101 and can display information in various ways. For example, the output device 104 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device 105 communicates with the processor 101 and can receive user input in various ways. For example, the input device 105 may be a mouse, keyboard, touchscreen device, or sensing device, etc.
[0090] The computer 10 described above can be a general-purpose device or a special-purpose device. In specific implementations, the computer 10 can be a desktop computer, a portable computer, a network server, a handheld computer (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device, or something similar. Figure 1 Devices with similar structures. This application does not limit the type of computer 10 to any particular embodiment.
[0091] It should be noted that computer 10 can be a complete machine, a functional component or part that implements the functions of a computer, or a chip.
[0092] Figure 2 The various functional modules shown are used to implement the methods provided in the embodiments of this application. Specifically, the processor 101 can call code in the memory 102 to execute... Figure 2 The functions of each module are shown.
[0093] refer to Figure 2 Data acquisition module 30 is used to acquire sample data;
[0094] The risk detection module 40 is used to perform risk detection on the sample data acquired by the data acquisition module 30 and identify high-risk sample data.
[0095] Risk rule extraction module 50 is used to automatically extract risk rules;
[0096] The risk rule storage module 60 is used to store extracted risk rules or manually added risk rules in the memory 102;
[0097] The data management module 70 is used to divide the data samples into sample data and distribute the labeled sample data and unlabeled sample data in the memory 102.
[0098] This application provides a risk rule extraction method, such as... Figure 3 As shown, the method includes the following steps:
[0099] Step 301: Obtain N sets of sample data; any one set of sample data in the N sets includes attribute values corresponding to M variables.
[0100] The N sets of sample data are training data used to extract multivariate risk rules. These N sets of sample data can be used to learn multivariate risk rules. For example, the sample data can be expense reimbursement data; by learning from this data, rules indicating potential risks in expense reimbursement processes can be obtained.
[0101] A set of sample data includes attribute values for multiple variables. Variables can be "time," "amount," etc., and attribute values are the specific values of the variables. For example, the attribute value for "amount" can be any number, and the attribute value for "time" can be "year**month**day." Furthermore, N is an integer greater than or equal to 2, and M is an integer greater than or equal to 2.
[0102] Step 302: Process the N sets of sample data using an auto-coding algorithm and an association rule algorithm to determine X variables; the risk coefficient of the sample data corresponding to the X variables is greater than a first threshold and the association degree of the X variables is greater than a second threshold.
[0103] Wherein, the X variables belong to the M variables, and X is an integer less than or equal to M. The risk coefficient is used to describe the risk of the sample data; for example, it can be the error risk of reconstructing the sample coefficients. The larger the error score, the greater the risk of the sample data. The correlation degree is used to describe the correlation between different variables. The correlation degree can be the support degree described in the embodiments of this application. The higher the correlation degree between different variables, the higher the probability that different variables appear in the same sample data.
[0104] In this embodiment, the initial inbound and outbound sample data can be learned to output high-risk sample data. Furthermore, association rules and multiple strongly correlated variables can be used to determine risk rules based on multiple variables, thereby providing support for the risk detection results.
[0105] One possible implementation involves identifying multiple high-risk, strongly correlated variables from the aforementioned M variables through the following three steps:
[0106] Step S1: Use an auto-coding algorithm to determine S groups of sample data in the N groups of sample data whose risk coefficient is greater than the first threshold.
[0107] Specifically, the AE model can be used to screen high-risk sample data. For an explanation of how the AE model works, please refer to [link to documentation / reference]. Figure 3Specifically, the encoder first encodes the input data, and the decoder then decodes the encoder's output. The decoder outputs the reconstructed data from the original data. An error score can also be calculated between the original data and the reconstructed data. The larger the error score, the greater the difficulty in reconstructing the original sample data, i.e., the higher the risk of the original sample data. The error score can be the risk coefficient described in the embodiments of this application, which can characterize the risk of sample data; a larger error score indicates a higher risk coefficient. Furthermore, the threshold corresponding to the error score can be a first threshold corresponding to the risk coefficient, and the S groups of sample data with a risk coefficient greater than the first threshold can be the S groups of sample data with an error score greater than the first threshold.
[0108] For example, firstly, for each group of sample data in the N groups of sample data, the sample data is encoded to obtain a first vector, which includes K components, where K is an integer less than or equal to M;
[0109] The first vector can also be decoded to obtain reconstructed sample data; the error score between the sample data and the reconstructed sample data can be calculated.
[0110] Finally, S groups of sample data with error scores greater than the first threshold can be selected.
[0111] It should be noted that the first threshold is a threshold value set based on risk. Sample data with an error score greater than the first threshold are considered high-risk sample data. Subsequently, high-risk sample data can be used to learn and determine strong association rules between high-risk variables.
[0112] Step S2: Filter the S groups of sample data obtained in step S1 to remove unimportant variables from each sample data set. Each group of sample data in the resulting S groups after filtering includes only the attribute values corresponding to T important variables. The T important variables are the T variables among the M variables whose importance coefficient is greater than a third threshold.
[0113] It should be noted that the AE model is a multi-layer neural network model. Based on the AE model, not only can high-risk sample data be identified, but also the importance of variables can be determined, thereby filtering out several more important variables.
[0114] Specifically, the importance of a variable is calculated based on the weight values of the neurons in the first two layers of the AE model. The importance coefficient of the i-th variable satisfies the following formula (1):
[0115]
[0116] Among them, w 1 ij| represents the absolute value of the weights of neurons in the first layer of the network, and max represents taking the maximum value and performing normalization.
[0117] According to the above formula (1), the important variables among the M variables can be determined, that is, the T variables whose importance coefficient is greater than (or equal to) the third threshold. Subsequently, the attribute values of the non-important variables (i.e., variables whose importance coefficient is less than the third threshold) in the S groups of high-risk sample data can be removed, and only the attribute values of the T important variables can be retained to obtain the sample data.
[0118] Step S3: Process the S groups of sample data using an association rule algorithm to determine the X variables among the T variables whose association degree is greater than the second threshold.
[0119] Specifically, a first matrix can be determined based on the S sets of sample data. The first matrix has S rows and T columns. The element in the i-th row and j-th column of the first matrix is a first value or a second value. The first value indicates that the attribute value corresponding to the j-th variable in the i-th set of sample data is abnormal, and the second value indicates that the attribute value corresponding to the j-th variable in the i-th set of sample data is normal. The i-th set of sample data is the i-th set of sample data in the S sets of sample data, and the j-th variable is the j-th variable in the T variables, i∈[1, 2, ..., S], j∈[1, 2, ..., T].
[0120] The first matrix can also be processed using an association rule algorithm to determine the X variables whose association degree is greater than the second threshold. Here, the association degree is the support degree described in the embodiments of this application.
[0121] For example, you can set a univariate Bar value to determine whether the attribute value of a certain variable in a set of sample data is abnormal, based on a univariate risk detection method. See the example below. Figure 4 The horizontal axis represents a single important variable x, and the vertical axis represents the attribute value P(x) of the variable. The dashed line in the graph represents a Bar value. If P(x) is below the Bar value, then the attribute value P(x) of the variable is an outlier. For example, if the attribute value of the j-th variable in the i-th sample data is greater than the Bar value of the j-th variable, then the attribute value of the j-th variable in the i-th sample data is outlier, and the element in the i-th row and j-th column of the first matrix is the first value "1"; if the attribute value of the j-th variable in the i-th sample data is less than the Bar value of the j-th variable, then the attribute value of the j-th variable in the i-th sample data is normal, and the element in the i-th row and j-th column of the first matrix is the first value "0".
[0122] It should be noted that, Figure 4The detection method shown is applicable to variables with multimodal distributions and can detect outliers of intermediate size, while traditional statistical methods (such as Z-score and Box-Plot) typically only detect outliers at the extremes. Furthermore, by adjusting the Bar value, the proportion of outliers can be adjusted, achieving simple and efficient variable detection.
[0123] pass Figure 4 After the detection method shown, an S-row, T-column Boolean matrix can be obtained, such as the first matrix described in the embodiments of this application. This matrix can reflect whether the attribute values of each important variable in each group of sample data are abnormal. The following matrix (1) is a possible implementation of the first matrix:
[0124]
[0125] One possible implementation, wherein the method of processing the first matrix using an association rule algorithm to obtain the X variables, includes: firstly determining at least one candidate set based on the first matrix; the candidate set includes any one or more variables from the T variables, and the value of the candidate set is either the first value or the second value.
[0126] The X variables whose correlation is greater than a second threshold can also be determined based on the at least one candidate set. The second threshold may be a support threshold.
[0127] Specifically, determining the X variables with a correlation greater than the second threshold based on the at least one candidate set includes:
[0128] The X variables satisfy:
[0129] Where Q represents the number of target rows in the first matrix, the elements in the X target columns of the target row are all first values, the X target columns correspond to the X variables, and p1 is the second threshold.
[0130] It should be noted that the X target columns correspond one-to-one with the X variables, that is, the element in the i-th column of the first matrix represents whether the i-th variable in each sample data is abnormal. The i-th column is associated with the i-th variable; for example, the first column of the first matrix is associated with the variable "amount".
[0131] If all X target columns in a row of the first matrix are the first value, it means that all X variables in the sample data corresponding to that row are abnormal.
[0132] For example, suppose the candidate options set includes three variables: {ABC}. Suppose variables A, B, and C correspond to the i-th, j-th, and t-th columns, respectively. In the first matrix, the elements in the i-th, j-th, and t-th columns of the Q rows are all the first value. That is, in the S sample data, variables A, B, and C appear simultaneously in Q samples, and variables A, B, and C are all anomalous. Then the support of this candidate options set is Q / S.
[0133] It should be noted that, assuming the element in the i-th row and j-th column of the first matrix is the first value, such as "1", it represents that the j-th variable in the i-th sample data is abnormal. Taking variable A as an example, assuming the first column element in the first matrix represents the state of variable A in each sample data, and assuming the element in the second row and first column is "1", it means that variable A in the second sample data out of the S sample data is abnormal.
[0134] If the support of the candidate set is greater than the second threshold, then the variables in the candidate set are output; that is, the identified high-risk, strongly correlated variables are included in the candidate set. If the support of the candidate set is equal to the second threshold, then the variables in the candidate set are output; that is, the identified high-risk, strongly correlated variables are included in the candidate set. Conversely, if the support of the candidate set is less than the second threshold, then the variables in the candidate set cannot be included in the final output high-risk, strongly correlated variables.
[0135] This application applies association rule algorithm to risk rule extraction. By calculating the correlation between multiple important high-risk variables, it outputs multiple strongly correlated important high-risk variables.
[0136] 303. Determine risk rules based on the X variables, wherein the risk rules are used to indicate the conditions that the X variables must satisfy to cause a risk event.
[0137] In practice, after identifying several strongly correlated, important, high-risk variables, risk rules can be created based on these variables and their corresponding bar values. Specifically, "risk rules are used to indicate the conditions that at least two variables must satisfy to cause a risk event."
[0138] For example, the strongly correlated high-risk variables determined in step 302 are "number of applications" and "amount". The bar value corresponding to the variable "number of applications" is 10, and the bar value corresponding to the variable "amount" is 3000. Therefore, the risk rule is: when the "number of applications" in the sample data is greater than 10 and the "amount" is greater than 3000, the sample data is abnormal sample data.
[0139] It should be noted that in the method described above, when processing the first matrix using the association rule algorithm, the weight coefficients of the sample data are all 1. For example, suppose the candidate set includes three variables: {ABC}. In the S sample data, X samples simultaneously contain variables A, B, and C, and variables A, B, and C are all anomalous. The weight values of these S sample data are all 1.
[0140] In one possible implementation, a weighted association rule algorithm can be used to determine multiple strongly associated high-risk variables, wherein the weight value of the sample data is an example of the sample data, and the step of determining the X variables with an association degree greater than the second threshold based on the at least one candidate set includes:
[0141] The X variables satisfy:
[0142] Where Q represents the number of target rows in the first matrix, the elements of the X target columns in the target rows are all first values, the X target columns correspond to the X variables, p2 is the second threshold, and y t Z represents the error score of the sample data corresponding to the t-th target row among the Q target rows, and Z represents the sum of the error scores of the S groups of sample data.
[0143] For example, suppose the candidate set includes three variables {ABC}. In the S sample data, X samples simultaneously contain variables A, B, and C, and all variables A, B, and C are anomalous. The risk scores of these S sample data are 0.35, 0.4, and 0.65, respectively. The sum of the error scores of the S sample data is 11.28. Then, the support of the candidate set {ABC} is (0.32 + 0.4 + 0.65) / 11.28 ≈ 0.81. Assuming this is greater than the preset threshold p2, the output strongly correlated high-risk variables could be ABC. Furthermore, a risk rule can be defined: when A is greater than y1, B is greater than y2, and C is greater than y3, the sample data is anomalous. Here, y1 is the bar value corresponding to variable A, y2 is the bar value corresponding to variable B, and y3 is the bar value corresponding to variable C.
[0144] It should be noted that when using a weighted association rule algorithm to identify multiple strongly associated high-risk variables, the support threshold can be reset. For example, p2 is different from p1; p2 is the support threshold using a weighted association rule algorithm, while p1 is the support threshold using an unweighted association rule algorithm.
[0145] The risk rule extraction method described in this application is illustrated below with specific examples. Figure 5 As shown, the method includes the following steps:
[0146] 501. Prepare data.
[0147] Taking company expense reimbursement data as an example, this application describes the risk rule extraction method provided in its embodiments. The accurate data in step 501 includes training data and validation data. The training data is used to extract risk rules, and the validation data is used to verify the extracted risk rules. The data includes attribute values of multiple variables, such as "amount," "time," and "region."
[0148] In practice, reimbursement data for a certain period of time (e.g., reimbursement data within the past year) can be extracted from the database as training data, and reimbursement data for a period of time after that period can be extracted as validation data.
[0149] 502. Preprocess the data.
[0150] Step 502 requires preprocessing the distributions of the training and validation data, but the preprocessing for the two types of data is the same.
[0151] Specifically, refer to Figure 6 Data preprocessing includes the following steps:
[0152] 5021. Delete invalid data.
[0153] Deleting invalid data includes deleting duplicate data, deleting abnormal data, and deleting attribute values of invalid variables.
[0154] First, if a data item is duplicated, the duplicate data can be deleted to ensure the validity of the data.
[0155] Additionally, if most of the attribute values for a single expense report are missing, then that expense report is considered invalid and can be deleted.
[0156] Finally, variables that do not affect the risk rule extraction results can be deleted. It should be noted that some variables in the reimbursement data are meaningless or do not affect the final result. To reduce the data volume, the attribute values of these variables can be deleted. For example, the variable "Reimbursement Form Number" does not affect the risk rule extraction result and is therefore meaningless; its corresponding attribute value can be deleted from the reimbursement data. Similarly, the variable "Whether it is a violation" is a marker variable in the reimbursement data that severely interferes with the risk rule extraction result; its corresponding attribute value should also be deleted.
[0157] 5022. Complete the missing attribute values.
[0158] After deleting certain variables, if some variables still have missing values in the data, these missing values need to be filled in. The methods for filling in missing values differ depending on the type of variable: for numerical variables, use the average of all attribute values of the variable to fill in the missing values; for categorical variables, use the value of the category with the largest number of categories to fill in the missing values.
[0159] 5023. Standardize the data.
[0160] Specifically, the mean of all variables can be shifted to zero, and the variances can be normalized. For categorical variables, each category of each variable is encoded with 0 or 1; for example, if a variable belongs to a certain category, its attribute value is 1, and if it does not belong to that category, its label is 0.
[0161] 503. Train the AE model based on the data processed in step 502.
[0162] It should be noted that the AE model consists of an encoder and a decoder, both of which are neural network structures. Each encoder and decoder contains three layers of neural networks.
[0163] Specifically, an AE model is obtained by training the processed data using an unsupervised learning method. First, an initial AE model is designed. After the initial AE model is constructed, the weight parameters of each layer of the neural network in the AE model can be initialized with random values.
[0164] After the initial AE model is built, we divide the entire dataset into data blocks and input them into the initial AE model to train the weight parameters of the AE model, thus obtaining the final AE model. The training process of the AE model is a process of optimizing the cost function, which is the sum of the mean square error (MSE) of the input and output and the L2 penalty term.
[0165] The L2 penalty term serves to prevent the model from overfitting.
[0166] 504. Use the AE model to identify key variables for high-risk situations.
[0167] Specifically, the verification data is input into the AE model. The output of the AE model is the error score (i.e., the risk coefficient described in the embodiments of this application) for each reimbursement data and the importance coefficient of each variable. The reimbursement data can also be sorted according to the error score to determine the top S reimbursement data with higher errors. Specifically, S reimbursement data with error scores greater than a first threshold can be selected. Furthermore, the AE model can also output the importance coefficient of each variable. The variables in the reimbursement data can be sorted according to the importance coefficient to determine the top T variables with higher importance coefficients. Specifically, T important variables with error scores greater than a third threshold can be selected.
[0168] Step 504 identifies S high-risk reimbursement data points, where each reimbursement data point includes T important variables.
[0169] 505. Based on the AE model, determine the key variables with high risk and generate a Boolean matrix.
[0170] First, based on the Bar value detection method, check whether the values of each of the T important variables in each of the S high-risk reimbursement data are abnormal, and generate an S-row T-column Boolean matrix based on the detection results.
[0171] The principle of the Bar value detection method has been explained in detail above and will not be repeated here. The elements in the Boolean matrix are 0 or 1, where 1 indicates that the value of the corresponding variable in the corresponding reimbursement data is normal, and 0 indicates that the value of the corresponding variable in the corresponding reimbursement data is abnormal.
[0172] Figure 7 The image shows a simulation diagram of Boolean matrices generated from three different datasets. From left to right, the datasets corresponding to these three Boolean matrices are: the 500 expense reports with the highest AE error scores, the 500 expense reports marked as violations, and 500 randomly selected expense reports. (Reference) Figure 7 It can be seen that if the error score detected by the AE model for a reimbursement data point is relatively high, then the probability of its important variables being identified as outliers using the Bar value detection method is also relatively high. Furthermore, reimbursement data points with higher error scores tend to have more outliers. In addition, the Boolean matrix determined based on 500 reimbursement data points with violations also shows a significant number of outliers, considerably higher than the Boolean matrix determined based on random samples. Therefore, the Bar value detection method can effectively detect univariate outliers, and the error score output by the AE model can reflect the risk level of the data.
[0173] When extracting risk rules in the subsequent process, only the leftmost Boolean matrix is used, which is the Boolean matrix generated based on the important high-risk variables output by the AE model.
[0174] 506. Use association rule algorithms to process the first matrix to obtain important high-risk variables with strong associations.
[0175] In step 506, the Boolean matrix generated in step 505 is analyzed using association rules without weights, meaning that the weight coefficients of the frequent itemsets determined by the first matrix are all 1.
[0176] After determining the frequent itemsets, each itemset can be substituted into the above formula (1) to determine the support of each itemset (i.e., the correlation degree described in the embodiments of this application), and the itemsets with support greater than the preset threshold (i.e., the second threshold described in the embodiments of this application, assumed to be 0.3) are output. The variables in the itemsets are strongly correlated variables, and risk rules for multiple variables can be created based on the strongly correlated variables.
[0177] Multivariate risk rules can be determined based on strongly correlated, important, high-risk variables. These risk rules indicate that multiple variables lead to the risk rules. Table 1 below illustrates the multivariate risk rules.
[0178] Table 1
[0179] Number Support Variables included in the rule Number of variables 43 0.564 AB 2 50 0.544 AC 2 73 0.544 ABC 3 132 0.574 ABC 3
[0180] It should be noted that in Table 1, the first column is the risk rule number, the second column is the support of the risk rule calculated from the association rule without weights, the third column is the variables included in the risk rule, and the fourth column is the number of variables included in the risk rule. Risk rules can be generated by referring to Table 1 and combining the bar values of each variable. Referring to Table 1, we can see that two risk rules with two variables and two risk rules with three variables were obtained.
[0181] 507. Use a weighted association rule algorithm to perform a second screening of the association rules determined in step 506.
[0182] In step 507, a weighted association rule algorithm can be used to redetermine multiple strongly associated high-risk variables, thereby redetermining the risk rules. Detailed steps for using the weighted association rule algorithm to determine multiple strongly associated variables are described above and will not be repeated here.
[0183] It should be noted that the weighting coefficients mentioned above are the error scores of the reimbursement data obtained by the AE model. During the secondary screening, the minimum support level can be reset, for example, to 0.7.
[0184] Existing technologies can only extract simple univariate risk rules, failing to summarize risks caused by multiple variables or extract multivariate risk rules. In this application, an association rule algorithm is used to learn from high-risk sample data to identify multiple strongly correlated high-risk variables. Furthermore, by combining the bar values of each variable, multivariate risk rules can be determined, thus solving the problem of the inability to extract and summarize multivariate risk rules and providing a basis for risk detection results.
[0185] When dividing each function into modules according to its corresponding function. Figure 8 A schematic diagram of a possible structure of the device involved in the above embodiments is shown. For example, Figure 8 The apparatus shown can be the computer described in the embodiments of this application, or it can be a component in a computer that implements the above-described method. For example... Figure 8 As shown, the device includes a processing unit 801 and a transceiver unit 802. The processing unit may be one or more processors, and the transceiver unit may be a transceiver.
[0186] Processing unit 801 is used to support computer execution. Figure 3 , Figure 5 The method described herein, and / or other processes used in the techniques described herein. Processing unit 801 may include: a data acquisition module 30 and a risk rule extraction module 50.
[0187] The transceiver unit 802 is used to support communication between the computer and other devices, and / or other processes for the techniques described herein. It may be an interface circuit of the device or a network interface.
[0188] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.
[0189] In one possible implementation, Figure 8 The device shown can also be a chip used in a computer. The chip can be a system-on-a-chip (SOC) or a baseband chip with communication functions, etc.
[0190] For example, when using integrated units, the structural schematic diagram of the device provided in the embodiments of this application is as follows: Figure 9 As shown. In Figure 9The device includes a processing module 901 and a communication module 902. The processing module 901 controls and manages the operation of the device, for example, executing the steps performed by the processing unit described above, and / or other processes for performing the techniques described herein. The communication module 902 executes the steps performed by the transceiver unit described above, supporting interaction between the device and other devices, such as interaction with other terminal devices. Figure 9 As shown, the device may also include a storage module 903, which is used to store the device's program code and data.
[0191] When the processing module 901 is a processor, the communication module 902 is a transceiver, and the storage module 903 is a memory, the device is... Figure 1 The apparatus shown.
[0192] This application provides a computer-readable storage medium storing instructions; the instructions are used to perform actions such as... Figure 3 or Figure 5 The method shown.
[0193] This application provides a computer program product including instructions that, when executed on a device, cause the device to perform the following... Figure 3 or Figure 5 The method shown.
[0194] This application provides a wireless device, comprising: the wireless device storing instructions; when the wireless device is in... Figure 1 , Figure 8 , Figure 9 When the device is operated as shown, the device achieves the following: Figure 3 or Figure 5 The method is shown. The device can be a chip, etc.
[0195] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the database access device can be divided into different functional modules to complete all or part of the functions described above.
[0196] In the several embodiments provided in this application, it should be understood that the disclosed database access apparatus and method can be implemented in other ways. For example, the database access apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection of the database access apparatus or unit may be electrical, mechanical, or other forms.
[0197] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0198] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0199] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, disks, or optical discs.
[0200] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for extracting risk rules, characterized in that, include: Obtain N sets of sample data; Any one of the N sets of sample data includes attribute values corresponding to M variables; N is an integer greater than or equal to 2, and M is an integer greater than or equal to 2; For each of the N groups of sample data, the sample data is encoded to obtain a first vector, which includes K components, where K is an integer less than or equal to M. Decode the first vector to obtain reconstructed sample data; Calculate the error score between the sample data and the reconstructed sample data; S groups of sample data with error scores greater than a first threshold are filtered, retaining only the attribute values corresponding to T variables to obtain the S groups of sample data. The T variables are the T variables among the M variables whose importance coefficients are greater than a third threshold. The S groups of sample data are detected using the Bar value detection method. A first matrix is generated based on the detection results. An association rule algorithm is used to process the first matrix to determine the X variables whose association degree is greater than a second threshold. The first matrix is a Boolean matrix. The X variables belong to the M variables, and X is an integer less than or equal to M. Risk rules are determined based on the X variables and the corresponding Bar values of the X variables. The risk rules are used to indicate the conditions that the X variables must satisfy to cause a risk event.
2. The method according to claim 1, characterized in that, The first matrix has S rows and T columns. The element in the i-th row and j-th column of the first matrix is either a first value or a second value. The first value indicates that the attribute value corresponding to the j-th variable in the i-th group of sample data is abnormal, and the second value indicates that the attribute value corresponding to the j-th variable in the i-th group of sample data is normal. The i-th group of sample data is the i-th group of sample data in the S groups of sample data, and the j-th variable is the j-th variable in the T variables. .
3. The method according to claim 2, characterized in that, The step of processing the first matrix using an association rule algorithm to obtain the X variables includes: At least one candidate set is determined based on the first matrix; the attribute of the candidate set is any one or more of the T variables, and the value of the candidate set is either the first value or the second value; Based on the at least one candidate set, determine the X variables whose correlation is greater than the second threshold.
4. The method according to claim 3, characterized in that, Based on the at least one candidate set, the X variables with a correlation greater than the second threshold are determined, including: The X variables satisfy: ; in, This represents the number of target rows in the first matrix. The elements in the X target columns of each target row are all first values, and these X target columns correspond to the X variables. This is the second threshold.
5. The method according to claim 3, characterized in that, Based on the at least one candidate set, the X variables with a correlation greater than the second threshold are determined, including: The X variables satisfy: ; in, This represents the number of target rows in the first matrix. The elements in the X target columns of each target row are all first values, and these X target columns correspond to the X variables. The second threshold, represent The error score of the sample data corresponding to the t-th target row in the n target rows. This represents the sum of error scores for the S groups of sample data.
6. An apparatus, characterized in that, include: The data acquisition module is used to acquire N sets of sample data; Any one of the N sets of sample data includes attribute values corresponding to M variables; where N is an integer greater than or equal to 2, and M is an integer greater than or equal to 2. The risk rule extraction module is specifically used for, For each of the N groups of sample data, the sample data is encoded to obtain a first vector, which includes K components, where K is an integer less than or equal to M. Decode the first vector to obtain reconstructed sample data; calculate the error score between the sample data and the reconstructed sample data. S groups of sample data with error scores greater than the first threshold are filtered out, and only the attribute values corresponding to T variables are retained to obtain the S groups of sample data. The T variables are the T variables among the M variables whose importance coefficients are greater than the third threshold. The S groups of sample data are detected using the Bar value detection method. A first matrix is generated based on the detection results. An association rule algorithm is used to process the first matrix to determine the X variables whose association degree is greater than a second threshold. The first matrix is a Boolean matrix. The X variables belong to the M variables, and X is an integer less than or equal to M. The risk rule extraction module is further configured to determine risk rules based on the X variables and the Bar values corresponding to the X variables, wherein the risk rules are used to indicate the conditions that the X variables must satisfy to cause a risk event.
7. The apparatus according to claim 6, characterized in that, The first matrix has S rows and T columns. The element in the i-th row and j-th column of the first matrix is a first value or a second value. The first value indicates that the attribute value corresponding to the j-th variable in the i-th group of sample data is abnormal, and the second value indicates that the attribute value corresponding to the j-th variable in the i-th group of sample data is normal. The i-th group of sample data is the i-th group of sample data in the S groups of sample data, and the j-th variable is the j-th variable in the T variables, i∈[1,2,….S], j∈[1,2,….T].
8. The apparatus according to claim 7, characterized in that, The risk rule extraction module is specifically used for, At least one candidate set is determined based on the first matrix; the attribute of the candidate set is any one or more of the T variables, and the value of the candidate set is either the first value or the second value; Based on the at least one candidate set, determine the X variables whose correlation is greater than the second threshold.
9. The apparatus according to claim 8, characterized in that, The risk rule extraction module is specifically used for, The X variables satisfy: ; in, This represents the number of target rows in the first matrix. The elements in the X target columns of each target row are all first values, and these X target columns correspond to the X variables. This is the second threshold.
10. The apparatus according to claim 8, characterized in that, The risk rule extraction module is specifically used for, The X variables satisfy: ; in, This represents the number of target rows in the first matrix. The elements in the X target columns of each target row are all first values, and these X target columns correspond to the X variables. The second threshold, represent The error score of the sample data corresponding to the t-th target row in the n target rows. This represents the sum of error scores for the S groups of sample data.
11. An apparatus, characterized in that, It includes at least one processor and a memory, wherein the at least one processor is coupled to the memory; The memory is used to store computer programs; The at least one processor is configured to execute a computer program stored in the memory to cause the apparatus to perform the method as described in any one of claims 1 to 5.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions that, when executed, implement the method as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Hazardous chemicals transportation risk factor statistical method and analysis method based on big data
CN108694517A
Risk assessment model training method and device, risk assessment method and device and electronic equipment
CN109598414A