Anomaly value detection method and device, electronic equipment and storage medium

Abstract

The application relates to an outlier detection method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a user-inputted to-be-predicted sample; inputting the to-be-predicted sample into an outlier detection model to obtain an outlier of the to-be-predicted sample, wherein the outlier represents an abnormal condition of the to-be-predicted sample; the outlier detection model is obtained through model training based on user-inputted training samples, and the training samples have corresponding labels. The application changes the unsupervised modeling process of the isolation forest into a supervised modeling process, evolves the traditional isolation forest algorithm into an algorithm more suitable for the actual scene of financial risk control, improves the effect of the application level, and overcomes the limitations of the isolation forest algorithm in the financial risk control scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information extraction, and in particular to an outlier detection method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Isolation Forest is one of the traditional algorithms for outlier detection, wherein outlier detection is a process of finding outliers in data, and an outlier is a data point that is significantly different from other data points in a given data set.

[0003] However, since the Isolation Forest algorithm is unsupervised modeling, the use of the Isolation Forest algorithm in the financial risk control scene has limitations.

[0004] At present, there is no effective solution to the above-mentioned limitations of the Isolation Forest algorithm in the financial risk control scene. SUMMARY

[0005] The present application provides an outlier detection method, device, electronic device, and storage medium to solve the limitations of the existing Isolation Forest algorithm in the financial risk control scene.

[0006] In a first aspect, the present application provides an outlier detection method, comprising:

[0007] obtaining a user-inputted prediction sample;

[0008] inputting the prediction sample into an outlier detection model to obtain an outlier of the prediction sample, wherein the outlier represents an abnormal situation of the prediction sample; the outlier detection model is obtained by model training based on a user-inputted training sample, and the training sample has a corresponding label.

[0009] Optionally, the outlier detection model is generated, comprising:

[0010] obtaining a user-inputted training sample, wherein the training sample has a corresponding label;

[0011] training a single decision tree based on the user-inputted training sample and label;

[0012] decomposing the trained single decision tree to obtain a first rule set after decomposition;

[0013] in a case where information entropy of a rule in the first rule set hitting the training sample is less than or equal to a set threshold, the rule is put into a pure entropy queue;

[0014] in a case where information entropy of a rule in the first rule set hitting the training sample is greater than a set threshold, the rule is put into a priority queue;

[0015] compare the number of rules in the priority queue with a preset number to obtain a comparison result;

[0016] in a case where the comparison result is that the number of rules in the priority queue is less than or equal to the preset number, use the forest of trained decision trees as an anomaly value detection model.

[0017] Optionally, the method further comprises:

[0018] in a case where the comparison result is that the number of rules in the priority queue is greater than the preset number, re-extract rules in the priority queue to obtain extracted rules;

[0019] determine a first sub-sample based on the extracted rules, wherein the first sub-sample includes samples covered by leaf nodes in the trained decision trees;

[0020] train a new single decision tree based on the extracted rules and the first sub-sample;

[0021] decompose the trained new single decision tree to obtain a second rule set after decomposition;

[0022] in a case where information entropy of a rule in the second rule set after decomposition hitting the first sub-sample is less than or equal to a set threshold, put the rule into a pure entropy queue;

[0023] in a case where information entropy of a rule in the second rule set after decomposition hitting the first sub-sample is greater than the set threshold, put the rule into the priority queue;

[0024] return to the step of comparing the number of rules in the priority queue with the preset number to obtain the comparison result.

[0025] Optionally, the inputting the to-be-predicted sample into the anomaly value detection model to obtain an anomaly value of the to-be-predicted sample comprises:

[0026] obtain a third rule set in which the to-be-predicted sample hits the pure entropy queue;

[0027] compare an average response rate of all rules in the third rule set hitting the training sample with a training sample average response rate to obtain a comparison result;

[0028] in a case where the comparison result is that the average response rate of the rules in the third rule set hitting the training sample is greater than the training sample average response rate, add a length of the rule to a preset score on the basis of the preset score to update the preset score;

[0029] in a case where the comparison result is that the average response rate of the rules in the third rule set hitting the training sample is less than or equal to the training sample average response rate, subtract the length of the rule from the preset score on the basis of the preset score to update the preset score.

[0030] When comparing the average response rate of all rules in the third rule set that hit the training sample with the average response rate of the training sample, the preset score is determined as the outlier of the sample to be predicted.

[0031] Optionally, before training a single decision tree based on user-input training samples and labels, the method further includes:

[0032] Detect whether the training samples and labels input by the user are compliant;

[0033] Training a single decision tree based on user-input training samples and labels includes:

[0034] If the user-input training sample and its corresponding label are found to be compliant, a single decision tree is trained based on the user-input training sample and label.

[0035] Optionally, the step of decomposing the trained single decision tree to obtain the first set of rules after decomposition includes:

[0036] Starting from the root node of the trained single decision tree, decompose it to obtain the path from the root node to each leaf node of the single decision tree.

[0037] The path from the root node of the single decision tree to each leaf node is determined as multiple rules after the single decision tree is decomposed, and the multiple rules constitute the first rule set.

[0038] Optionally, the method further includes:

[0039] Obtain the decision tree parameters input by the user;

[0040] The training of a single decision tree based on user-input training samples and labels includes:

[0041] Training a single decision tree based on user-input training samples, labels, and decision tree parameters.

[0042] Secondly, this application provides an outlier detection device, the device comprising:

[0043] The sample acquisition unit is used to acquire the sample to be predicted input by the user.

[0044] An outlier acquisition unit is used to input the sample to be predicted into an outlier detection model to obtain outliers of the sample to be predicted, wherein the outliers represent the abnormal situation of the sample to be predicted; the outlier detection model is obtained by training the model based on training samples input by the user, and the training samples have corresponding labels.

[0045] In a third aspect, the present application provides an electronic device, comprising: at least one communication interface; at least one bus connected with the at least one communication interface; at least one processor connected with the at least one bus; at least one memory connected with the at least one bus, wherein the processor executes the method for detecting outliers according to any one of the above aspects through a computer program.

[0046] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions for executing the method for detecting outliers according to any one of the above aspects.

[0047] The above technical solution provided by the embodiments of the present application has the following advantages compared with the prior art: the method provided by the embodiments of the present application obtains a user-inputted to-be-predicted sample; inputs the to-be-predicted sample into an outlier detection model to obtain an outlier of the to-be-predicted sample, wherein the outlier represents an abnormal situation of the to-be-predicted sample; the outlier detection model is obtained through model training based on user-inputted training samples, and the training samples have corresponding labels. The present application changes the unsupervised modeling process of the Isolation Forest into a supervised modeling process, evolves the traditional Isolation Forest algorithm into an algorithm more suitable for the actual scene of financial risk control, improves the effect at the application level, and overcomes the limitations of the Isolation Forest algorithm in the financial risk control scene. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0050] One or more embodiments are illustrated by way of example in the drawings that are for illustrative purposes only, and are not construed to limit the embodiments, and elements having the same reference numerals in the drawings represent similar elements, unless otherwise specified, and the drawings do not constitute a proportional limitation.

[0051] Figure 1 A flowchart of the method for detecting outliers provided by the embodiments of the present application;

[0052] Figure 2 A flowchart of generating an outlier detection model provided by the embodiments of the present application;

[0053] Figure 3 A schematic diagram of an outlier detection device provided for an embodiment of the present application;

[0054] Figure 4 A schematic diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0056] The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. For the purpose of simplification, the components and arrangements of the specific examples are described in the following. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeatedly refer to numbers and / or letters in different examples. Such repetition is for the purpose of simplification and clarity, and does not indicate the relationship between the various embodiments and / or arrangements discussed.

[0057] First, some names or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

[0058] Information entropy: the expected value of information, describing the uncertainty of information. The greater the entropy, the higher the degree of chaos of the set information.

[0059] Decision tree: composed of a root node, a plurality of internal nodes and leaf nodes, wherein the leaf nodes correspond to decision results, and the other nodes (root node, internal node) correspond to attribute judgment rules. The decision tree is essentially a layer-by-layer recursive judgment according to the conditions.

[0060] Isolation forest is one of the traditional algorithms for outlier detection, wherein outlier detection is the process of finding outliers in data, and outliers are data points that are significantly different from other data points in a given data set.

[0061] However, since the isolation forest algorithm is unsupervised modeling, the use of the isolation forest algorithm in the financial risk control scene has limitations.

[0062] In order to solve the problem that the existing isolation forest algorithm has limitations in the financial risk control scene, the application provides an outlier detection method, which changes the unsupervised modeling process into a supervised modeling process, and evolves the traditional isolation forest algorithm into an algorithm more suitable for the actual scene of financial risk control, overcoming the limitations of the use of the traditional isolation forest algorithm in the financial risk control scene.

[0063] Figure 1 A flowchart of an outlier detection method provided by an embodiment of the application is shown in the following Figure 1 An outlier detection method provided by an embodiment of the application is described.

[0064] As shown in Figure 1 An outlier detection method includes:

[0065] S101, obtaining a user-inputted to-be-predicted sample;

[0066] Among them, outlier detection has important significance in data mining. For example, if the outliers are caused by the variation of the data itself, analyzing them can find hidden deeper, potential and valuable information. For example, finding fraud in finance and insurance, hacking behavior, and also finding the consumption behavior of people with extremely low or high consumption. In the case where the user needs to detect outliers in the data, the user-inputted to-be-predicted sample is obtained.

[0067] S102, inputting the to-be-predicted sample into an outlier detection model to obtain an outlier value of the to-be-predicted sample, wherein the outlier value represents an abnormal situation of the to-be-predicted sample, and the outlier detection model is obtained by model training based on a user-inputted training sample, and the training sample has a corresponding label.

[0068] In this embodiment, the user-inputted to-be-predicted sample is inputted into the outlier detection model, and the outlier detection model outputs the outlier value of the to-be-predicted sample.

[0069] It should be noted that the outlier value is used to represent the abnormal situation and abnormal probability of the user-inputted to-be-predicted sample. The greater the value of the outlier value, the more abnormal the user-inputted to-be-predicted sample; the smaller the value of the outlier value, the less abnormal the user-inputted to-be-predicted sample.

[0070] The application obtains a user-inputted to-be-predicted sample, inputs the to-be-predicted sample into an outlier value detection model, and obtains an outlier value of the to-be-predicted sample, wherein the outlier value represents an abnormal situation of the to-be-predicted sample, and the outlier value detection model is obtained by model training based on a user-inputted training sample, and the training sample has a corresponding label. The application changes an unsupervised modeling process of an isolated forest into a supervised modeling process, evolves a traditional isolated forest algorithm into an algorithm more suitable for a financial risk control actual scene, and improves the effect at the application level and overcomes the limitations of the isolated forest algorithm in the financial risk control scene.

[0071] As an optional embodiment, Figure 2 A flowchart for generating an outlier value detection model is provided for the embodiments of the application. The following describes the generation of the outlier value detection model in combination with Figure 2 The generation of the outlier value detection model by the embodiments of the application is described, and the generation of the outlier value detection model comprises:

[0072] S201, obtaining a user-inputted training sample, wherein the training sample has a corresponding label;

[0073] The application performs modeling by obtaining a user-inputted training sample, wherein the training sample has a corresponding label, thereby realizing a supervised modeling process.

[0074] S202, training a single decision tree based on the user-inputted training sample and label;

[0075] It can be understood that the single decision tree model is trained when the user-inputted training sample and the label corresponding to the training sample are obtained.

[0076] S203, decomposing the trained single decision tree to obtain a first rule set after decomposition;

[0077] The single decision tree model is trained, and the trained single decision tree is obtained when the training reaches a preset condition. It should be noted that the tree depth of the decision tree can be fixedly set in the model.

[0078] After obtaining the trained single decision tree, the trained single decision tree is decomposed to obtain a first rule set after decomposition, wherein the first rule set comprises one or more rules.

[0079] S204, in a case where the information entropy of a rule in the first rule set hits the training sample is less than or equal to a set threshold, the rule is put into a pure entropy queue;

[0080] The set threshold is preset, and the set threshold is generally 0.2. In a case where the information entropy of the rule in the first rule set hitting the training sample is less than or equal to 0.2, it is considered that the rule is used to divide the training sample to obtain a relatively determined result, and the rule is put into the pure entropy queue.

[0081] S205, in a case where the information entropy of the rule in the first rule set hitting the training sample is greater than the set threshold, the rule is put into the priority queue.

[0082] The set threshold is the same as the foregoing, and the set threshold is generally 0.2. In a case where the information entropy of the rule in the first rule set hitting the training sample is greater than 0.2, it is considered that the rule is used to divide the training sample to obtain a result that is not relatively determined, and the rule is put into the priority queue, to facilitate re-extraction of the rule.

[0083] S206, the number of rules in the priority queue is compared with a preset number to obtain a comparison result.

[0084] After all the rules in the first rule set are divided into the pure entropy queue and the priority queue, the number of rules in the priority queue is compared with the preset number. It needs to be noted that the preset number is generally set to 0.

[0085] S207, in a case where the comparison result is that the number of rules in the priority queue is less than or equal to the preset number, a forest formed by the trained decision tree is used as an anomaly value detection model.

[0086] In a case where the number of rules in the priority queue is less than or equal to the preset number, that is, in a case where the number of rules in the priority queue is less than or equal to 0, it is considered that the anomaly value detection model is trained, and a forest formed by the trained decision tree is used as an anomaly value detection model.

[0087] The application enhances the explainability of the anomaly value detection model by disassembling the tree structure of the original decision tree into a rule structure.

[0088] Optionally, the method further includes:

[0089] In a case where the comparison result is that the number of rules in the priority queue is greater than the preset number, the rules in the priority queue are re-extracted to obtain extracted rules.

[0090] It can be understood that in a case where the number of rules in the priority queue is greater than the preset number, that is, in a case where the number of rules in the priority queue is greater than 0, the rules in the priority queue need to be re-extracted to obtain extracted rules.

[0091] Exemplarily, one rule in the priority queue is "male, age>30", the rule is re-extracted to obtain an extracted rule, which can be "male, age>30".

[0092] A first sub-sample is determined based on the extracted rule, wherein the first sub-sample includes samples covered by a leaf node in the trained decision tree;

[0093] Exemplarily, the extracted rule is "male, age>30", and the first sub-sample is determined as samples covered by "male, age>30". It can be understood that, which rule in the priority queue is re-extracted, the first sub-sample corresponding to the re-extracted rule is samples covered by the rule in the corresponding priority queue.

[0094] Training of a new single decision tree is performed based on the extracted rule and the first sub-sample;

[0095] The training of the new single decision tree, i.e., the training of the second decision tree in the forest, is performed according to the extracted rule, the first sub-sample, and labels corresponding to the first sub-sample.

[0096] The trained new single decision tree is decomposed to obtain a second rule set after decomposition;

[0097] The second rule set includes one or more rules. After the training of the new single decision tree using the first sub-sample and corresponding labels is completed, i.e., when the training reaches a preset condition, the trained new single decision tree is decomposed to obtain the second rule set after decomposition.

[0098] In a case where information entropy of a rule in the second rule set hitting the first sub-sample is less than or equal to a set threshold, the rule is put into a pure entropy queue;

[0099] The set threshold is preset, and generally, the set threshold is 0.2. In a case where information entropy of a rule in the second rule set hitting the first sub-sample is less than or equal to 0.2, it is considered that a relatively certain result can be obtained by using the rule to divide the first sub-sample, and thus the rule is put into the pure entropy queue.

[0100] In a case where information entropy of a rule in the second rule set hitting the first sub-sample is greater than the set threshold, the rule is put into the priority queue;

[0101] The set threshold is the same as described above, and generally, the set threshold is 0.2. In a case where information entropy of a rule in the second rule set hitting the first sub-sample is greater than 0.2, it is considered that the result obtained by using the rule to divide the first sub-sample is not a relatively certain result, and thus the rule is put into the priority queue, facilitating re-extraction of the rule.

[0102] Return to perform the step of comparing the number of rules in the priority queue with the preset number, obtaining the comparison result.

[0103] It should be noted that in the case of the comparison result being that the number of rules in the priority queue is greater than the preset number, the above-mentioned re-extraction of the rules in the priority queue is repeatedly performed to obtain the extracted rules; the first sub-sample is determined based on the extracted rules, wherein the first sub-sample includes the samples covered by the leaf nodes in the trained decision tree; the training of the new single decision tree is performed based on the extracted rules and the first sub-sample; the trained new single decision tree is decomposed to obtain the second rule set after decomposition; in the case that the information entropy of the rules in the second rule set hitting the first sub-sample is less than or equal to the set threshold, the rules are put into the pure entropy queue; in the case that the information entropy of the rules in the second rule set hitting the first sub-sample is greater than the set threshold, the rules are put into the priority queue; return to perform the step of comparing the number of rules in the priority queue with the preset number, obtaining the comparison result, until the comparison result is that the number of rules in the priority queue is less than or equal to the preset number, that is, until the number of rules in the priority queue is 0. In the case that the number of rules in the priority queue is 0, all rules are located in the pure entropy queue, then the final anomaly value detection model is obtained, and the forest composed of the multiple trained decision trees is taken as the anomaly value detection model.

[0104] In the case that all rules are located in the pure entropy queue, the final anomaly value detection model is obtained, which considers the actual difference between the positive and negative samples and splits the score at the positive and negative ends.

[0105] As an optional embodiment, the anomaly value of the to-be-predicted sample is obtained by inputting the to-be-predicted sample into the anomaly value detection model, comprising:

[0106] The third rule set hit by the to-be-predicted sample in the pure entropy queue is obtained.

[0107] After the to-be-predicted sample input by the user is input into the anomaly value detection model, the anomaly value detection model obtains the third rule set hit by the to-be-predicted sample in the pure entropy queue, and the third rule set includes one or more rules.

[0108] The response rate of all rules in the third rule set hitting the training sample is compared with the average response rate of the training sample to obtain a comparison result.

[0109] It should be noted that for each rule in the pure entropy queue, the average response rate of the training sample hit by the rule on the modeling data set can be distinguished, that is, the average value of the target y value and the average response rate of the training sample, and the response rate of each rule in the third rule set hitting the training sample is compared with the average response rate of the training sample to obtain a comparison result.

[0110] In the case that the comparison result is that the average response rate of the rule in the third rule set hitting the training sample is greater than the average response rate of the training sample, the length of the rule is added to the preset score, and the preset score is updated.

[0111] The preset score is generally 0. The length of the rule can be the number of times of splitting of the sample hit by the rule. For example, the rule is "male, age greater than 30, age greater than 35", and the number of times of splitting of the sample hit by the rule is 3, and the length of the rule is 3.

[0112] In the case that the comparison result is that the average response rate of the rule hitting the training sample is greater than the average response rate of the training sample, the length of the rule is added to the preset score, and the preset score is updated.

[0113] The average response rate of the rule hitting the training sample is the probability of the sample with a label value of 1 in the sample covered by the rule. For example, the rule is "male, age greater than 30", and the probability of the corresponding label being 1 in the sample covered by the rule is 0.3, and the label represents income greater than 20,000 yuan. The average response rate of the training sample is the probability of the corresponding label being 1 in the training sample, which is 0.2. It can be seen that in this example, the average response rate of the rule hitting the training sample is greater than the average response rate of the training sample.

[0114] In the case that the comparison result is that the average response rate of the rule in the third rule set hitting the training sample is less than or equal to the average response rate of the training sample, the length of the rule is subtracted from the preset score, and the preset score is updated.

[0115] In the case that the comparison result is that the average response rate of the rule hitting the training sample is greater than the average response rate of the training sample, the length of the rule is subtracted from the preset score. It can be understood that in the case that the average response rate of the first rule hitting the training sample is greater than the average response rate of the training sample, the length of the rule is subtracted from the preset score of 0. For example, the length of the rule is 3, and the updated preset score is -3.

[0116] It should be noted that the preset score is updated after the comparison of the average response rate of each rule in the third rule set hitting the training sample and the average response rate of the training sample. Therefore, in the case that the comparison of the average response rate of all rules in the third rule set hitting the training sample and the average response rate of the training sample is completed, the preset score is determined as the abnormal value of the to-be-predicted sample. It can be understood that the preset score is the total score of all rules in the third rule set.

[0117] As an optional embodiment, before the training of the single decision tree based on the user-input training sample and the label, the method further comprises:

[0118] Detecting whether the user-input training sample and the label corresponding to the training sample are compliant.

[0119] Wherein, the detection of whether the user-input training sample is compliant comprises:

[0120] Detecting whether the user-input training sample is a binary classification variable, i.e. whether the data type and data value range of the training sample are within the range of the binary classification variable.

[0121] The detection of whether the label corresponding to the training sample is compliant comprises:

[0122] Detecting whether the label corresponding to the training sample is a numerical variable, i.e. whether the data type and data value range of the label corresponding to the training sample are within the range of the numerical variable.

[0123] It can be understood that in the case where any one of the user-input training sample and the label corresponding to the training sample is detected to be non-compliant, the training of the single decision tree based on the user-input training sample and the label and the subsequent steps are aborted; in the case where both the user-input training sample and the label corresponding to the training sample are detected to be compliant, the training of the single decision tree based on the user-input training sample and the label and the subsequent steps are performed.

[0124] As an optional embodiment, the trained single decision tree is decomposed to obtain a first rule set after decomposition, comprising:

[0125] Starting from the root node of the trained single decision tree, the path from the root node to each leaf node of the single decision tree is obtained;

[0126] The path from the root node to each leaf node of the single decision tree is determined as a plurality of rules after decomposition of the single decision tree, and the plurality of rules constitute the first rule set.

[0127] Optionally, the method further comprises:

[0128] Obtaining a user-input decision tree parameter;

[0129] The training of the single decision tree based on the user-input training sample and the label comprises:

[0130] Training the single decision tree based on the user-input training sample, the label and the decision tree parameter.

[0131] It should be noted that the decision tree parameter can also be a user-input tree depth of the decision tree and a node split evaluation index, etc.

[0132] Of course, before training the single decision tree based on the user-inputted training sample, label and decision tree parameter, it is also necessary to detect whether the user-inputted decision tree parameter is compliant, and in the case that the user-inputted decision tree parameter, the numerical type and data value range of the label corresponding to the training sample are within the required range, train the single decision tree based on the user-inputted training sample, label and decision tree parameter.

[0133] According to another embodiment of the present application, an outlier detection device is provided, as shown in Figure 3 The device comprises:

[0134] The to-be-predicted sample acquisition unit 301 is configured to acquire a user-inputted to-be-predicted sample.

[0135] The outlier acquisition unit 302 is configured to input the to-be-predicted sample into an outlier detection model to obtain an outlier of the to-be-predicted sample, wherein the outlier represents an abnormal situation of the to-be-predicted sample; the outlier detection model is obtained by model training based on a user-inputted training sample, and the training sample has a corresponding label.

[0136] It should be noted that the to-be-predicted sample acquisition unit 301 in this embodiment can be configured to perform step S101 in the embodiments of the present application, and the outlier acquisition unit 302 in this embodiment can be configured to perform step S102 in the embodiments of the present application.

[0137] It should be noted that the above units have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in the above embodiments. It should be noted that the above units, as part of the device, can be implemented by software or hardware.

[0138] Through the above units, the unsupervised modeling process of the Isolation Forest is changed into a supervised modeling process, and the traditional Isolation Forest algorithm is evolved into an algorithm more suitable for the actual scene of financial risk control, thereby improving the effect of the application level and overcoming the limitations of the Isolation Forest algorithm in the financial risk control scene.

[0139] As an optional embodiment, the device further comprises an outlier detection model generation unit, wherein the outlier detection model generation unit is configured to generate an outlier detection model.

[0140] The outlier detection model generation unit comprises:

[0141] The training sample acquisition unit is configured to acquire a user-inputted training sample, and the training sample has a corresponding label;

[0142] a single decision tree training unit configured to train a single decision tree based on the training samples and the labels input by the user;

[0143] a single decision tree decomposition unit configured to decompose the trained single decision tree to obtain a first rule set after decomposition;

[0144] a pure entropy queue obtaining unit configured to put a rule in a pure entropy queue if information entropy of the rule in the first rule set hitting the training sample is less than or equal to a set threshold value

[0145] a priority queue obtaining unit configured to put a rule in a priority queue if information entropy of the rule in the first rule set hitting the training sample is greater than the set threshold value;

[0146] a first comparison unit configured to compare a number of rules in the priority queue with a preset number to obtain a comparison result;

[0147] an outlier detection model generation subunit configured to, if the comparison result is that the number of rules in the priority queue is less than or equal to the preset number, take the forest of trained decision trees as an outlier detection model.

[0148] As an optional embodiment, the device further comprises:

[0149] a rule re-extraction unit configured to, if the comparison result is that the number of rules in the priority queue is greater than the preset number, re-extract the rules in the priority queue to obtain extracted rules;

[0150] a first sub-sample determination unit configured to determine a first sub-sample based on the extracted rules, wherein the first sub-sample includes samples covered by leaf nodes in the trained decision trees;

[0151] a new single decision tree training unit configured to train a new single decision tree based on the extracted rules and the first sub-sample;

[0152] a new single decision tree decomposition unit configured to decompose the trained new single decision tree to obtain a second rule set after decomposition;

[0153] the pure entropy queue obtaining unit is further configured to put a rule in a pure entropy queue if information entropy of the rule in the second rule set hitting the first sub-sample is less than or equal to the set threshold value;

[0154] the priority queue obtaining unit is further configured to put a rule in a priority queue if information entropy of the rule in the second rule set hitting the first sub-sample is greater than the set threshold value;

[0155] a return execution unit configured to return to execute the step of comparing the number of rules in the priority queue with the preset number to obtain the comparison result.

[0156] As an optional embodiment, the outlier obtaining unit 302 comprises: an outlier obtaining subunit, configured to obtain a third rule set in which a rule in the third rule set hits a pure entropy queue of the to-be-predicted sample;

[0157] Compare the average response rate of all rules in the third rule set hitting the training sample with the average response rate of the training sample to obtain a comparison result;

[0158] In a case where the comparison result is that the average response rate of the rule in the third rule set hitting the training sample is greater than the average response rate of the training sample, the preset score is updated by adding the length of the rule on the basis of the preset score;

[0159] In a case where the comparison result is that the average response rate of the rule in the third rule set hitting the training sample is less than or equal to the average response rate of the training sample, the preset score is updated by subtracting the length of the rule on the basis of the preset score;

[0160] In a case where the comparison of the average response rate of all rules in the third rule set hitting the training sample with the average response rate of the training sample is completed, the preset score is determined as the outlier of the to-be-predicted sample.

[0161] As an optional embodiment, the apparatus further comprises:

[0162] A compliance detection unit, configured to detect whether the training sample input by the user and the label corresponding to the training sample are compliant;

[0163] A single decision tree training unit, comprising: a single decision tree training subunit, configured to, in a case where it is detected that the training sample input by the user and the label corresponding to the training sample are compliant, train a single decision tree based on the training sample and the label input by the user.

[0164] As an optional embodiment, the single decision tree decomposition unit comprises: a single decision tree decomposition subunit, configured to decompose from a root node of the trained single decision tree to obtain a path from the root node to each leaf node of the single decision tree;

[0165] The path from the root node to each leaf node of the single decision tree is determined as a plurality of rules after decomposition of the single decision tree, and the plurality of rules constitute the first rule set.

[0166] As an optional embodiment, the apparatus further comprises:

[0167] A decision tree parameter obtaining unit, configured to obtain a decision tree parameter input by the user;

[0168] The single decision tree training unit is further configured to train the single decision tree based on the training sample, the label and the decision tree parameter input by the user.

[0169] It should be noted that the examples and application scenarios implemented by the above-described units and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above-described units, as part of the device, can be implemented in software or hardware.

[0170] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described outlier detection method is also provided.

[0171] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, the electronic device may include one or more ( Figure 4 (Only one is shown) processor 401, communication interface 402, memory 403 and communication bus 404, wherein processor 401, communication interface 402 and memory 403 communicate with each other through communication bus 404;

[0172] Memory 403 is used to store computer programs;

[0173] When the processor 401 executes the program stored in the memory 403, it implements the steps of the above method embodiment.

[0174] The bus mentioned in the above server diagram can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0175] Communication interface 402 is used for communication between the above-mentioned electronic device and other devices.

[0176] The memory 403 may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0177] The processor 401 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0178] Those of ordinary skill in the art can understand that, Figure 4 The structure shown is only schematic, and does not limit the structure of the server described above. For example, the server can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 4 The structure shown is only schematic, and does not limit the structure of the server described above. For example, the server can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 4 The structure shown is only schematic, and does not limit the structure of the server described above. For example, the server can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure.

[0179] The computer readable storage medium of the embodiment of the present application further stores a computer program, and the computer program is executed by a processor to implement the method steps of any one of the foregoing method embodiments.

[0180] Optionally, in the embodiment, the storage medium described above can include, but is not limited to: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0181] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0182] Those skilled in the art can clearly understand the implementation of the embodiments by the description of the above embodiments. The embodiments can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0183] It is to be understood that the terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "has" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order in which they are described unless specifically indicated as such. It is also to be understood that additional or alternative steps can be employed.

[0184] The above description is merely that of specific embodiments of the present application, and thus is not intended to limit the present application. Various modifications to these embodiments can be apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Accordingly, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An outlier detection method, characterized in that, The method includes: Obtain the user-inputted sample to be predicted; The sample to be predicted is input into an outlier detection model to obtain outliers of the sample to be predicted. The outlier detection model is applied to a financial risk control scenario, and the outliers represent the abnormal situation of the sample to be predicted. The outlier detection model is trained based on training samples input by the user, and the training samples have corresponding labels. Generating the outlier detection model includes: Obtain training samples input by the user, wherein the training samples have corresponding labels; Training a single decision tree based on user-input training samples and labels; The trained single decision tree is decomposed to obtain the first set of rules after decomposition; If the information entropy of a rule in the first rule set that hits a training sample is less than or equal to a set threshold, the rule is placed in the pure entropy queue. If the information entropy of a rule in the first rule set that hits a training sample is greater than a set threshold, the rule is placed in a priority queue. The number of rules in the priority queue is compared with a preset number to obtain the comparison result; If the comparison result shows that the number of rules in the priority queue is less than or equal to a preset number, the forest composed of the trained decision trees will be used as the outlier detection model.

2. The method according to claim 1, characterized in that, The method further includes: If the comparison result shows that the number of rules in the priority queue is greater than a preset number, the rules in the priority queue are extracted again to obtain the extracted rules; The first subsample is determined based on the extraction rules, wherein the first subsample includes samples covered by the leaf nodes in the trained decision tree; Training a new single decision tree based on the extraction rules and the first subsample; The trained new single decision tree is decomposed to obtain the decomposed second rule set; If the information entropy of a rule in the second rule set that hits the first subsample is less than or equal to a set threshold, the rule is placed in the pure entropy queue. If the information entropy of a rule in the second rule set that hits the first subsample is greater than a set threshold, the rule is placed in a priority queue. Return to the step of comparing the number of rules in the priority queue with the preset number to obtain the comparison result.

3. The method according to claim 1, characterized in that, The step of inputting the sample to be predicted into the outlier detection model to obtain the outlier of the sample to be predicted includes: Obtain the third rule set of the sample to be predicted hitting the pure entropy queue; The average response rate of all rules in the third rule set hitting the training samples is compared with the average response rate of the training samples to obtain the comparison result. If the comparison result shows that the average response rate of the rules in the third rule set hitting the training samples is greater than the average response rate of the training samples, the preset score is updated by adding the length of the rule to the preset score. If the comparison result shows that the average response rate of the rules in the third rule set hitting the training samples is less than or equal to the average response rate of the training samples, the preset score is updated by subtracting the length of the rule from the preset score. When comparing the average response rate of all rules in the third rule set that hit the training sample with the average response rate of the training sample, the preset score is determined as the outlier of the sample to be predicted.

4. The method according to claim 1, characterized in that, Prior to training a single decision tree based on user-input training samples and labels, the method further includes: Detect whether the training samples and their corresponding labels input by the user are compliant; Training a single decision tree based on user-input training samples and labels includes: If the user-input training sample and its corresponding label are found to be compliant, a single decision tree is trained based on the user-input training sample and label.

5. The method according to claim 1, characterized in that, The process of decomposing the trained single decision tree to obtain the first set of rules includes: Starting from the root node of the trained single decision tree, decompose it to obtain the path from the root node to each leaf node of the single decision tree. The path from the root node of the single decision tree to each leaf node is determined as multiple rules after the single decision tree is decomposed, and the multiple rules constitute the first rule set.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the decision tree parameters input by the user; The training of a single decision tree based on user-input training samples and labels includes: Training a single decision tree based on user-input training samples, labels, and decision tree parameters.

7. An outlier detection device, characterized in that, The device includes: The sample acquisition unit is used to acquire the sample to be predicted input by the user. An outlier acquisition unit is used to input the sample to be predicted into an outlier detection model to obtain outliers of the sample to be predicted. The outlier detection model is applied to a financial risk control scenario, and the outliers represent the abnormal situation of the sample to be predicted. The outlier detection model is obtained by training the model based on training samples input by the user, and the training samples have corresponding labels. Generating the outlier detection model includes: Obtain training samples input by the user, wherein the training samples have corresponding labels; Training a single decision tree based on user-input training samples and labels; The trained single decision tree is decomposed to obtain the first set of rules after decomposition; If the information entropy of a rule in the first rule set that hits a training sample is less than or equal to a set threshold, the rule is placed in the pure entropy queue. If the information entropy of a rule in the first rule set that hits a training sample is greater than a set threshold, the rule is placed in a priority queue. The number of rules in the priority queue is compared with a preset number to obtain the comparison result; If the comparison result shows that the number of rules in the priority queue is less than or equal to a preset number, the forest composed of the trained decision trees will be used as the outlier detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the steps of the outlier detection method according to any one of claims 1-6 through the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the calculation program is executed by the processor, it implements the steps of the outlier detection method as described in any one of claims 1 to 6.

Images