A data recognition method, device, apparatus, and storage medium

Abstract

The present disclosure provides a data recognition method, device and equipment and storage medium, relates to the technical field of computers, in particular to the technical field of big data. The specific implementation scheme is: obtaining a training sample set from a target memory, the training sample set including target samples; performing rule extraction on the training sample set according to a strategy mining algorithm by a target processor to obtain a candidate rule set; screening rules in the candidate rule set according to a target sample rate of the rules to generate an optimal rule set; and recognizing to-be-recognized data according to the optimal rule set to obtain a recognition result. The data recognition method, device and equipment and storage medium provided by the present disclosure realize an end-to-end automatic process in the rule generation process, solve the problem that a large amount of manual work is required for rule generation and the dependence on expert experience is high, and the generated candidate rule set is screened to ensure that rules with better reasoning and judgment effects can be generated.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computers, in particular to a data recognition method and device, equipment and storage medium in the technical field of big data. BACKGROUND

[0002] The rules are generally generated by the characteristics and characteristic data corresponding to the data objects, which can reflect the local laws of the data, and thus are often used for classifying the data or recognizing a specific type of data object. For example, in the field of industrial product detection, detection rules generated by industrial product characteristics and characteristic data can be used to identify low-quality industrial products; in the field of financial risk control, risk control rules generated by user characteristics and characteristic data can be used to identify risk users. In the era of big data, how to achieve accurate recognition of data is crucial to all industries. In the prior art, the data is generally recognized by using the previous rule set or a rule set generated manually by business personnel according to experience. SUMMARY

[0003] The present disclosure provides a more efficient data recognition method, device, equipment and storage medium.

[0004] According to an aspect of the present disclosure, a data recognition method is provided, comprising: obtaining a training sample set from a target storage, the training sample set comprising a target sample; performing rule extraction on the training sample set by a target processor according to a strategy mining algorithm to obtain a candidate rule set; screening rules in the candidate rule set according to a target sample rate of the rules in the candidate rule set to generate an optimal rule set; and recognizing to-be-recognized data according to the optimal rule set to obtain a recognition result.

[0005] According to another aspect of the present disclosure, a data recognition device is provided, comprising: an obtaining module configured to obtain a training sample set from a target storage, the training sample set comprising a target sample; an extracting module configured to perform rule extraction on the training sample set by a target processor according to a strategy mining algorithm to obtain a candidate rule set; a generating module configured to screen rules in the candidate rule set according to a target sample rate of the rules in the candidate rule set to generate an optimal rule set; and a recognizing module configured to recognize to-be-recognized data according to the optimal rule set to obtain a recognition result.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0007] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method of the present disclosure.

[0008] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of the present disclosure.

[0009] It should be understood that the contents described in this part are not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not limit the present disclosure. Among them:

[0011] Figure 1 is a flowchart of a data recognition method according to a first embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of a data recognition method according to a second embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of a data recognition method according to a third embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of a data recognition method according to a fourth embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of a data recognition method according to a fifth embodiment of the present disclosure;

[0016] Figure 6 is a flowchart of a data recognition method according to a sixth embodiment of the present disclosure;

[0017] Figure 7 is a flowchart of a data recognition method according to a seventh embodiment of the present disclosure;

[0018] Figure 8 is a structural schematic diagram of a data recognition device according to a ninth embodiment of the present disclosure;

[0019] Figure 9 is a block diagram of an electronic device for implementing the data recognition method of the present embodiment. DETAILED DESCRIPTION

[0020] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are meant to be exemplary in nature, and include various details intended to facilitate understanding of the present disclosure. Thus, it should be apparent to those skilled in the art that various modifications and changes can be made in the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, it should be apparent to those skilled in the art that the descriptions set forth in this specification are meant to be illustrative only and should not be taken as restrictive.

[0021] Figure 1 is a flowchart of a data recognition method according to a first embodiment of the present disclosure, as shown in Figure 1 , the method mainly includes:

[0022] Step S101, obtaining a training sample set from a target memory, the training sample set including a target sample.

[0023] In the present embodiment, first, a training sample set needs to be obtained from a target memory, the training sample set including training samples corresponding to actual business scenarios. For example, in a financial risk control scenario, a training sample can include a user account, a user credit, a user loan balance, and whether the user is overdue, etc., wherein the user loan balance is the loan balance that the user has not repaid; in a commodity recommendation scenario, a training sample can include a customer account, a customer browsing history, a customer purchase record, and a customer return record, etc.; in an industrial product detection scenario, a training sample can include an industrial product number, an industrial product shape parameter, an industrial product color, and an industrial product weight, etc. Of course, the training sample can also come from other business scenarios, and the present disclosure does not limit the business scenario to which the training sample belongs. The training sample set includes a target sample, which can be a specific type of training sample. For example, in a financial risk control scenario, the target sample can be a training sample corresponding to a user account with high risk; in an industrial product detection scenario, the target sample can be a training sample corresponding to an industrial product with low quality.

[0024] Step S102, extracting rules from the training sample set according to a strategy mining algorithm by a target processor to obtain a candidate rule set.

[0025] In the present embodiment, after obtaining the training sample set, the target processor needs to extract rules from the training sample set according to a strategy mining algorithm to obtain a candidate rule set. Specifically, each training sample in the training sample set includes features corresponding to the training sample. For example, in a financial risk control scenario, if a training sample includes a user account, a user credit, a user loan balance, and whether the user is overdue, then the user account, the user credit, the user loan balance, and whether the user is overdue can all be features of the training sample. The strategy mining algorithm can extract rules that meet the preset conditions from all features of the training sample set and add the extracted rules to the candidate rule set.

[0026] In an implementation, the strategy mining algorithm can be a decision tree algorithm, a binning algorithm, etc. If the strategy mining algorithm is a decision tree algorithm, a decision tree is trained according to the training sample set, a path corresponding to each leaf node of the generated decision tree is a rule, and rules meeting preset conditions can be added to the candidate rule set. If the strategy mining algorithm is a binning algorithm, binning is performed on each feature of the training sample set, each binning corresponds to a rule, and rules meeting preset conditions can be added to the candidate rule set. If no binning of a feature meets the preset conditions, the binning of the feature is crossed with binning of other features to generate new binning, and rules meeting the preset conditions corresponding to the new binning are added to the candidate rule set. The preset conditions can be set according to actual conditions.

[0027] In step S103, rules in the candidate rule set are screened according to target sample rates of the rules to generate an optimal rule set.

[0028] In step S104, data to be identified is identified according to the optimal rule set to obtain an identification result.

[0029] In the embodiment, training samples hit by multiple rules in the candidate rule set can be repeated, and identification capabilities of all rules are different. Therefore, rules in the candidate rule set need to be screened according to a strategy generation algorithm to generate an optimal rule set, and then data to be identified is identified using the optimal rule set to obtain an identification result. The data to be identified is data to be identified, and the identification process is similar to that of the prior art, which will not be described here.

[0030] In an implementation, the training sample set includes target samples, and a target sample rate of each rule in the candidate rule set can be calculated, where the target sample rate is a ratio of a number of target samples hit by the rule to a number of all training samples hit by the rule. Then, rules in the candidate rule set are sorted in descending order according to the target sample rate, and multiple rules at the top of the sorting are selected to generate the optimal rule set, that is, multiple rules with higher target sample rates are selected to generate the optimal rule set. The number of selected rules can be determined according to actual conditions.

[0031] In the first embodiment of the disclosure, the target processor automatically generates a candidate rule set according to a strategy mining algorithm, and then directly screens rules in the candidate rule set to generate an optimal rule set. An end-to-end automated process is realized in the generation of the optimal rule set, the problem that a large amount of manual work is required for rule generation and the problem that the dependence on expert experience is high are solved, the candidate rule set is screened to ensure that the optimal rule set with better identification effect can be generated, and the accuracy of the identification result is further improved by identifying data to be identified using the optimal rule set.

[0032] In the second embodiment of the present disclosure, the policy mining algorithm is a decision tree algorithm, Figure 2 is a flowchart of a data recognition method according to the second embodiment of the present disclosure, as Figure 2 shown, step S102 mainly includes:

[0033] Step S201, according to the preset parameters and the training sample set, training a decision tree.

[0034] Step S202, calculate the target sample rate of the leaf node in the decision tree, and add the rule corresponding to the leaf node with the target sample rate greater than the first preset threshold to the candidate rule set.

[0035] In this embodiment, first, according to the preset parameters, a decision tree is trained on the training sample set, each training sample will fall into a leaf node of the decision tree, and the path corresponding to each leaf node in the decision tree is a rule. Then, the target sample rate of all leaf nodes in the decision tree is calculated, and the rule corresponding to the leaf node with the target sample rate greater than the first preset threshold, i.e. the path corresponding to the leaf node with the target sample rate greater than the first preset threshold, is added to the candidate rule set. Wherein, the target sample rate is the ratio of the number of target samples in the leaf node to the number of all training samples in the leaf node. Specifically, the preset parameters can include the maximum number of layers and the minimum number of leaf node samples, wherein the maximum number of layers is the number of layers of the trained decision tree, and the minimum number of leaf node samples is the minimum number of training samples hit by the leaf node, i.e. if the number of training samples falling into a leaf node is less than the minimum number of leaf node samples, the leaf node will be pruned. The preset parameters and the first preset threshold can be set according to actual conditions.

[0036] Step S203, delete the features corresponding to the rules from the training sample set, and retrain the decision tree according to the preset parameters and the training sample set after deleting the features, until the first preset condition is met.

[0037] In this embodiment, after adding the rules corresponding to the leaf nodes with the target sample rate greater than the first preset threshold in the trained decision tree to the candidate rule set, all features corresponding to the above rules can also be deleted from the training sample set. Then, retrain the decision tree according to the preset parameters and the training sample set after deleting the features, and add the rules corresponding to the leaf nodes with the target sample rate greater than the first preset threshold in the retrained decision tree to the candidate rule set until the first preset condition is met.

[0038] In an embodiment, not only the features corresponding to the extracted rules can be deleted from the training sample set, but also all the training samples corresponding to the extracted rules can be deleted, and then the decision tree is retrained. In the process of extracting rules by using the decision tree, the rules and the training samples in the training sample set are filtered constantly, so that the training sample set used for training the decision tree each time is different, the accuracy of the extracted rules is further improved, and the repetition rate of the extracted rules is reduced.

[0039] In an embodiment, the first preset condition can be that there is no feature in the training sample set or the number of features in the training sample set is insufficient to train a decision tree. The first preset condition can also be that the number of rules in the candidate rule set reaches a certain value, which can be set according to actual conditions.

[0040] In the second embodiment of the present disclosure, according to the decision tree algorithm, the rules in the training sample set are extracted, the rules corresponding to the leaf nodes with a target sample rate greater than a first preset threshold in the trained decision tree are added to the candidate rule set, and the rules and the training samples in the training sample set are filtered constantly, so that the accuracy of the candidate rule set is further improved, and the repetition rate of the extracted rules is reduced.

[0041] In the third embodiment of the present disclosure, the strategy mining algorithm is an extreme gradient boosting algorithm, Figure 3 is a flowchart of a data recognition method according to the third embodiment of the present disclosure, as shown in Figure 3 The step S102 mainly includes:

[0042] In step S301, a sample weight is added to each training sample in the training sample set.

[0043] In this embodiment, first, a sample weight needs to be added to each training sample in the training sample set. Specifically, the sample weight can be set according to actual business scenarios. For example, in the field of financial risk control, if a training sample includes a user account, a user limit, a user balance, and whether the user is overdue, the sample weight of each training sample can be calculated according to the following formula: wherein sum(user limit) is the sum of the user limits of all training samples in the training sample set. Of course, according to different actual business scenarios, the sample weight of each training sample can also be determined in other ways.

[0044] In an embodiment, in the field of financial risk control, the sample weight of each training sample can also be calculated according to the following formula: wherein if the user is overdue, the user is overdue is determined as 1; if the user is not overdue, the user is overdue is determined as 0.

[0045] In an implementable manner, in the field of financial risk control, the sample weight of each training sample can also be calculated according to the following formula: Wherein, if the user is overdue, the user is determined to be overdue as 1; if the user is not overdue, the user is determined to be overdue as 0; sum (user is overdue x loan balance) is the product sum of all training samples in the training sample set.

[0046] Step S302, input the training sample set after adding the sample weight into the extreme gradient boosting model for training to obtain multiple rule trees.

[0047] Step S303, calculate the target sample rate of the leaf nodes in the multiple rule trees, and add the rule corresponding to the leaf node with the target sample rate greater than the first preset threshold to the candidate rule set.

[0048] In the embodiment, after adding the sample weight for each training sample in the training sample set, the training sample set after adding the sample weight is input into the extreme gradient boosting (XGBoost, eXtreme Gradient Boosting) model for training to obtain multiple rule trees, then the target sample rate of all leaf nodes in the multiple rule trees is calculated, and the rule corresponding to the leaf node with the target sample rate greater than the first preset threshold is added to the candidate rule set.

[0049] In an implementable manner, because the XGBoost model is used to train multiple rule trees with the same training sample set, the rules extracted from the multiple rule trees may be repeated, so it is also necessary to perform relevance screening on the rules in the candidate rule set. For example, according to the multiple rule trees trained, the training sample corresponding to each rule in the candidate rule set is determined, if the coincidence degree of the training samples corresponding to two rules is greater than a certain value, which can be set according to actual conditions, the rule with the lower target sample rate of the two rules is deleted from the candidate rule set.

[0050] In the third embodiment of the present disclosure, the sample weight is added for each training sample in the training sample set, and the training sample set after adding the sample weight is input into the XGBoost model for rule extraction, which can ensure that the extracted rules have higher accuracy, and the relevance screening is performed on the rules in the candidate rule set, so that a more optimal candidate rule set can be generated.

[0051] In the fourth embodiment of the present disclosure, the strategy mining algorithm is a joint ranking algorithm, Figure 4 is a flowchart of a data recognition method according to the fourth embodiment of the present disclosure, as shown in Figure 4 The step S102 mainly includes:

[0052] Step S401, adding a sample weight for each training sample in the training sample set.

[0053] Step S401 is similar to step S301, and thus is not described herein.

[0054] Step S402, performing a binning operation on the features corresponding to the training sample set, and sorting the features according to information values to obtain a sorting result.

[0055] In this embodiment, a binning algorithm needs to be called to perform a binning operation on all features corresponding to the training sample set, and all features are sorted according to information values to obtain a sorting result. The information value (IV, Information Value) is used to evaluate the prediction ability of a feature, and the size of the information value represents the strength of the prediction or judgment ability of the feature. The information value can be calculated in a manner commonly used in the art, which is not described herein. Specifically, the binning algorithm can be chi-square binning, equidistant binning, or equal frequency binning, and the present disclosure does not limit the binning algorithm.

[0056] Step S403, calculating the target sample rate and the sample weight mean of each binning of each feature according to the sorting result, and finding a first continuous binning with a target sample rate greater than a first preset threshold.

[0057] Step S404, sorting the first continuous binning according to the sample weight mean to obtain a second continuous binning, and adding a rule corresponding to the second continuous binning to the candidate rule set.

[0058] In this embodiment, according to the sorting result, the target sample rate and the sample weight mean of all bins of each feature are calculated in turn from the feature with the largest information value, and a first continuous binning with a target sample rate greater than a first preset threshold is found. Then, the first continuous binning is sorted according to the sample weight mean of all bins to obtain a second continuous binning, and a rule corresponding to the second continuous binning is added to the candidate rule set. Specifically, the target sample rate is the ratio of the target sample number corresponding to the bin to the training sample number corresponding to the bin, and the sample weight mean is the mean of the sample weights of all training samples corresponding to the bin.

[0059] In an implementation, the first continuous bins refer to bins in a continuous position in which the target sample rate of each feature is greater than a first preset threshold. For example, after a feature is binned, five bins are obtained, namely bin 1, bin 2, bin 3, bin 4 and bin 5. The target sample rate of bin 1, bin 3, bin 4 and bin 5 is greater than the first preset threshold. Bin 3, bin 4 and bin 5 jointly form the first continuous bins. If the first continuous bins formed by bin 3, bin 4 and bin 5 are sorted according to the sample weight mean of each bin, the sorting result is bin 4, bin 5 and bin 3, bin 4 and bin 5 jointly form the second continuous bins. At this time, the rule corresponding to bin 1 and the rule corresponding to the second continuous bins formed by bin 4 and bin 5 can be added to the candidate rule set, that is, two rules are added to the candidate rule set.

[0060] In step S405, if the target sample rate of all bins of the current feature is not greater than the first preset threshold, the bins of the current feature and the bins of the next feature are crossed to obtain new bins, and the target sample rate and the sample weight mean of the new bins are continuously calculated until all features are traversed.

[0061] In the embodiment, if the target sample rate of all bins of the current feature is not greater than the first preset threshold, all bins of the current feature and all bins of the next feature are crossed to obtain new bins, that is, each bin of the current feature is combined with all bins of the next feature. For example, the current feature and the next feature each have five bins, and after crossing, there are 25 new bins. The target sample rate and the sample weight mean of the new bins are continuously calculated, that is, steps 403 and 404 are repeated until all features are traversed.

[0062] In an implementation, after all features are traversed, the features and / or samples corresponding to all rules in the candidate rule set can be deleted from the training sample set, and the training sample set after the features and / or samples are deleted is used to repeat steps S402 to S405.

[0063] In the fourth embodiment of the present disclosure, a joint sorting algorithm is used to extract rules from the training sample set. First, all features are sorted and binned according to the information value, and the first continuous bins in which the target sample rate is greater than the first preset threshold are sequentially found on each feature. Then, the first continuous bins are sorted according to the sample weight mean to obtain the second continuous bins, and the rules corresponding to the second continuous bins are added to the candidate rule set. In this way, the joint sorting method of multiple indexes is used to ensure that the optimal and more accurate rules are extracted.

[0064] In the fifth embodiment of the present disclosure, the strategy mining algorithm is a pole searching algorithm. Figure 5is a flowchart of a data recognition method according to the fifth embodiment of the present disclosure, as shown in Figure 5 Step S102 mainly includes:

[0065] Step S501, according to the information value, the features corresponding to the training sample set are sorted, and the sorting result is obtained.

[0066] The specific process of step S501 has been discussed in step S402, and will not be repeated here.

[0067] Step S502, according to the sorting result, the target sample rate of the first extreme point interval at the extreme point of the monotone direction of each feature is calculated, and the rule corresponding to the first extreme point interval with the target sample rate greater than the first preset threshold is added to the candidate rule set.

[0068] In this embodiment, after sorting all the features of the training sample set, the target sample rate of the first extreme point interval at the extreme point of the monotone direction of each feature is calculated according to the sorting result, and then the rule corresponding to the first extreme point interval with the target sample rate greater than the first preset threshold is added to the candidate rule set. Wherein, the monotone direction refers to the direction of the feature increasing or decreasing, the extreme point of the monotone direction refers to the maximum value or the minimum value of the feature along the monotone direction, and the extreme point interval refers to the interval near the extreme point of the monotone direction, for example, the monotone direction of a monotonically increasing feature is from the minimum value to the maximum value of the feature, and the extreme point of the monotone direction is the maximum value of the feature, and the first extreme point interval can be an interval containing the maximum value; the monotone direction of a monotonically decreasing feature is from the maximum value to the minimum value of the feature, and the extreme point of the monotone direction is the minimum value of the feature, and the first extreme point interval can be an interval containing the minimum value.

[0069] In an implementable manner, if a feature is a monotonically increasing feature, the minimum value of which is 0 and the maximum value of which is 100, the monotone direction of the feature is from 0 to 100, and the extreme point of the monotone direction of the feature is 100. A quantile can be set near the extreme point of the monotone direction, for example, 0.95, and the corresponding first extreme point interval is [95, 100], and the target sample rate of the interval is the ratio of the target sample number falling into the interval to the total number of training samples falling into the interval. Wherein, the quantile can be set according to actual conditions.

[0070] In an implementable manner, the lift Lift of the first extreme point interval at the extreme point of the monotone direction of each feature can also be calculated according to the sorting result, and the rule corresponding to the first extreme point interval with the lift greater than the first threshold is added to the candidate rule set, wherein the lift can be calculated by the following formula: The first threshold can be determined according to actual conditions.

[0071] In an implementable manner, a sample weight can also be added to each training sample set in the training sample set, and a sample weight average of the first pole interval at each feature monotonic direction pole is calculated, and a rule corresponding to the first pole interval with a Lift greater than a first threshold and a sample weight average greater than a second threshold is added to the candidate rule set.

[0072] In step S503, if the target sample rate of the first pole interval of the current feature is not greater than a first preset threshold, the first pole interval is adjusted to obtain a second pole interval, and the target sample rate of the second pole interval is calculated until a second preset condition is met.

[0073] In the embodiment, if the target sample rate of the first pole interval of the current feature is not greater than a first preset threshold, the first pole interval is adjusted to obtain a second pole interval, and the target sample rate of the second pole interval is calculated until a second preset condition is met. The second preset condition can be that the number of training samples in the candidate rule set reaches a certain value, or the number of adjustments of the first pole interval reaches a certain value, etc. The above values can be determined by the user.

[0074] In an implementable manner, the idea of binary search can be used to adjust the first pole interval by adjusting the quantile. For example, if the monotonic direction of the current feature is from 0 to 100, the monotonic direction pole of the feature is 100, and the quantile is 0.95, i.e. the first pole interval is [95, 100], if the target sample rate of the first pole interval is not greater than a first preset threshold, the first pole interval can be adjusted by adjusting the quantile, for example, the quantile is adjusted to 0.975, and the second pole interval is [97.5, 100], and then the target sample rate of the second pole interval is calculated. If the target sample rate is greater than the first preset threshold, the rule corresponding to the second pole interval is added to the candidate rule set, and then the lower bound of the second pole interval can be adjusted, i.e. if the target sample rate of the second pole interval is greater than the first preset threshold, the second pole interval can be adjusted to The target sample rate of the second pole interval is recalculated. Specifically, the adjustment method of the first pole interval can be determined by the user according to the actual situation.

[0075] In step S504, if the target sample rate of all pole intervals on the current feature is not greater than a first preset threshold, the first pole interval is stored, and the first pole interval is combined with all pole intervals of the next feature to obtain a new pole interval, and the target sample rate of the new pole interval is recalculated until all features are traversed.

[0076] In the embodiment, if the target sample rate of all pole point intervals on the current feature is not greater than the first preset threshold, the first pole point interval of the current feature is stored, which is equivalent to storing the initial pole point interval of the current feature, and when extracting the rule of the next feature, each pole point interval of the next feature is merged with the first pole point interval of the current feature to obtain a new pole point interval, the target sample rate of the new pole point interval is recalculated, and the rule corresponding to the new pole point interval with the target sample rate greater than the first preset threshold is added to the candidate rule set until all features are traversed.

[0077] In the fifth embodiment of the present disclosure, the pole point search algorithm is used to extract rules from the training sample set, to judge whether the target sample rate of the pole point interval at the monotonic direction pole point of the feature is greater than the first preset threshold, if yes, the corresponding rule of the pole point interval is added to the candidate rule set, if not, the range of the pole point interval is adjusted, and the target sample rate of the adjusted pole point interval is re-judged. In addition, the first pole point interval of the current feature is merged with all pole point intervals of the next feature in the present embodiment, which improves the efficiency of rule extraction and ensures that a more accurate candidate rule set is extracted.

[0078] In the second to fifth embodiments of the present disclosure, each strategy mining algorithm is composed of a series of operators, the operators have fixed inputs and outputs, and the end-to-end automated process is realized by connecting all the operators in series, which solves the problem that a large amount of manual work is required for rule generation and the dependence on expert experience is high.

[0079] Figure 6 is a flowchart of a data recognition method according to the sixth embodiment of the present disclosure, as shown in Figure 6 Before step S102, the method further includes:

[0080] Step S601, scoring the training sample set according to the credit scoring card to obtain a scoring result.

[0081] Step S602, binning the training sample set according to the scoring result to obtain a binning result.

[0082] Step S603, filtering the training sample set according to the second preset condition and the binning result to obtain a specified sample set.

[0083] In the embodiment, before extracting rules from the training sample set, the training sample set needs to be filtered. The training samples in the training sample set can be scored according to the credit scoring card, then a binning operation is performed on the training sample set according to the scoring result to obtain a binning result, and then the specified training samples are selected from the binning result according to the second preset condition to obtain a specified sample set.

[0084] In an implementation, the credit score card can be established by using a machine learning model, such as a logistic regression model, and the second preset condition can be that the training samples corresponding to the last two bins in the binning result are taken as the specified sample set. For example, if the training sample set is divided into bin 1, bin 2, bin 3, bin 4 and bin 5 according to the score result of the credit score card, the training samples corresponding to bin 4 and bin 5 can be taken as the specified sample set.

[0085] In an implementation, the step S102 includes: performing rule extraction on the specified sample set according to the strategy mining algorithm by the target processor, to obtain the candidate rule set, that is, only the specified sample set screened out is used for rule extraction, and after the candidate rule set is obtained, the second preset condition is also added to the candidate rule set, so as to ensure the efficiency and quality of rule extraction.

[0086] In the sixth embodiment of the present disclosure, the training sample set is screened before rule extraction is performed on the training sample set, to obtain the specified sample set, and the target processor performs rule extraction on the specified sample set according to the strategy mining algorithm, to obtain the candidate rule set, which can ensure the efficiency and quality of rule extraction.

[0087] Figure 7 is a flowchart of a data recognition method according to the seventh embodiment of the present disclosure, as shown in Figure 7 The step S103 mainly includes:

[0088] In step S701, the lift of the rules in the candidate rule set is calculated, and the lift includes the ratio of the target sample rate of the rule to the target sample rate of the training sample set.

[0089] In step S702, the rules are sorted according to the lift, to obtain a sorting result.

[0090] In step S703, a preset number of rules are selected according to the sorting result to generate the optimal rule set.

[0091] In the present embodiment, the lift of all rules in the candidate rule set is first calculated, which can be the ratio of the target sample rate of a rule to the target sample rate of the training sample set, then all rules are sorted according to the lift, and finally a preset number of rules are selected according to the sorting result to generate the optimal rule set, for example, the first 20 rules with the highest lift are selected according to the sorting result to generate the optimal rule set. The preset number can be determined according to actual conditions.

[0092] In an implementable manner, a sample weight can also be added to each training sample in the training sample set, and then the sample weight average of the training samples corresponding to each rule in the candidate rule set is calculated, and all rules in the candidate rule set are sorted according to the sample weight average, and according to the sorting result, a preset number of rules are selected to generate the optimal rule set, for example, the top 20 rules with the highest sample weight average are selected to generate the optimal rule set.

[0093] In the eighth embodiment of the present disclosure, step S103 mainly includes:

[0094] The rules in the candidate rule set are traversed to determine whether the addition of the rule to the optimal rule set increases the lift of the optimal rule set, and the lift includes the ratio of the target sample rate of the optimal rule set to the target sample rate of the training sample set; if the lift of the optimal rule set increases, the rule is added to the optimal rule set.

[0095] In the present embodiment, the greedy algorithm is used to generate the optimal rule set according to the candidate rule set, specifically, all rules in the candidate rule set are traversed to determine whether the addition of the rule to the optimal rule set increases the lift of the optimal rule set, and if the lift of the optimal rule set increases, the rule is added to the optimal rule set.

[0096] In an implementable manner, it can also be determined whether the addition of the rule to the optimal rule set increases the sample weight average of the optimal rule set, and if the sample weight average of the optimal rule set increases, the rule is added to the optimal rule set.

[0097] In the seventh and eighth embodiments of the present disclosure, the rules in the candidate rule set are screened by using the sorting method or the greedy algorithm to generate the optimal rule set, which can ensure that the rule with the highest accuracy is selected from the optimal rule set to generate the global optimal rule set.

[0098] Figure 8 FIG. 9 is a structural schematic diagram of a data recognition device according to the ninth embodiment of the present disclosure, as shown in the figure, the device mainly includes: Figure 8

[0099] The acquisition module 80 is configured to acquire the training sample set from the target memory, and the training sample set includes target samples; the extraction module 81 is configured to extract rules from the training sample set according to a policy mining algorithm by using the target processor to obtain a candidate rule set; the generation module 82 is configured to screen the rules in the candidate rule set according to the target sample rate of the rules in the candidate rule set to generate an optimal rule set; and the recognition module 83 is configured to recognize the to-be-recognized data according to the optimal rule set to obtain a recognition result.

[0100] ​In an implementation, the strategy mining algorithm is a decision tree algorithm, and the extraction module 81 mainly includes: a first training submodule configured to train a decision tree according to preset parameters and a training sample set; a first calculation submodule configured to calculate a target sample rate of a leaf node in the decision tree, and add a rule corresponding to a leaf node with a target sample rate greater than a first preset threshold to a candidate rule set; and a retraining submodule configured to delete a feature corresponding to the rule from the training sample set, retrain the decision tree according to the preset parameters and the training sample set after the feature is deleted, and stop until a first preset condition is met.

[0101] In an implementation, the strategy mining algorithm is an extreme gradient boosting algorithm, and the extraction module 81 mainly includes: a first adding submodule configured to add a sample weight to a training sample in the training sample set; a second training submodule configured to input the training sample set after the sample weight is added into an extreme gradient boosting model for training to obtain a plurality of rule trees; and a second calculation submodule configured to calculate a target sample rate of a leaf node in the plurality of rule trees, and add a rule corresponding to a leaf node with a target sample rate greater than a first preset threshold to a candidate rule set.

[0102] In an implementation, the strategy mining algorithm is a joint ranking algorithm, and the extraction module 81 mainly includes: a second adding submodule configured to add a sample weight to a training sample in the training sample set; a first binning submodule configured to perform a binning operation on a feature corresponding to the training sample set, and rank the feature according to an information value to obtain a ranking result; a third calculation submodule configured to calculate a target sample rate and a sample weight average of each bin of the feature according to the ranking result, and find a first continuous bin with a target sample rate greater than a first preset threshold; a first ranking submodule configured to rank the first continuous bin according to the sample weight average to obtain a second continuous bin, and add a rule corresponding to the second continuous bin to a candidate rule set; and a first traversal submodule configured to, if the target sample rate of all bins of a current feature is not greater than the first preset threshold, cross the bins of the current feature with bins of a next feature to obtain new bins, continue to calculate a target sample rate and a sample weight average of the new bins, and stop until all features are traversed.

[0103] In an implementable manner, the strategy mining algorithm is a pole searching algorithm, and the extraction module 81 mainly comprises: a second sorting submodule, configured to sort the features corresponding to the training sample set according to information values to obtain a sorting result; a fourth calculation submodule, configured to calculate a target sample rate of a first pole interval at a monotonous direction pole of each feature according to the sorting result, and add a rule corresponding to a first pole interval with a target sample rate greater than a first preset threshold to the candidate rule set; an adjustment submodule, configured to, if the target sample rate of the first pole interval of the current feature is not greater than the first preset threshold, adjust the first pole interval to obtain a second pole interval, and calculate the target sample rate of the second pole interval until a second preset condition is met; and a second traversal submodule, configured to, if the target sample rates of all pole intervals on the current feature are all not greater than the first preset threshold, store the first pole interval, and combine the first pole interval with all pole intervals of a next feature to obtain a new pole interval, and recalculate the target sample rate of the new pole interval until all features are traversed.

[0104] In an implementable manner, the device further comprises: a scoring module, configured to score the training sample set according to a credit scoring card to obtain a scoring result; a binning module, configured to bin the training sample set according to the scoring result to obtain a binning result; and a screening module, configured to screen the training sample set according to a second preset condition and the binning result to obtain a specified sample set; and the extraction module 81 is further configured to extract rules from the specified sample set according to the strategy mining algorithm to obtain the candidate rule set.

[0105] In an implementable manner, the generation module 82 mainly comprises: a fifth calculation submodule, configured to calculate lift degrees of the rules in the candidate rule set, the lift degree comprising a ratio of a target sample rate of the rule to a target sample rate of the training sample set; a third sorting submodule, configured to sort the rules according to the lift degrees to obtain a sorting result; and a selection submodule, configured to select a preset number of rules according to the sorting result to generate the optimal rule set.

[0106] In an implementable manner, the generation module 82 mainly comprises: a judgment submodule, configured to traverse the rules in the candidate rule set, and judge whether the lift degree of the optimal rule set increases when the rule is added to the optimal rule set, the lift degree comprising a ratio of a target sample rate of the optimal rule set to a target sample rate of the training sample set; and an adding submodule, configured to, if the lift degree of the optimal rule set increases, add the rule to the optimal rule set.

[0107] In the technical solution of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0108] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0109] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0110] As shown in Figure 9 The device 900 includes a computing unit 901 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 902 or a computer program loaded into a random access memory (RAM) 903 from a storage unit 908. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0111] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; the storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0112] The computing unit 901 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs various methods and processes described above, such as a data recognition method. For example, in some embodiments, a data recognition method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded onto the RAM 903 and executed by the computing unit 901, one or more steps of a data recognition method described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform a data recognition method by any other appropriate means, such as by means of firmware.

[0113] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0114] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine or entirely on a remote machine or server.

[0115] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0117] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0118] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0119] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology described in the present disclosure are achieved.

[0120] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data recognition method, comprising: obtaining a training sample set from a target memory, the training sample set comprising target samples; training samples in the training sample set comprising a user account, a user credit, a user loan balance and whether the user is overdue; the target sample is the training sample corresponding to the user account with high risk; extracting rules from the training sample set according to a strategy mining algorithm by a target processor to obtain a candidate rule set; screening rules in the candidate rule set according to a target sample rate of the rules in the candidate rule set to generate an optimal rule set; recognizing to-be-recognized data according to the optimal rule set to obtain a recognition result; wherein the strategy mining algorithm is a joint ranking algorithm, and the extracting rules from the training sample set according to the strategy mining algorithm by the target processor to obtain the candidate rule set comprises: adding sample weights to the training samples in the training sample set; performing a binning operation on features corresponding to the training sample set, and ranking the features according to information values to obtain a ranking result; calculating a target sample rate and a sample weight average of each bin of the features according to the ranking result, and finding a first continuous bin with a target sample rate greater than a first preset threshold; ranking the first continuous bin according to the sample weight average to obtain a second continuous bin, and adding a rule corresponding to the second continuous bin to the candidate rule set; if the target sample rate of all bins of the current feature is not greater than the first preset threshold, then cross the bins of the current feature with the bins of the next feature to obtain new bins, and continue to calculate the target sample rate and the sample weight average of the new bins until all the features are traversed; wherein the determination method of the sample weight is at least one of the following: wherein, is the sample weight, is the sum of the user ratings of all training samples in the training sample set; wherein if the user is overdue, the user overdue is determined as 1; if the user is not overdue, the user overdue is determined as 0. wherein sum(user overdue x loan balance) is the product sum of user overdue and loan balance of all training samples in the training sample set.

2. The method of claim 1, wherein, The strategy mining algorithm is a decision tree algorithm, and the extracting rules from the training sample set according to the strategy mining algorithm by the target processor to obtain the candidate rule set comprises: training a decision tree according to a preset parameter and the training sample set; calculating a target sample rate of leaf nodes in the decision tree, and adding a rule corresponding to a leaf node with a target sample rate greater than a first preset threshold to the candidate rule set; deleting features corresponding to the rule from the training sample set, and retraining a decision tree according to the preset parameter and the training sample set after deleting the features until a first preset condition is met.

3. The method of claim 1, wherein, The strategy mining algorithm is an extreme gradient boosting algorithm, and the extracting rules from the training sample set according to the strategy mining algorithm by the target processor to obtain the candidate rule set comprises: adding sample weights to the training samples in the training sample set; inputting the training sample set with added sample weights into an extreme gradient boosting model for training to obtain multiple rule trees; The target sample rate of the leaf nodes in the multiple rule trees is calculated, and a rule corresponding to a leaf node with a target sample rate greater than a first preset threshold is added to the candidate rule set.

4. The method of claim 1, wherein, The strategy mining algorithm is a pole searching algorithm, and the target processor extracts rules from the training sample set according to the strategy mining algorithm to obtain a candidate rule set, including: Features corresponding to the training sample set are sorted according to information values to obtain a sorting result; The target sample rate of a first pole interval at each monotonous direction pole of each feature is calculated according to the sorting result, and a rule corresponding to a first pole interval with a target sample rate greater than a first preset threshold is added to the candidate rule set; If the target sample rate of the first pole interval of the current feature is not greater than the first preset threshold, the first pole interval is adjusted to obtain a second pole interval, and the target sample rate of the second pole interval is calculated until a second preset condition is met; If the target sample rate of all pole intervals on the current feature is not greater than the first preset threshold, the first pole interval is stored, and the first pole interval is combined with all pole intervals of a next feature to obtain a new pole interval, and the target sample rate of the new pole interval is recalculated until all features are traversed.

5. The method according to any one of claims 2 to 4, before the target processor extracts rules from the training sample set according to the strategy mining algorithm to obtain a candidate rule set, the method further includes: The training sample set is scored according to a credit scoring card to obtain a scoring result; The training sample set is binned according to the scoring result to obtain a binning result; The training sample set is filtered according to a second preset condition and the binning result to obtain a specified sample set; The target processor extracts rules from the specified sample set according to the strategy mining algorithm to obtain a candidate rule set. The target processor extracts rules from the specified sample set according to the strategy mining algorithm to obtain a candidate rule set.

6. The method of claim 5, wherein, The rules in the candidate rule set are filtered according to the target sample rates of all rules in the candidate rule set to generate an optimal rule set, including: The lift of the rules in the candidate rule set is calculated, the lift including a ratio of the target sample rate of the rule to the target sample rate of the training sample set; The rules are sorted according to the lift to obtain a sorting result; A preset number of rules are selected according to the sorting result to generate the optimal rule set.

7. The method of claim 5, wherein, The rules in the candidate rule set are filtered according to the target sample rates of all rules in the candidate rule set to generate an optimal rule set, including: The rules in the candidate rule set are traversed to determine whether the lift of the optimal rule set is increased when the rule is added to the optimal rule set, the lift including a ratio of the target sample rate of the optimal rule set to the target sample rate of the training sample set; If the lift of the optimal rule set is increased, the rule is added to the optimal rule set.

8. A data recognition device, including: The acquisition module is configured to acquire a training sample set from a target memory, the training sample set comprising target samples; The training samples in the training sample set comprise a user account, a user credit, a user loan balance, and whether the user is overdue; and the target sample is a training sample corresponding to a user account with high risk; The extraction module is configured to perform rule extraction on the training sample set according to a strategy mining algorithm by a target processor to obtain a candidate rule set; The generation module is configured to filter rules in the candidate rule set according to a target sample rate of the rules in the candidate rule set to generate an optimal rule set; The identification module is configured to identify to-be-identified data according to the optimal rule set to obtain an identification result. The strategy mining algorithm is a joint ranking algorithm, and the extraction of the candidate rule set from the training sample set by the target processor according to the strategy mining algorithm comprises: adding a sample weight to each training sample in the training sample set; performing a binning operation on features corresponding to the training sample set and ranking the features according to information values to obtain a ranking result; calculating a target sample rate and a sample weight average of each bin of each feature according to the ranking result, and finding a first continuous bin with a target sample rate greater than a first preset threshold; ranking the first continuous bin according to the sample weight average to obtain a second continuous bin, and adding a rule corresponding to the second continuous bin to the candidate rule set; if the target sample rate of all bins of a current feature is not greater than the first preset threshold, performing a cross operation on the bins of the current feature and bins of a next feature to obtain new bins, and continuing to calculate a target sample rate and a sample weight average of the new bins until all features are traversed; The sample weight is determined by at least one of the following methods: wherein, is the sample weight, is the sum of the user ratings of all training samples in the training sample set; If the user is overdue, the user overdue is determined as 1; if the user is not overdue, the user overdue is determined as 0. Wherein, sum(user overdue x loan balance) is the product sum of user overdue and loan balance of all training samples in the training sample set.

9. An electronic device comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-7.

11. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-7.

Images