Model training method and device, model identification method and device, electronic equipment and storage medium

Abstract

The invention provides a model training method and device, an identification method and device, electronic equipment and a storage medium, and relates to the technical field of machine learning. The model training method comprises the steps of training a first model based on an initial training sample, and generating a basic classification model; obtaining a classification result of the initial training sample by the basic classification model; determining a predicted positive sample in the classification result as an enhanced training sample, determining classification features of the basic classification model as training features, separating a real case and a false positive case in the predicted positive sample, determining the separated real case and false positive case as training targets, training the second model, generating an enhanced classification model, and generating a screening rule by the enhanced classification model, the screening rule is used for screening out a plurality of target subsets from the enhanced training sample, and the accuracy rate of the plurality of target subsets is greater than the target accuracy rate. By means of the technical scheme, the model recall rate can be guaranteed, and meanwhile the model recognition accuracy can be improved.

Description

Background Art

[0002] Classification models are mainly used in data mining to predict the occurrence of events and estimate the tendency of events. In actual application scenarios, classification models need to ensure a high precision rate while ensuring the recall rate. For example, a spam recognition model can prevent useful emails from being ignored only when the precision rate is very high. Another example is the customer prediction model for overdue repayments. For customers with a high risk of overdue payments, the platform sales staff will urge them to pay. If the precision of the prediction model is not high, it will affect the customer experience.

[0003] In related technologies, the accuracy of classification models is improved by optimizing algorithms or feature construction. However, algorithm-based optimization takes a long time and has no obvious effect in a short period of time. Feature-based optimization is limited by complex business scenarios and data loss, and cannot achieve ideal optimization results. Therefore, there is an urgent need for an optimization solution that can improve the accuracy of classification models while ensuring the recall rate.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a classification model training method, a classification model training device, a classification target recognition method, a classification target recognition device, an electronic device and a computer-readable storage medium, which can at least to some extent improve the problem that the classification model in the related technology cannot take into account both the model recall rate and the model recognition accuracy rate at the same time.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a classification model training method is provided, including: training a first model based on an initial training sample to generate a basic classification model; obtaining a classification result of the basic classification model for the initial training sample; determining a predicted positive sample in the classification result as an enhanced training sample, determining a classification feature of the basic classification model as a training feature, separating true positive examples and false positive examples in the predicted positive sample as a training target, training a second model to generate an enhanced classification model, so as to generate a screening rule by the enhanced classification model, wherein the screening rule is used to screen out a plurality of target subsets from the enhanced training sample, and the precision of the plurality of target subsets is greater than the target precision.

[0008] In one embodiment, determining the predicted positive samples in the classification results as enhanced training samples includes: adding sample labels to the predicted positive samples to obtain the enhanced training samples, wherein a first label is added to the true positive samples and a second label is added to the false positive samples.

[0009] In one embodiment, determining the classification features of the basic classification model as training features includes: determining the sample features of the initial training samples and the predicted probabilities corresponding to the classification results output by the basic classification model as the classification features to obtain the training features.

[0010] In one embodiment, the second model is a decision tree classification model, and the training of the second model to generate an enhanced classification model includes: calculating the information gain rate of each of the sample features based on the sample labels to determine the sample feature with the maximum information gain rate as a split attribute to generate a root node of the decision tree classification model; splitting the root node based on the predicted probability corresponding to the split attribute to generate a decision tree branch of the next layer, and dividing the corresponding subsets based on the enhanced training samples; continuing to configure the split attributes of the child nodes of the decision tree classification model based on the information gain rates of other sample features to continue to generate the decision tree branches of the next layer and divide the corresponding subsets; detecting that the decision tree classification model reaches a specified depth, generating the enhanced classification model.

[0011] In one embodiment, the specified depth is configured based on a target fitting effect of the decision tree classification model.

[0012] In one embodiment, the generation of screening rules by the enhanced classification model includes: calculating the precision of each of the subsets respectively; screening the target subset from the subset based on the relationship between the precision of the subset and the target precision, and the number of the target subsets; generating the screening rules based on the limited range of the predicted probability corresponding to the target subset, wherein the number of the target subsets is positively correlated with the recall rate of the multiple target subsets.

[0013] In one embodiment, the training of the first model based on the initial training samples to generate a basic classification model includes: dividing the initial training samples into positive samples and negative samples; extracting the first sample features of the positive samples and the second sample features of the negative samples to generate a training set based on the first sample features and the second sample features; training the first model based on the training set to obtain the basic classification model, wherein the first model includes any one of an extreme gradient boosting tree XGBoost model, a distributed gradient boosting tree LightGBM model, and a symmetric decision tree CatBoost model.

[0014] According to another aspect of the present disclosure, a classification target recognition method is provided, including: inputting the data to be recognized into a basic classification model, outputting a recognition result and a recognition probability; extracting a positive recognition result from the recognition result; inputting the positive recognition result and the recognition probability into an enhanced classification model, and filtering out a plurality of recognition target subsets from the positive recognition result based on a screening rule of the enhanced classification model, wherein the accuracy of the plurality of recognition target subsets is greater than the target accuracy.

[0015] In one embodiment, the enhanced classification model is generated based on decision tree classification model training, the forward recognition result and the recognition probability are input into the enhanced classification model, and a plurality of recognition target subsets are screened out from the forward recognition result based on the screening rules of the enhanced classification model, including: inputting the forward recognition result and the recognition probability into the enhanced classification model, so that the enhanced classification model divides the forward recognition result into a plurality of subsets to be recognized based on sample features of the forward recognition result and the recognition probability; and screening out the plurality of recognition target subsets from the plurality of subsets to be recognized based on the screening rules.

[0016] According to another aspect of the present disclosure, a classification model training device is provided, comprising: a first model training module, used to train a first model based on an initial training sample to generate a basic classification model; an acquisition module, used to obtain the classification result of the basic classification model for the initial training sample; a second model training module, used to determine the predicted positive sample in the classification result as an enhanced training sample, determine the classification feature of the basic classification model as a training feature, separate the true positive examples and the false positive examples in the predicted positive sample as a training target, train the second model to generate an enhanced classification model, so as to generate a screening rule by the enhanced classification model, and the screening rule is used to screen out multiple target subsets from the enhanced training sample, and the precision of the multiple target subsets is greater than the target precision.

[0017] According to another aspect of the present disclosure, a classification target recognition device is provided, including: an input module, used to input the data to be recognized into a basic classification model, and output a recognition result and a recognition probability; an extraction module, used to extract a positive recognition result from the recognition result; an output module, used to input the positive recognition result and the recognition probability into an enhanced classification model, and based on the screening rules of the enhanced classification model, a plurality of recognition target subsets are screened out from the positive recognition results, and the accuracy of the plurality of recognition target subsets is greater than the target accuracy.

[0018] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned classification model training methods by executing the executable instructions.

[0019] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned classification model training methods is implemented.

[0020] The classification model training and classification target recognition scheme provided by the embodiments of the present disclosure obtains an enhanced classification model by adopting the classification features of the basic classification model and the initial classification results of the basic classification model to train the enhanced model. The enhanced classification model can further separate the predicted positive samples in the initial classification results, and sort out screening rules for screening multiple target subsets with higher precision based on the separated multiple subsets, so that multiple target subsets can be screened out based on the screening rules. By screening a larger number of target subsets, the model recognition precision can be improved while ensuring the model recall rate.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0023] Figure 1 A schematic diagram showing the structure of a classification model training system in an embodiment of the present disclosure;

[0024] Figure 2 A flow chart of a classification model training method according to an embodiment of the present disclosure is shown;

[0025] Figure 3 A schematic diagram showing another classification target recognition scheme in an embodiment of the present disclosure;

[0026] Figure 4 A flowchart showing another classification model training method in an embodiment of the present disclosure is shown;

[0027] Figure 5 A schematic diagram showing an enhanced classification model in an embodiment of the present disclosure is shown;

[0028] Figure 6A flowchart of another classification model training method in an embodiment of the present disclosure is shown;

[0029] Figure 7 A schematic flow chart showing a classification target recognition method in an embodiment of the present disclosure;

[0030] Figure 8 A schematic block diagram showing a classification model identification solution in an embodiment of the present disclosure;

[0031] Fig. 9 A schematic diagram showing a classification model training device in an embodiment of the present disclosure;

[0032] Fig.10 A schematic diagram showing a classification target recognition device in an embodiment of the present disclosure;

[0033] Fig.11 A schematic diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0035] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0036] The solution provided in the present application obtains an enhanced classification model by training an enhanced model using the classification features of a basic classification model and the initial classification results of the basic classification model. The enhanced classification model can further separate the predicted positive samples in the initial classification results, and sort out screening rules for screening multiple target subsets with higher precision based on the separated multiple subsets, so that multiple target subsets can be screened out based on the screening rules. By screening a larger number of target subsets, the model recognition precision can be improved while ensuring the model recall rate.

[0037] To facilitate understanding, several nouns (abbreviations) involved in this application are first explained below.

[0038] True Positive TP (Ture Positive): The number of positive examples judged to be positive.

[0039] False Negative FN (False Negative): The number of positive examples mistakenly judged as negative.

[0040] False Positive FP (False Positive): The number of negative examples mistakenly judged as positive.

[0041] True Negative TN (True Negative): The number of negative examples judged as negative.

[0042] Precision: The ratio of the number of samples predicted by the model as True and the actual result as True to the total number of samples predicted by the model as True, that is, TP / TP+FP.

[0043] Recall: The ratio of the number of samples predicted by the model as True and the actual result as True to the total number of samples actually as True in the test data set, that is, TP / TP+FN.

[0044] Figure 1 1 is a schematic diagram of a computer system provided by an exemplary embodiment of the present application. The system includes: a plurality of terminals 120 and a server cluster 140.

[0045] The terminal 120 may be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, or the terminal 120 may also be a personal computer (PC), such as a laptop computer and a desktop computer.

[0046] Among them, the terminal 120 may be installed with an application for providing classification model training.

[0047] The terminal 120 is connected to the server cluster 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0048] The server cluster 140 is a server, or is composed of several servers, or is a virtualization platform, or is a cloud computing service center. The server cluster 140 is used to provide background services for applications that provide classification model training. Optionally, the server cluster 140 undertakes the main computing work and the terminal 120 undertakes the secondary computing work; or, the server cluster 140 undertakes the secondary computing work and the terminal 120 undertakes the main computing work; or, the terminal 120 and the server cluster 140 adopt a distributed computing architecture for collaborative computing.

[0049] In some optional embodiments, the server cluster 140 is used to store classification model training program and recognition program information.

[0050] Optionally, the client of the application installed in different terminals 120 is the same, or the client of the application installed on the two terminals 120 is the client of the same type of application of different control system platforms. Based on the different terminal platforms, the specific form of the client of the application may also be different, for example, the application client may be a mobile client, a PC client or a World Wide Web (Web) client.

[0051] Those skilled in the art will appreciate that the number of the terminals 120 may be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or a greater number. The embodiment of the present application does not limit the number and device type of the terminals.

[0052] Optionally, the system may also include a management device ( Figure 1 (not shown), the management device is connected to the server cluster 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0053] Optionally, the above-mentioned wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0054] The classification model training and classification target recognition method in this example implementation will be described in more detail below in conjunction with the accompanying drawings and embodiments.

[0055] like Figure 2 As shown, according to an embodiment of the present disclosure, a classification model training method specifically includes:

[0056] Step S202: training the first model based on the initial training samples to generate a basic classification model.

[0057] Among them, the classification model includes a basic classification model and an enhanced classification model.

[0058] The initial training samples may include true positive examples TP, false negative examples FN, false positive examples FP, and true negative examples TN.

[0059] The first model is specifically a classification algorithm model. Training the first model specifically involves respectively extracting sample features of true positive examples TP, false negative examples FN, false positive examples FP, and true negative examples TN to obtain a feature vector for each sample, and using the feature vector to perform model training to obtain a basic classification model.

[0060] Step S204, obtaining the classification results of the basic classification model for the initial training samples.

[0061] In this case, the basic classification model is used to output the predicted probability of each sample feature in addition to the classification result.

[0062] Step S206, determine the predicted positive samples in the classification results as enhanced training samples, determine the classification features of the basic classification model as training features, separate the true positive examples and the false positive examples in the predicted positive samples as training targets, train the second model, generate an enhanced classification model, and generate screening rules by the enhanced classification model, the screening rules are used to screen out multiple target subsets from the enhanced training samples, and the accuracy of the multiple target subsets is greater than the target accuracy.

[0063] The number of target subsets may be determined based on the required recall rate, the second model may be a decision tree classification model, etc., and the enhanced training samples may be further divided into a training set and a test set.

[0064] Specifically, taking risky user identification as an example, the model recognition accuracy based on the basic classification model is 71%, and the recall rate is 55%. However, the target accuracy required for the model to be put online is not less than 80%. In other words, the model recognition accuracy of the basic classification model does not meet the usage requirements. By adding an enhanced classification model, a subset with a relatively high accuracy rate is screened out from the predicted positive samples, that is, a set of risky customers that the model can identify more accurately is found to meet the target accuracy requirement.

[0065] like Figure 3 As shown, the big circle represents the entire initial training sample set, the left side is the predicted positive sample set 302, and the right side is the predicted negative sample set 304. Through the screening rule, the target subset 3022 is screened out from the predicted positive sample set on the left.

[0066] In this embodiment, the enhanced model is trained by using the classification features of the basic classification model and the initial classification results of the basic classification model to obtain the enhanced classification model. The enhanced classification model can further separate the predicted positive samples in the initial classification results, and sort out screening rules for screening multiple target subsets with higher precision based on the separated multiple subsets. Multiple target subsets can be screened based on the screening rules, and a larger number of target subsets can be screened based on the classification model composed of the basic classification model and the enhanced classification model, which can improve the model recognition precision while ensuring the model recall rate.

[0067] In one embodiment, determining the predicted positive samples in the classification results as enhanced training samples includes: adding sample labels to the predicted positive samples to obtain enhanced training samples, wherein a first label is added to a true positive sample and a second label is added to a false positive sample.

[0068] In this embodiment, a first label is added to the true positive examples in the predicted positive samples, and a second label is added to the false positive examples in the predicted positive samples, so that during the training of the second model, the true positive examples and the false positive examples are separated based on the added labels to further obtain a target subset with a higher accuracy rate.

[0069] In one embodiment, determining the classification features of the basic classification model as training features includes: determining the sample features of the initial training sample and the prediction probability corresponding to the classification result output by the basic classification model as classification features to obtain the training features.

[0070] In this embodiment, the sample characteristics characterize the characteristics of true positives and false positives respectively, and the predicted probability characterizes the probability that each sample detected by the basic classification model is actually a true positive. By determining the sample characteristics and the predicted probability as classification features, the enhanced training samples can be divided into multiple subsets based on the classification features during the training process of the second model, so that the target subset can be further filtered out from the multiple subsets based on the filtering rules to ensure that the filtered target subset meets the high-precision filtering requirements.

[0071] like Figure 4 As shown, in one embodiment, the second model is a decision tree classification model, and training the second model to generate an enhanced classification model includes:

[0072] Step S402, calculating the information gain rate of each sample feature based on the sample label, so as to determine the sample feature with the maximum information gain rate as the split attribute, so as to generate the root node of the decision tree classification model.

[0073] Among them, the information gain rate is used to characterize the quality of the splitting result when the feature is used as the splitting node of the decision tree. The larger the information gain rate, the better the splitting result.

[0074] Step S404, segmenting the root node based on the predicted probability corresponding to the split attribute, generating a decision tree branch of the next layer, and dividing the corresponding subsets based on the enhanced training samples.

[0075] Among them, for the sample features and corresponding prediction probabilities of the root node, a decision tree branch is generated based on whether the actual prediction probability of the sample is greater than the prediction probability of the root node, and the enhanced training samples are further divided into two subsets to obtain two child nodes.

[0076] Step S406, continue to configure the sub-node splitting attributes of the decision tree classification model based on the information gain rate of other sample features, so as to continue to generate the decision tree branches of the next layer and divide the corresponding subsets.

[0077] Among them, the child nodes are further split based on their predicted probabilities and corresponding subsets are obtained.

[0078] Specifically, the larger the predicted probability, the greater the probability that the corresponding sample is a true positive example, and thus the greater the model recognition accuracy of the subset consisting of such samples.

[0079] Step S408: Detect that the decision tree classification model reaches a specified depth and generate an enhanced classification model.

[0080] In this embodiment, model training is performed based on enhanced training samples with added sample labels, sample features, and the predicted probability of the initial classification model to construct a decision tree classification model as an enhanced classification model, so as to separate true positives and false positives in the predicted positive samples based on the enhanced classification model to obtain subsets in different prediction probability intervals, thereby being able to further screen out subsets with more true positives to ensure the reliability of the screening operation on the predicted positive sample set.

[0081] In one embodiment, the specified depth is configured based on a target fitting effect of the decision tree classification model.

[0082] In this embodiment, as the depth of the tree increases, the accuracy of the training set increases, but the effect of the test set decreases. Therefore, by reasonably setting the depth of the decision tree, sample features that have little effect on the precision rate but a greater impact on the recall rate can be removed to prevent the decision tree from overfitting.

[0083] In one embodiment, generating screening rules by the enhanced classification model includes: calculating the precision of each subset respectively; screening target subsets from the subsets based on the relationship between the precision of the subsets and the target precision, and the number of target subsets; generating screening rules based on a limited range of predicted probabilities corresponding to the target subsets, wherein the number of target subsets is positively correlated with the recall rates of the multiple target subsets.

[0084] In this embodiment, after calculating the recognition accuracy of each subset, by taking into account the number of target subsets and the target accuracy, it is determined which subsets can be selected to obtain a higher recognition accuracy while preventing the recall rate from decreasing too much relative to the initial classification model, so as to extract the corresponding screening rules through the screened target subsets that meet the above conditions, that is, the subsets within the limited range of prediction probabilities can make the accuracy of multiple target subsets greater than the target accuracy after being screened out, so that the classification model is reusable while ensuring simple and efficient application.

[0085] like Figure 5As shown in the figure, [x1,x2,x3,…,xn] is used to represent the sample feature vector, a0, a1, a2, a3,…,an are the corresponding prediction probabilities, [f,t] represents the predicted positive sample set, the true positive example is f, the false positive example is t, and the corresponding precision calculation formula is f / (f+t), [f1,t1],…,[fn,tn] represent the split subsets.

[0086] like Figure 5 As shown, the root node of the decision tree is constructed based on the predicted positive sample set and the sample feature x0 with the maximum information gain rate, and two subsets [f1, t1] and [f2, t2] are divided based on whether it is greater than a0. Furthermore, for [f1, t1], two subsets [f3, t3] and [f4, t4] are further divided based on whether it is greater than a1. For [f2, t2], two subsets [f5, t5] and [f6, t6] are further divided based on whether it is greater than a2, until a complete decision tree is generated to obtain an enhanced classification model.

[0087] Based on the enhanced classification model, the high-precision screening rules are used to screen out multiple target subsets. Compared with the recognition accuracy of 71% and the recall rate of 55% of the model using only basic classification, the accuracy is increased to 83% and the recall rate is 46%. While improving the accuracy, a small decrease in the recall rate is guaranteed.

[0088] In one embodiment, the first model is trained based on the initial training samples to generate a basic classification model, including: dividing the initial training samples into positive samples and negative samples; extracting the first sample features of the positive samples and the second sample features of the negative samples to generate a training set based on the first sample features and the second sample features; training the first model based on the training set to obtain the basic classification model, wherein the first model includes any one of the extreme gradient boosting tree XGBoost model, the distributed gradient boosting tree LightGBM model, and the symmetric decision tree CatBoost model.

[0089] Specifically, taking the XGBoost model as an example, xgb is an implementation of the boosting algorithm, which mainly reduces the deviation, that is, reduces the error of the model. Therefore, it uses multiple base learners, each of which is relatively simple to avoid overfitting. Through the learning of multiple learners, the difference between the model value and the actual value is continuously reduced. The model can be trained by the XGBoost classification algorithm. The sample feature data is input into the model, and a positive or negative prediction result will be output, and the corresponding probability value p will be output. Hereinafter, this classification model is referred to as the basic classification model, and the positive or negative prediction result outputted by it is called the basic classification model classification result, and the probability result outputted by it is called the basic classification model output probability value.

[0090] like Figure 6As shown, according to another embodiment of the present disclosure, the classification model training method specifically includes:

[0091] Step S602, obtaining a set of predicted positive samples identified by the basic classification model.

[0092] Among them, the true positive set in the predicted positive sample set is A1, and the false positive set is A2. The A1 samples are marked as 1, and the A2 samples are marked as 0.

[0093] Step S604: Using the probability values ​​and sample features predicted by the basic classification model as training features for the decision tree classification model.

[0094] Step S606: training a decision tree classification model to separate decision tree branches with different accuracies.

[0095] Step S608, calculating the precision of each branch subset, selecting a subset with a higher precision, and sorting out screening rules according to the tree shape of the decision tree.

[0096] Step S610, calculating the precision of each branch subset of the decision tree.

[0097] Step S612: A high-precision set of a test set or a prediction set can be found through screening rules to obtain a high-precision output result.

[0098] like Figure 7 As shown, a classification target recognition method according to an embodiment of the present disclosure includes:

[0099] Step S702: input the data to be identified into the basic classification model, and output the identification result and identification probability.

[0100] Step S704: extract the positive recognition result from the recognition results.

[0101] Step S706, inputting the forward recognition result and the recognition probability into the enhanced classification model, and filtering out multiple recognition target subsets from the forward recognition result based on the screening rules of the enhanced classification model, wherein the precision of the multiple recognition target subsets is greater than the target precision.

[0102] In this embodiment, a classification model is constructed by adopting a basic classification model and an enhanced classification model, so that the enhanced classification model further classifies and recognizes the recognition results output by the basic classification model, and the predicted positive samples in the recognition results are further separated, and based on the separated multiple subsets, screening rules for screening multiple target subsets with higher precision are sorted out. Multiple target subsets can be screened out based on the screening rules, and a larger number of target subsets can be screened out based on the classification model composed of the basic classification model and the enhanced classification model, which can improve the model recognition precision while ensuring the model recall rate.

[0103] In one embodiment, an enhanced classification model is generated based on decision tree classification model training, and the positive recognition results and recognition probabilities are input into the enhanced classification model. Based on the screening rules of the enhanced classification model, multiple recognition target subsets are screened out from the positive recognition results, including: inputting the positive recognition results and recognition probabilities into the enhanced classification model, so that the enhanced classification model divides the positive recognition results into multiple subsets to be recognized based on sample features and recognition probabilities of the positive recognition results; and screening out multiple recognition target subsets from the multiple subsets to be recognized based on the screening rules.

[0104] like Figure 8 As shown, a classification target recognition solution according to an embodiment of the present disclosure includes a data processing module 802 , a basic classification model 804 , an enhanced classification model 806 and an output module 808 .

[0105] The data processing module 802 is used to perform feature processing and label processing on the data to be input, generate multiple sample features based on the feature processing, namely x1, x2, x3 to xn, and generate a label y based on the label processing.

[0106] The XGBoost classification model is trained using the above data to obtain a basic classification model 804.

[0107] The basic classification model 804 is used to output classification results, including predicted positive samples and predicted probabilities.

[0108] The decision tree classification model is trained using sample features, predicted positive samples and predicted probabilities to obtain an enhanced classification model 806.

[0109] The enhanced classification model 806 is used to generate screening rules, and the output module 808 is used to screen out a recognition target subset from the forward recognition result based on the screening rules output by the enhanced classification model, that is, the classification result, and the accuracy of the recognition target subset is greater than the target accuracy.

[0110] In this embodiment, based on the basic classification model, the prediction results and features are used to separate the prediction results again through a decision tree, and screening rules that meet different business accuracy requirement sets are efficiently found. The prediction results are screened again through the screening rules to obtain high-precision prediction results that can be actually used. This solution is reusable, simple and efficient, and has reference significance for many data mining projects.

[0111] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0112] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0113] Refer to the following Fig. 9 To describe the classification model training device 900 according to this embodiment of the present invention. Fig. 9 The classification model training device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0114] The classification model training device 900 is in the form of a hardware module. The components of the classification model training device 900 may include but are not limited to: a first model training module 902, which is used to train the first model based on the initial training sample to generate a basic classification model; an acquisition module 904, which is used to obtain the classification results of the basic classification model for the initial training sample; a second model training module 906, which is used to determine the predicted positive sample in the classification result as the enhanced training sample, determine the classification feature of the basic classification model as the training feature, separate the true positive examples and false positive examples in the predicted positive sample as the training target, train the second model, generate an enhanced classification model, and generate a screening rule by the enhanced classification model, the screening rule is used to screen out multiple target subsets from the enhanced training sample, and the precision of the multiple target subsets is greater than the target precision.

[0115] Refer to the following Fig.10 hereinafter, the classification target recognition device 1000 according to this embodiment of the present invention is described. Fig.10 The classified object recognition device 1000 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0116] The classification target recognition device 1000 is in the form of a hardware module. The components of the classification target recognition device 1000 may include but are not limited to: an input module 1002, which is used to input the data to be recognized into the basic classification model and output the recognition result and the recognition probability; an extraction module 1004, which is used to extract the positive recognition result from the recognition result; an output module 1006, which is used to input the positive recognition result and the recognition probability into the enhanced classification model, and filter out multiple recognition target subsets from the positive recognition result based on the screening rules of the enhanced classification model, and the accuracy of the multiple recognition target subsets is greater than the target accuracy.

[0117] Refer to the following Fig.11The electronic device 1100 according to this embodiment of the present invention is described. Fig.11 The electronic device 1100 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0118] like Fig.11 As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include but are not limited to: at least one processing unit 1110, at least one storage unit 1120, and a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110).

[0119] The storage unit stores program codes, which can be executed by the processing unit 1110, so that the processing unit 1110 executes the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 1110 can execute the following steps: Figure 1 Steps S202 and S206 shown in , and other steps defined in the classification model training method of the present disclosure.

[0120] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 11201 and / or a cache storage unit 11202 , and may further include a read-only storage unit (ROM) 11203 .

[0121] The storage unit 1120 may also include a program / utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0122] Bus 1130 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0123] The electronic device 1100 may also communicate with one or more external devices 1160 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1150. As shown, the network adapter 1150 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0124] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0125] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0126] The program product for implementing the above method according to an embodiment of the present invention may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0127] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0128] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0129] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0130] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0131] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0132] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0133] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A classification model training method, characterized in that: include: Training the first model based on the initial training samples to generate a basic classification model; Obtaining classification results of the basic classification model on the initial training samples; The predicted positive samples in the classification results are determined as enhanced training samples, the classification features of the basic classification model are determined as training features, the true positive examples and the false positive examples in the predicted positive samples are separated and determined as training targets, the second model is trained to generate an enhanced classification model, and the enhanced classification model is used to generate screening rules, and the screening rules are used to screen out multiple target subsets from the enhanced training samples, and the accuracy of the multiple target subsets is greater than the target accuracy.

2. The classification model training method according to claim 1, characterized in that: The step of determining the predicted positive samples in the classification results as enhanced training samples comprises: Adding sample labels to the predicted positive samples to obtain the enhanced training samples, wherein a first label is added to the true positive samples and a second label is added to the false positive samples.

3. The classification model training method according to claim 2, characterized in that: Determining the classification features of the basic classification model as training features includes: The sample features of the initial training samples and the predicted probabilities corresponding to the classification results output by the basic classification model are determined as the classification features to obtain the training features.

4. The classification model training method according to claim 3, characterized in that: The second model is a decision tree classification model, and training the second model to generate an enhanced classification model includes: Calculating the information gain rate of each of the sample features based on the sample labels, so as to determine the sample feature with the maximum information gain rate as a split attribute, so as to generate a root node of the decision tree classification model; The root node is segmented based on the predicted probability corresponding to the split attribute to generate a decision tree branch of the next layer, and a corresponding subset is divided based on the enhanced training sample; Continue to configure the splitting attributes of the subnodes of the decision tree classification model based on the information gain rates of other sample features, so as to continue to generate the decision tree branches of the next layer and divide the corresponding subsets; It is detected that the decision tree classification model reaches a specified depth, and the enhanced classification model is generated.

5. The classification model training method according to claim 4, characterized in that: The specified depth is configured based on a target fitting effect of the decision tree classification model.

6. The classification model training method according to claim 4, characterized in that: The generation of screening rules by the enhanced classification model includes: Calculate the precision of each subset respectively; Filtering the target subset from the subset based on the relationship between the accuracy of the subset and the target accuracy, and the number of the target subsets; generating the screening rule based on the limited range of the predicted probability corresponding to the target subset, The number of the target subsets is positively correlated with the recall rates of the multiple target subsets.

7. The classification model training method according to any one of claims 1 to 6, characterized in that: The training of the first model based on the initial training sample to generate a basic classification model comprises: Dividing the initial training samples into positive samples and negative samples; Extracting a first sample feature of the positive sample and a second sample feature of the negative sample to generate a training set based on the first sample feature and the second sample feature; The first model is trained based on the training set to obtain the basic classification model. Wherein, the first model includes any one of an extreme gradient boosting tree model, a distributed gradient boosting tree model and a symmetric decision tree model.

8. A classification target recognition method, characterized in that: include: Input the data to be identified into the basic classification model and output the identification result and identification probability; Extracting positive recognition results from the recognition results; The forward recognition result and the recognition probability are input into an enhanced classification model, and a plurality of recognition target subsets are screened out from the forward recognition result based on a screening rule of the enhanced classification model, wherein the precision of the plurality of recognition target subsets is greater than the target precision.

9. The method for identifying classified targets according to claim 8, characterized in that: The enhanced classification model is generated based on the training of the decision tree classification model, the forward recognition result and the recognition probability are input into the enhanced classification model, and a plurality of recognition target subsets are screened out from the forward recognition result based on the screening rules of the enhanced classification model, including: Inputting the positive recognition result and the recognition probability into the enhanced classification model, so that the enhanced classification model divides the positive recognition result into a plurality of subsets to be recognized based on the sample features of the positive recognition result and the recognition probability; The multiple identification target subsets are screened out from the multiple subsets to be identified based on the screening rules.

10. A classification model training device, characterized in that: include: A first model training module, used to train the first model based on the initial training sample to generate a basic classification model; An acquisition module, used to obtain the classification result of the basic classification model on the initial training sample; The second model training module is used to determine the predicted positive samples in the classification results as enhanced training samples, determine the classification features of the basic classification model as training features, separate the true positive examples and the false positive examples in the predicted positive samples as training targets, train the second model, generate an enhanced classification model, and generate screening rules by the enhanced classification model, wherein the screening rules are used to screen out multiple target subsets from the enhanced training samples, and the accuracy of the multiple target subsets is greater than the target accuracy.

11. A classification target recognition device, characterized in that: include: The input module is used to input the data to be identified into the basic classification model and output the identification result and identification probability; An extraction module, used for extracting positive recognition results from the recognition results; The output module is used to input the forward recognition result and the recognition probability into the enhanced classification model, and filter out multiple recognition target subsets from the forward recognition result based on the screening rules of the enhanced classification model, wherein the accuracy of the multiple recognition target subsets is greater than the target accuracy.

12. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the classification model training method described in any one of claims 1 to 7 and / or the classification target recognition method described in claim 8 or 9 by executing the executable instructions.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the classification model training method described in any one of claims 1 to 7, and implements the classification target recognition method described in claim 8 or 9.

Images