Method and device for constructing gradient boosting decision tree (GBDT) model, and prediction method and device

Abstract

The application discloses a method and device for constructing a gradient boosting decision tree (GBDT) model, and relates to the technical field of machine learning, and mainly aims to solve the problem of low accuracy of the existing trained decision tree model. The main technical scheme of the application is as follows: obtaining a sample data set, wherein the sample data set comprises positive sample data with positive labels and unlabeled sample data without labels; when training each regression tree of the GBDT model, a positive sample training subset is constructed based on the positive sample data in the sample data set, a negative sample training subset is constructed by sampling the unlabeled sample data in the sample data set, the positive sample training subset and the multiple negative sample training subsets are combined to obtain a training set of the current regression tree, the current regression tree is trained based on the training set of the current regression tree, and a GBDT model is constructed according to each regression tree. The application is used in the construction process of the GBDT.

Description

[0001] This application is a divisional application of patent application No. 201910526406.6, filed on June 18, 2019, entitled "Method, Apparatus and Prediction Method and Apparatus for Constructing GBDT Model". Technical Field

[0002] This invention relates to the field of machine learning technology, and in particular to a method and apparatus for constructing a gradient boosting decision tree (GBDT) model, as well as a method and apparatus for making predictions using the model. Background Technology

[0003] With the continuous advancement of technology, artificial intelligence (AI) technology has also gradually developed. Machine learning, in particular, is an inevitable product of AI research reaching a certain stage. It aims to improve system performance through computational means and by leveraging experience. In computer systems, "experience" typically exists in the form of "data." Machine learning algorithms can generate "models" from this data. In other words, by providing experiential data to machine learning algorithms, models can be created based on this data. When faced with new situations, the model will provide corresponding judgments, i.e., predict the outcome. Whether training a machine learning model or using a trained model for prediction, the data needs to be converted into machine learning samples that include various features.

[0004] Currently, in real-world applications, data acquisition is relatively easy, while data labeling requires significant human and material resources. Therefore, a dataset often contains a small amount of labeled data (positive samples) and a large amount of unlabeled data. In such cases, Positive and Unlabeled Learning (PU Learning) combined with Gradient Boosting Decision Tree (GBDT) algorithms is typically used to train the decision tree model. For example, the GBDT algorithm can be selected to train the corresponding GBDT model of the gradient boosting decision tree based on the sample data.

[0005] However, in practical applications, when training decision tree models based on PU learning, there are relatively few labeled "positive samples" in the sample data, and most of the data is unlabeled. Therefore, "overfitting" is very likely to occur when training gradient boosting decision tree models. Overfitting refers to the phenomenon of making the hypothesis too strict in order to obtain a consistent hypothesis, which leads to the low accuracy of decision tree models trained by the existing methods. Summary of the Invention

[0006] In view of the above problems, the present invention proposes a method and apparatus for constructing a gradient boosting decision tree (GBDT) model. The main purpose is to solve the problem of low accuracy of existing trained decision tree models and improve the accuracy of the trained model.

[0007] To achieve the above objectives, the present invention mainly provides the following technical solutions:

[0008] On the one hand, this invention provides a method for constructing a gradient boosting decision tree (GBDT) model, specifically including:

[0009] Obtain a sample dataset, which includes positively labeled positive sample data and unlabeled unlabeled sample data;

[0010] When training each regression tree of the GBDT model, a positive sample training subset is constructed based on the positive sample data in the sample dataset, and a negative sample training subset is constructed by sampling the unlabeled sample data in the sample dataset. The positive sample training subset and the multiple negative sample training subsets are combined to obtain the training set of the current regression tree. The current regression tree is trained based on the training set of the current regression tree, and then the gradient boosting decision tree GBDT model is constructed based on each regression tree.

[0011] Optionally, constructing a positive sample training subset based on the positive sample data in the sample dataset includes:

[0012] Take all positive sample data from the sample dataset to construct a positive sample training subset;

[0013] or,

[0014] A positive sample training subset is constructed by taking a portion of the positive sample data from the sample dataset.

[0015] Optionally, in a business scenario where the ratio of positive to negative samples is known, if the estimated ratio of negative sample data to positive sample data is x, then the data volume of the negative sample training subset is x times the data volume of the positive sample training subset; in a business scenario where the ratio of positive to negative samples is unknown, then the data volume of the negative sample training subset is 1 to 2 times the data volume of the positive sample training subset.

[0016] Optionally, training the current regression tree based on the training set includes:

[0017] The regression tree is iteratively trained using the training set of the current regression tree and the preset GBDT algorithm to obtain each regression tree corresponding to each iteration.

[0018] Optionally, the step of iteratively training the current regression tree using its training set and a preset GBDT algorithm to obtain each regression tree for each iteration includes:

[0019] A first training set is obtained from the sample dataset, and a first regression tree is trained based on the first training set, a preset GBDT algorithm, and a first parameter, wherein the first parameter is the mean of the actual results of all sample data in the sample dataset.

[0020] After the first regression tree is obtained through training, a second training set is selected from the sample training set, and a second regression tree is trained based on the second training set, the preset GBDT algorithm, and the second parameters. The second parameters are determined by the prediction results determined by the first regression tree based on the sample data in the second training set and the actual results of the sample data in the second training set. Both the first and second training sets consist of a positive sample training subset and a negative sample training subset. The negative sample training subset contained in the first training set is different from that in the second training set.

[0021] Optionally, before selecting a second training set from the sample training set and training a second regression tree based on the second training set, a preset GBDT algorithm, and second parameters, the method further includes:

[0022] The prediction results corresponding to the second training set are obtained by using the first regression tree to predict the second training set.

[0023] Based on the actual results of the second training set and the prediction results corresponding to the second training set, the residual between the actual results and the prediction results is determined, and the residual is determined as the second parameter.

[0024] Optionally, the method further includes:

[0025] Multiple training sets were constructed based on the aforementioned sample dataset;

[0026] Multiple candidate models are obtained by selecting from the set of machine learning algorithms, the set of hyperparameter combinations, and the multiple training sets. A candidate model is determined by a machine learning algorithm, a set of hyperparameters, and a training set.

[0027] At least one GBDT model and the plurality of candidate models are evaluated respectively, and a plurality of models that meet the preset conditions are selected.

[0028] The multiple models that meet the preset conditions are integrated to obtain the integrated composite model.

[0029] Optionally, before evaluating at least one of the GBDT models and the plurality of candidate models respectively, the method further includes:

[0030] A positive sample evaluation subset is constructed by sampling positive sample data in the sample dataset, and a negative sample evaluation subset is constructed by sampling unlabeled sample data in the sample dataset. The positive sample evaluation subset and the negative sample evaluation subset are combined to obtain the evaluation set.

[0031] The step of evaluating at least one GBDT model and the plurality of candidate models respectively, and selecting a plurality of models that meet preset conditions, includes:

[0032] The at least one GBDT model and the plurality of candidate models are evaluated according to the evaluation set to obtain the evaluation result for each model, and a plurality of models that meet the preset conditions are selected from the evaluation results.

[0033] Optionally, the sample dataset includes: target object data for target object recommendation, transaction data of stolen bank cards for bank card leakage detection, image / text data for image / text classification, and traffic data for malicious traffic detection;

[0034] Specifically, when the sample dataset is target object data, the recommended target object data is positive sample data, and the unrecommended target object data is negative sample data; when the sample dataset is transaction data of stolen bank cards, the transaction data with known leakage points is positive sample data, and the transaction data with unknown leakage points is negative sample data; when the sample dataset is image / text data, the classified image / text data is positive sample data, and the unclassified image / text data is negative sample data; when the sample dataset is traffic data, the known malicious traffic data is positive sample data, and the unknown traffic data is negative sample data.

[0035] On the other hand, the present invention also provides an apparatus for constructing a gradient boosting decision tree (GBDT) model, wherein the method includes:

[0036] The acquisition unit is used to acquire a sample dataset, which includes positively labeled positive sample data and unlabeled unlabeled sample data;

[0037] The construction unit is used to construct a positive sample training subset based on the positive sample data in the sample dataset when training each regression tree of the GBDT model, to sample the unlabeled sample data in the sample dataset to construct a negative sample training subset, to combine the positive sample training subset with the multiple negative sample training subsets to obtain the training set of the current regression tree, to train the current regression tree based on the training set of the current regression tree, and then to construct a gradient boosting decision tree GBDT model based on each regression tree.

[0038] Optionally, the building unit includes:

[0039] The first construction module is used to construct a positive sample training subset by taking all positive sample data from the sample dataset, or to construct a positive sample training subset by taking a portion of the positive sample data from the sample dataset.

[0040] Optionally, the building unit includes:

[0041] The second construction module is used to, when the ratio of positive to negative samples is known in a business scenario, and the estimated ratio of negative sample data to positive sample data is x, make the data volume of the negative sample training subset x times the data volume of the positive sample training subset; when the ratio of positive to negative samples is unknown in a business scenario, make the data volume of the negative sample training subset 1 to 2 times the data volume of the positive sample training subset.

[0042] Optionally, the building unit includes:

[0043] The training module is used to perform iterative training using the training set of the current regression tree and the preset GBDT algorithm to obtain each regression tree corresponding to each iteration of training.

[0044] Optionally, the training module includes:

[0045] The first training submodule is used to obtain a first training set from the sample dataset and train a first regression tree based on the first training set, a preset GBDT algorithm, and a first parameter, wherein the first parameter is the mean of the actual results of all sample data in the sample dataset.

[0046] The second training submodule is used to select a second training set from the sample training set after the first regression tree is obtained through training, and to train a second regression tree based on the second training set, the preset GBDT algorithm, and the second parameters. The second parameters are determined by the prediction results determined by the first regression tree based on the sample data in the second training set and the actual results of the sample data in the second training set. Both the first training set and the second training set consist of a positive sample training subset and a negative sample training subset. The negative sample training subset contained in the first training set is different from that in the second training set.

[0047] Optionally, the training module further includes:

[0048] The prediction submodule is used to predict the second training set using the first regression tree to obtain the prediction result corresponding to the second training set.

[0049] The determination submodule is used to determine the residual between the actual result and the prediction result based on the actual result of the second training set and the prediction result corresponding to the second training set, and to determine the residual as the second parameter.

[0050] Optionally, the device further includes:

[0051] A training set construction unit is used to construct multiple training sets based on the sample dataset;

[0052] The training unit is used to select from the set of machine learning algorithms, the set of hyperparameter combinations, and the multiple training sets to train multiple candidate models, wherein a machine learning algorithm, a set of hyperparameters, and a training set determine a candidate model.

[0053] An evaluation unit is used to evaluate at least one GBDT model and the plurality of candidate models respectively, and select a plurality of models that meet preset conditions;

[0054] An integration unit is used to integrate multiple models that meet preset conditions to obtain an integrated composite model.

[0055] Optionally, the device further includes:

[0056] The evaluation set construction unit is used to sample positive sample data in the sample dataset to construct a positive sample evaluation subset, sample unlabeled sample data in the sample dataset to construct a negative sample evaluation subset, and combine the positive sample evaluation subset and the negative sample evaluation subset to obtain the evaluation set.

[0057] The evaluation unit is specifically used to evaluate the at least one GBDT model and the plurality of candidate models according to the evaluation set, obtain the evaluation result for each model, and select a plurality of models that meet the preset conditions from the evaluation results.

[0058] Optionally, the sample dataset includes: target object data for target object recommendation, transaction data of stolen bank cards for bank card leakage detection, image / text data for image / text classification, and traffic data for malicious traffic detection;

[0059] Specifically, when the sample dataset is target object data, the recommended target object data is positive sample data, and the unrecommended target object data is negative sample data; when the sample dataset is transaction data of stolen bank cards, the transaction data with known leakage points is positive sample data, and the transaction data with unknown leakage points is negative sample data; when the sample dataset is image / text data, the classified image / text data is positive sample data, and the unclassified image / text data is negative sample data; when the sample dataset is traffic data, the known malicious traffic data is positive sample data, and the unknown traffic data is negative sample data.

[0060] On the other hand, the present invention provides a method for implementing target object recommendation, comprising:

[0061] Obtain data on the target object to be predicted;

[0062] The gradient boosting decision tree GBDT model is obtained according to the method described in any one of the first aspects;

[0063] The obtained gradient boosting decision tree (GBDT) model is used to perform a target object recommendation task.

[0064] The target object is a product or service provided via the Internet.

[0065] On the other hand, the present invention provides a method for detecting bank card leakage points, including:

[0066] Obtain transaction data of stolen bank cards from the leak points to be detected;

[0067] According to the method described in any one of the first aspects, the gradient boosting decision tree GBDT model is obtained;

[0068] The obtained gradient boosting decision tree (GBDT) model is used to perform a bank card leakage detection task;

[0069] The transaction dataset contains the transaction time and terminal device identifier for each transaction for each bank card.

[0070] On the other hand, embodiments of the present invention provide a method for implementing image / text classification, including:

[0071] Obtain the image / text data to be predicted;

[0072] According to the method described in any one of the first aspects, the gradient boosting decision tree GBDT model is obtained;

[0073] The resulting gradient boosting decision tree (GBDT) model is used to perform image / text classification tasks.

[0074] On the other hand, embodiments of the present invention also provide a method for detecting malicious traffic, including:

[0075] Obtain the traffic data to be detected;

[0076] According to the method described in any one of the first aspects, a gradient boosting decision tree (GBDT) model is obtained;

[0077] The obtained gradient boosting decision tree (GBDT) model is used to perform a detection task on the traffic data to be detected.

[0078] On the other hand, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by one or more computing devices, implements the method described in any of the preceding claims.

[0079] On the other hand, the present invention provides a system including one or more computing devices and one or more storage devices, wherein a computer program is recorded on the one or more storage devices, and the computer program, when executed by the one or more computing devices, causes the one or more computing devices to perform any of the methods described above.

[0080] By employing the above technical solution, the present invention provides a method and apparatus for constructing a Gradient Boosting Decision Tree (GBDT) model. This method acquires a sample dataset, and then, when training each regression tree of the GBDT model, constructs a positive sample training subset based on the positive sample data in the sample dataset, and constructs a negative sample training subset by sampling the unlabeled sample data in the sample dataset. The positive sample training subset is combined with the multiple negative sample training subsets to obtain the training set for the current regression tree. The current regression tree is then trained based on the training set of the current regression tree. Finally, a Gradient Boosting Decision Tree (GBDT) model is constructed based on each regression tree. Compared to existing technologies, the present invention can train the regression tree of each GBDT model using the positive and negative sample training subsets obtained from the sample dataset. Since the current regression tree training set used to train each regression tree is extracted from the sample training set, the differences between multiple trees can be guaranteed, avoiding the overfitting problem caused by existing training methods, thereby improving the accuracy of the trained Gradient Boosting Decision Tree (GBDT) model.

[0081] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0082] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0083] Figure 1 The flowchart illustrates a method for constructing a gradient boosting decision tree (GBDT) model according to an embodiment of the present invention.

[0084] Figure 2 The flowchart of another method for constructing a gradient boosting decision tree (GBDT) model proposed in an embodiment of the present invention is shown;

[0085] Figure 3 This diagram illustrates the components of a device for constructing a gradient boosting decision tree (GBDT) model according to an embodiment of the present invention.

[0086] Figure 4 This invention presents a block diagram of another device for constructing a gradient boosting decision tree (GBDT) model according to an embodiment of the present invention.

[0087] Figure 5 A block diagram of a system for implementing target object recommendation according to an embodiment of the present invention is shown;

[0088] Figure 6 This diagram illustrates the composition of a system for detecting bank card leakage points according to an embodiment of the present invention.

[0089] Figure 7 A block diagram of a system for implementing image / text classification according to an embodiment of the present invention is shown;

[0090] Figure 8 The diagram shows a block diagram of a system for detecting malicious traffic according to an embodiment of the present invention. Detailed Implementation

[0091] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0092] This invention provides a method for constructing a Gradient Boosting Decision Tree (GBDT) model. This method can be applied to scenarios such as bank card fraud detection, product and service recommendation, image or text classification, and malicious traffic detection. In these scenarios, labeled positive sample data constitutes a small portion, while the majority of the sample data is unlabeled. The method described in this invention aims to address the low accuracy problem caused by multifitting in existing models by constructing a highly accurate GBDT model. The specific steps of this method are as follows: Figure 1 As shown, it includes:

[0093] 101. Obtain the sample dataset.

[0094] The sample dataset includes positively labeled data and unlabeled data. In many real-world scenarios, data is readily available, but labeling it requires significant human and material resources. For example, in malicious traffic detection, bank data breach detection, and music recommendation, only a small amount of positive data (known malicious traffic, breaches, and user-preferred music) and a large amount of unlabeled data are often obtained. In such cases, PU (Programmable Logic Provider) learning can typically be used to train a model, enabling the trained model to analyze and automatically classify the data in these scenarios.

[0095] PU (Positive and unlabeled learning) refers to training a classification model using only positive and unlabeled sample data. Previous studies typically used negative class samples from the unlabeled samples to train the classifier; however, when only positive sample data is available, it is difficult to select reliable results for the model and its parameters.

[0096] Based on this, and addressing the problems existing in the prior art, in this embodiment of the invention, a sample dataset is first obtained according to the method described in this step. This sample dataset includes positively labeled positive sample data and unlabeled, unlabeled sample data. For example, in practical applications, when the task being performed is to detect malicious traffic, the positive sample data in the sample dataset can be understood as known malicious traffic data, while the unlabeled sample data can be understood as traffic data that has not been detected.

[0097] 102. When training each regression tree of the GBDT model, a positive sample training subset is constructed based on the positive sample data in the sample dataset, and a negative sample training subset is constructed by sampling the unlabeled sample data in the sample dataset. The positive sample training subset and the multiple negative sample training subsets are combined to obtain the training set of the current regression tree. The current regression tree is trained based on the training set of the current regression tree, and then a gradient boosting decision tree GBDT model is constructed based on each regression tree.

[0098] GBDT (Gradient Boosting Decision Tree), also known as MART (Multiple Additive Regression Tree), is an iterative decision tree algorithm. It consists of multiple decision trees, and the conclusions of all trees are summed to make the final prediction. When it was first proposed, it was considered, along with SVM, to be an algorithm with strong generalization ability. In recent years, it has attracted more attention because it has been used in machine learning models for search ranking.

[0099] Therefore, based on the characteristics of the GBDT algorithm, in this embodiment of the invention, when training the gradient boosting decision tree (GBDT) model, each regression tree is trained using a training set. Specifically, existing decision tree models often suffer from overfitting due to the small number of training samples. Therefore, we can first construct a separate training set for each regression tree to be trained, using both positive and negative samples from the sample dataset. This training set can be constructed by using positive sample data to create a positive sample training subset and sampling from unlabeled sample data to obtain a corresponding negative sample training subset. The positive and negative sample training subsets are then combined to obtain the training set required for the regression tree to be trained, i.e., the training set for the current regression tree.

[0100] After determining the training set for the current regression tree, the corresponding tree can be trained using the GBDT algorithm. This process is repeated iteratively to obtain all regression trees corresponding to the training set. The regression trees are then combined to obtain the gradient boosting decision tree GBDT model corresponding to the training set.

[0101] To better illustrate the method for constructing a gradient boosting decision tree (GBDT) model provided in the embodiments of the present invention, another embodiment is also provided to refine and expand upon the steps in the above embodiments. Specifically, as follows: Figure 2 As shown, it includes:

[0102] 201. Obtain the sample dataset.

[0103] The sample dataset includes positively labeled positive sample data and unlabeled unlabeled sample data.

[0104] Specifically, in practical applications, the sample dataset may include: target object data in target object recommendation scenarios; transaction data of stolen bank cards in bank card leakage detection scenarios; image / text data in image / text classification scenarios; and traffic data in malicious traffic detection scenarios.

[0105] In particular, based on different practical application scenarios, the positive sample dataset and unlabeled sample data in each sample dataset also vary:

[0106] For example, when the sample dataset is target object data, the target object data that has been recommended is positive sample data, and the target object data that has not been recommended is negative sample data;

[0107] When the sample dataset consists of transaction data from stolen bank cards, the transaction data with known leakage points are positive sample data, and the transaction data with unknown leakage points are negative sample data.

[0108] When the sample dataset is image / text data, the classified image / text data is positive sample data, and the unclassified image / text data is negative sample data;

[0109] When the sample dataset is traffic data, known malicious traffic data in the traffic data is positive sample data, and unknown traffic data is negative sample data.

[0110] 202. When training each regression tree of the GBDT model, a positive sample training subset is constructed based on the positive sample data in the sample dataset, and a negative sample training subset is constructed by sampling the unlabeled sample data in the sample dataset. The positive sample training subset and the multiple negative sample training subsets are combined to obtain the training set of the current regression tree. The current regression tree is trained based on the training set of the current regression tree, and the gradient boosting decision tree GBDT model is constructed based on each regression tree.

[0111] Specifically, when constructing a positive sample training subset, that is, constructing a positive sample training subset based on the positive sample data in the sample dataset, the construction process may include: taking all positive sample data from the sample dataset to construct a positive sample training subset, or taking a portion of the positive sample data from the sample dataset to construct a positive sample training subset. Simultaneously, when constructing a negative sample subset by sampling unlabeled sample data, the amount of data in the negative sample training subset is 1 to 2 times the amount of data in the positive sample training subset. For example, if the positive sample training subset contains 1000 data points, the number of data points in the negative sample training subset can be 1000 to 2000 data points. This ensures that when training the regression tree of each GBDT model, there is a training set containing different sample data for training, thereby ensuring the diversity of each tree. Furthermore, when the amount of data in the negative sample training subset is 1 to 2 times that of the positive sample training subset, compared to less than 1 times, it ensures that there is enough sample data in the training set, thereby ensuring higher accuracy of the trained model. At the same time, compared to more than 2 times, the large number of samples leads to a longer training time. Therefore, selecting a negative sample training subset with an amount of data that is 1 to 2 times that of the positive sample training subset can reduce the time consumed in the model training process while maintaining model accuracy, thus improving the efficiency of model training. It should be noted that in the process of determining the training set described above, the method for selecting training subsets of positive and negative samples in the above manner is implemented when the ratio of positive to negative samples is unknown in a business scenario. In another scenario, when the ratio of positive to negative samples in the sample data is known in a specific implementation scenario, or when the ratio between positive and negative samples in the entire sample data can be determined based on historical data, the positive and negative sample training subsets can be determined by the known ratio of positive to negative samples during the selection of negative samples. That is, when the ratio of positive to negative samples is known in a business scenario, and the estimated ratio of the amount of negative sample data to the amount of positive sample data is x, then the amount of data in the negative sample training subset is x times the amount of data in the positive sample training subset.

[0112] Once the training set for the current regression tree is determined, iterative training is performed using the training set and the preset GBDT algorithm to obtain each regression tree corresponding to each iteration. Specifically, the execution method for iteratively training each regression tree can be as follows:

[0113] First, the first regression tree is trained. Specifically, a first training set is obtained from the sample dataset, and the first regression tree is trained based on the first training set, a preset GBDT algorithm, and first parameters, where the first parameter is the mean of the actual results of all sample data in the sample dataset.

[0114] Then, subsequent regression trees are trained based on the first regression tree. Specifically, after the first regression tree is trained, a second training set is selected from the sample training set, and a second regression tree is trained based on the second training set, the preset GBDT algorithm, and the second parameters.

[0115] Here, the second parameter is determined by the prediction results obtained from the first regression tree and the actual results from the sample data in the second training set. Both the first and second training sets consist of a positive sample training subset and a negative sample training subset, but the negative sample training subset included in the first training set is different from that in the second training set. That is, when training the first regression tree, the average value of the actual results can be calculated from the sample training set first, and this average value can be determined as the first parameter. Then, the corresponding first tree, i.e., the first regression tree, can be trained based on this first parameter.

[0116] After training the first regression tree, the second parameter can be determined using this first regression tree. Specifically, firstly, the first regression tree is used to predict the second training set to obtain the prediction result corresponding to the second training set. Then, based on the actual result of the second training set and the prediction result corresponding to the second training set, the residual between the actual result and the prediction result is determined, and the residual is determined as the second parameter. In this embodiment of the invention, the first regression tree can be understood as the root regression tree. After training the first regression tree, multiple subsequent second regression trees can be trained iteratively. The second parameter in each iteration is calculated as the residual obtained by calculating the difference between the prediction result of the regression tree obtained in the previous iteration and the actual result of the training set in the current iteration.

[0117] For example, if the actual results of sample data A, sample data B, and sample data C in the training set are 6, 11, and 4 respectively, then according to the method in this step, the first parameter of the first regression tree can be determined to be the average of the three actual results, which is 7. In this way, the first regression tree can be trained based on the first parameter 7. After the first regression tree is determined, it can be used to predict sample data A. When the predicted result is 5, since the actual result of sample data A is 6, the difference between the actual result and the predicted result can be calculated to obtain the residual 1. This residual 1 is then used as the second parameter in the training process of the second regression tree.

[0118] 203. Construct multiple training sets based on the aforementioned sample dataset.

[0119] In the actual PU learning process, since positive samples are limited in the sample data, to obtain a more accurate model, the existing GBDT model can be supplemented by constructing models based on different algorithms. Furthermore, different models can be trained using different algorithms and training sets. Therefore, in this step, a training set can be constructed from the sample dataset for subsequent training of other models.

[0120] Specifically, the construction of a training set can be performed as follows: First, a positive sample training subset is constructed based on at least a portion of the positive sample data in the sample dataset, and multiple negative sample training subsets are constructed by sampling the unlabeled sample data in the sample dataset multiple times. Then, the positive sample training subsets and the multiple negative sample training subsets are combined to obtain multiple training sets. Alternatively, in constructing the positive sample training subsets in the training set, a single positive sample training set can be constructed as described above, or a portion of positive samples can be extracted from the sample dataset to construct the training set. Specifically, this can be done as follows: First, multiple positive sample training subsets are constructed based on at least a portion of the positive sample data in the sample dataset, and multiple negative sample training subsets are constructed by sampling the unlabeled sample data in the sample dataset multiple times. Then, each positive sample training subset is combined with the multiple negative sample training subsets to obtain multiple training sets.

[0121] 204. Select from the set of machine learning algorithms, the set of hyperparameter combinations, and the multiple training sets respectively to train multiple candidate models.

[0122] Specifically, the machine algorithm can be selected from a pre-defined set of machine algorithms, and the hyperparameters can be obtained from a set of hyperparameter combinations. Here, the candidate model can be determined by a machine learning algorithm, combined with a set of selected hyperparameters and a training set from multiple corresponding training sets. Specifically, a machine learning algorithm, a set of hyperparameters, and a training set determine a candidate model.

[0123] For example, the set of machine learning algorithms is: [Algorithm 1, Algorithm 2, Algorithm 3], the set of hyperparameter combinations is: [Hyperparameter combination 1, Hyperparameter combination 2, ..., Hyperparameter combination 10], and the training set includes: Training Set 1, Training Set 2, ..., Training Set 8. Then, selecting "Algorithm 1 + Hyperparameter combination 1 + Training Set 1" will determine one candidate decision tree model; selecting "Algorithm 2 + Hyperparameter combination 1 + Training Set 1" will determine another candidate decision tree model; selecting "Algorithm 1 + Hyperparameter combination 2 + Training Set 1" will determine yet another candidate decision tree model; selecting "Algorithm 1 + Hyperparameter combination 1 + Training Set 2" will determine yet another candidate decision tree model, and so on.

[0124] 205. Sample the positive sample data in the sample dataset to construct a positive sample evaluation subset, sample the unlabeled sample data in the sample dataset to construct a negative sample evaluation subset, and combine the positive sample evaluation subset and the negative sample evaluation subset to obtain an evaluation set.

[0125] Based on the aforementioned steps, multiple candidate models are obtained. These models have varying accuracies. Therefore, in this embodiment of the invention, it is necessary to evaluate these candidate models to obtain a relatively accurate model. Thus, the construction of the evaluation set based on the sample dataset can be specifically performed as follows: sampling positive sample data from the sample dataset to construct a positive sample evaluation subset; sampling unlabeled sample data from the sample dataset to construct a negative sample evaluation subset; and combining the positive and negative sample evaluation subsets to obtain the evaluation set. Furthermore, to further improve the accuracy of the evaluation results, multiple evaluation sets can be constructed in this step so that each candidate model can be evaluated multiple times using these multiple evaluation sets, and the overall evaluation effect can be determined based on the results of these multiple evaluations. Therefore, the construction of the evaluation set based on the sample dataset can also be specifically performed as follows: constructing multiple evaluation sets based on the sample dataset, where each evaluation set includes positive sample data and unlabeled sample data as negative sample data.

[0126] 206. Evaluate the at least one GBDT model and the plurality of candidate models according to the evaluation set to obtain the evaluation result for each model, and select a plurality of models that meet the preset conditions from the evaluation results.

[0127] When multiple evaluation sets are constructed, the process of selecting models that meet preset conditions from the evaluation results can be carried out as follows: First, for each candidate model, the candidate model is evaluated according to the multiple evaluation sets and preset evaluation conditions, resulting in multiple evaluation results. Then, the multiple evaluation results for each candidate model are fused, and the final evaluation result corresponding to the candidate model obtained by fusing the multiple evaluation results is taken as the actual evaluation result.

[0128] It should be noted that different preset evaluation conditions directly affect the evaluation method and results. Therefore, the evaluation results will differ depending on the preset evaluation conditions. For example, when the preset evaluation condition is the maximum margin method, the evaluation result for each candidate model is the classification margin of its prediction on the evaluation set. When the preset evaluation condition is the AUC value calculation method, the evaluation result for each candidate model is its AUC value on the evaluation set. The AUC value can be understood as a probability value. When you randomly select a positive and a negative sample, the probability that the current classification algorithm will rank the positive sample ahead of the negative sample based on the calculated score is the AUC value. The larger the AUC value, the more likely the current classification model is to rank the positive sample ahead of the negative sample, thus achieving better classification and a more accurate classification effect.

[0129] 207. Integrate the multiple models that meet the preset conditions to obtain the integrated composite model.

[0130] Since there are often multiple candidate models that meet the preset conditions when selecting candidate models whose evaluation results meet the preset conditions, and the accuracy of the above candidate models is not the same, in order to further ensure the accuracy of the obtained models, in this case, it is necessary to integrate the above models. The integration process can be as follows: assign corresponding weight values ​​to each selected candidate model according to the corresponding evaluation results, and integrate the selected candidate models according to the weight values.

[0131] This ensures that by integrating models that meet the prediction conditions, a final conforming model can be obtained. Based on the obtained GBDT model, the overfitting problem of the model can be further improved, thereby ensuring that the obtained model has good prediction accuracy.

[0132] Furthermore, the above-mentioned method for constructing the Gradient Boosting Decision Tree (GBDT) model is not only about obtaining a more accurate model, but also about its practical application in real-world scenarios to solve real-world problems, such as target object recommendation, bank card leakage detection, image / text classification, and malicious traffic detection. Therefore, the process of combining the above methods to solve the above problems can be illustrated as follows.

[0133] First, acquire the data to be predicted. This data may include: image / text data to be classified, transaction data of fraudulently used bank cards to be detected, target object data to be predicted, and traffic data to be detected. Specifically, the data to be predicted varies depending on the application scenario.

[0134] Then, the Gradient Boosting Decision Tree (GBDT) model is trained according to any of the aforementioned embodiments to obtain the GBDT model. Specifically, the execution process may be as follows: Obtain a sample dataset; then, when training each regression tree of the GBDT model, construct a positive sample training subset based on the positive sample data in the sample dataset; sample the unlabeled sample data in the sample dataset to construct a negative sample training subset; combine the positive sample training subset with the multiple negative sample training subsets to obtain the training set of the current regression tree; train the current regression tree based on the training set of the current regression tree; and then construct the GBDT model based on each regression tree. For example, when this method is executed in a malicious traffic scenario, the sample dataset obtained in this step is the traffic data during malicious traffic detection, where known malicious traffic data in the traffic data is positive sample data, and unknown traffic data is negative sample data.

[0135] Finally, the obtained Gradient Boosting Decision Tree (GBDT) model is used to perform a prediction task, which corresponds to the data to be predicted obtained in the preceding steps. For example, when the data to be predicted is traffic data to be detected in a malicious traffic detection scenario, the prediction task finally performed according to the GBDT model is the malicious traffic detection task.

[0136] Furthermore, the above examples can also be applied to scenarios involving recommendations for target objects. Additionally, as an implementation of the above-mentioned method for constructing a gradient boosting decision tree (GBDT) model, this embodiment of the invention provides a device for constructing a gradient boosting decision tree (GBDT) model. This device is mainly used to improve the problem of low model accuracy caused by overfitting and to improve the accuracy of the trained GBDT model. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiments, but it should be understood that the device in this embodiment can correspondingly implement all the contents of the aforementioned method embodiments. This device, as... Figure 3 As shown, it specifically includes:

[0137] The acquisition unit 31 can be used to acquire a sample dataset, which includes positively labeled positive sample data and unlabeled sample data.

[0138] The construction unit 32 can be used to construct a positive sample training subset based on the positive sample data in the sample dataset obtained by the acquisition unit 31 when training each regression tree of the GBDT model, to sample the unlabeled sample data in the sample dataset obtained by the acquisition unit 31 to construct a negative sample training subset, to combine the positive sample training subset with the multiple negative sample training subsets to obtain the training set of the current regression tree, to train the current regression tree based on the training set of the current regression tree, and then to construct a gradient boosting decision tree GBDT model based on each regression tree.

[0139] Furthermore, such as Figure 4 As shown, the building unit 32 includes:

[0140] The first construction module 321 can be used to construct a positive sample training subset by taking all positive sample data in the sample dataset, or it can be used to construct a positive sample training subset by taking a portion of the positive sample data in the sample dataset.

[0141] Furthermore, such as Figure 4 As shown, the building unit 32 includes:

[0142] The second construction module 322 can be used to make the data volume of the negative sample training subset x times the data volume of the positive sample training subset when the ratio of positive to negative samples is known in a business scenario; and to make the data volume of the negative sample training subset 1 to 2 times the data volume of the positive sample training subset when the ratio of positive to negative samples is unknown in a business scenario.

[0143] Furthermore, such as Figure 4 As shown, the building unit 32 includes:

[0144] The training module 323 can be used to perform iterative training on the training set of the current regression tree obtained by the first construction module and the second construction module, and the preset GBDT algorithm, to obtain each regression tree corresponding to each iteration of training.

[0145] Furthermore, such as Figure 4 As shown, the training module 323 includes:

[0146] The first training submodule 3231 can be used to obtain a first training set from the sample dataset and train a first regression tree based on the first training set, a preset GBDT algorithm, and a first parameter, wherein the first parameter is the mean of the actual results of all sample data in the sample dataset.

[0147] The second training submodule 3232 can be used to select a second training set from the sample training set after the first training submodule 3231 has trained the first regression tree, and train a second regression tree based on the second training set, the preset GBDT algorithm, and the second parameters. The second parameters are determined by the prediction results determined by the first regression tree based on the sample data in the second training set and the actual results of the sample data in the second training set. Both the first training set and the second training set consist of a positive sample training subset and a negative sample training subset. The negative sample training subset contained in the first training set is different from that in the second training set.

[0148] Furthermore, such as Figure 4 As shown, the training module 323 further includes:

[0149] The prediction submodule 3233 can be used to predict the second training set using the first regression tree obtained by the first training submodule 3231, and obtain the prediction result corresponding to the second training set.

[0150] The determination submodule 3234 can be used to determine the residual between the actual result and the prediction result based on the actual result of the second training set and the prediction result of the prediction submodule 3233 for the second training set, and to determine the residual as the second parameter so that the second training submodule 3232 can train the second regression tree based on the second parameter.

[0151] Furthermore, such as Figure 4 As shown, the device further includes:

[0152] The training set construction unit 33 can be used to construct multiple training sets based on the sample dataset obtained by the acquisition unit 31;

[0153] Training unit 34 can be used to select from the set of machine learning algorithms, the set of hyperparameter combinations and the multiple training sets obtained by training set construction unit 33 respectively, and train to obtain multiple candidate models, wherein a machine learning algorithm, a set of hyperparameters and a training set determine a candidate model.

[0154] Evaluation unit 35 can be used to evaluate at least one GBDT model constructed by construction unit 32 and multiple candidate models trained by training unit 34 respectively, and select multiple models that meet preset conditions.

[0155] The integration unit 36 ​​can be used to integrate multiple models that meet preset conditions after evaluation by the evaluation unit 35 to obtain an integrated composite model.

[0156] Furthermore, such as Figure 4 As shown, the device further includes:

[0157] The evaluation set construction unit 37 can be used to sample positive sample data in the sample dataset obtained by the acquisition unit 31 to construct a positive sample evaluation subset, sample unlabeled sample data in the sample dataset to construct a negative sample evaluation subset, and combine the positive sample evaluation subset and the negative sample evaluation subset to obtain an evaluation set.

[0158] The evaluation unit 35 can be specifically used to evaluate the at least one GBDT model and the multiple candidate models respectively according to the evaluation set obtained by the evaluation set construction unit 37, to obtain the evaluation result for each model, and to select multiple models that meet the preset conditions from the evaluation results.

[0159] Furthermore, such as Figure 4 As shown, the sample dataset includes: target object data for target object recommendation, transaction data of stolen bank cards for bank card leakage detection, image / text data for image / text classification, and traffic data for malicious traffic detection;

[0160] Specifically, when the sample dataset is target object data, the recommended target object data is positive sample data, and the unrecommended target object data is negative sample data; when the sample dataset is transaction data of stolen bank cards, the transaction data with known leakage points is positive sample data, and the transaction data with unknown leakage points is negative sample data; when the sample dataset is image / text data, the classified image / text data is positive sample data, and the unclassified image / text data is negative sample data; when the sample dataset is traffic data, the known malicious traffic data is positive sample data, and the unknown traffic data is negative sample data.

[0161] Based on the methods and apparatus described in the foregoing embodiments, and in conjunction with specific application scenarios, this invention also provides a system for implementing target object recommendation in the process of recommending target objects, such as music recommendations and product recommendations. For ease of reading, this device embodiment will not repeat the details of the foregoing method embodiments, but it should be understood that the system in this embodiment can correspondingly implement all the contents of the foregoing method embodiments. Specifically, as... Figure 5 As shown, it includes:

[0162] The target object data acquisition unit 51 can be used to acquire the target object data to be predicted;

[0163] The apparatus 52 for constructing a gradient boosting decision tree (GBDT) model can be used to obtain the GBDT model based on a sample dataset. The sample dataset used to train the GBDT model is a dataset about target objects, where data of target objects selected by the user are positive sample data, and data of target objects not selected by the user are unlabeled sample data. In embodiments of the present invention, the apparatus 52 for constructing the GBDT model can specifically be as follows: Figure 3 or Figure 4 As shown;

[0164] The execution unit 53 can be used to perform a target object recommendation task using the gradient boosting decision tree GBDT model obtained by the device 52 for constructing the gradient boosting decision tree GBDT model; wherein the target object is a product or service provided through the Internet.

[0165] Furthermore, considering specific application scenarios, this embodiment of the invention also provides a system for detecting bank card leakage points during the process of bank card leakage point detection. This system is used to detect leakage points in bank card transaction data. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiments, but it should be understood that the system in this embodiment can implement all the content of the aforementioned method embodiments. Specifically, as... Figure 6 As shown, it includes:

[0166] The transaction data acquisition unit 61 can be used to acquire the transaction data of the bank card that has been fraudulently used at the point of leakage to be detected;

[0167] The apparatus 62 for constructing a gradient boosting decision tree (GBDT) model is used to obtain the GBDT model based on a sample dataset. The sample dataset for training the GBDT model is a transaction dataset of fraudulently used bank cards. Transaction data from fraudulently used bank cards with marked vulnerabilities are positive sample data, while transaction data from fraudulently used bank cards without marked vulnerabilities are unmarked sample data. In embodiments of the present invention, the apparatus 62 for constructing the GBDT model can specifically be as follows: Figure 3 or Figure 4 As shown;

[0168] Execution unit 63 can be used to perform bank card leakage point detection tasks using the gradient boosting decision tree GBDT model obtained by the device 62 for constructing the gradient boosting decision tree GBDT model;

[0169] The transaction dataset contains the transaction time and terminal device identifier for each transaction for each bank card.

[0170] Furthermore, considering specific application scenarios, this embodiment of the invention also provides a system for classifying images / text during the text and image classification process. This system is used to classify text and images. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiments, but it should be understood that the system in this embodiment can implement all the content of the aforementioned method embodiments. Specifically, as... Figure 7 As shown, it includes:

[0171] The image / text data acquisition unit 71 can be used to acquire the image / text data to be predicted.

[0172] The apparatus 72 for constructing a gradient boosting decision tree (GBDT) model is used to obtain the GBDT model based on a sample dataset. The sample dataset used to train the GBDT model is an image / text dataset, where image / text data with existing classification labels are positive sample data, and image / text data without classification labels are unlabeled sample data. In embodiments of the present invention, the apparatus 72 for constructing the GBDT model can specifically be as follows: Figure 3 or Figure 4 As shown;

[0173] The execution unit 73 can be used to perform image / text classification tasks using the gradient boosting decision tree GBDT model obtained by the device 72 for constructing the gradient boosting decision tree GBDT model.

[0174] Furthermore, considering specific application scenarios, in the case of malicious traffic detection, this embodiment of the invention also provides a system for detecting malicious traffic from previously undetected traffic data. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiments, but it should be understood that the system in this embodiment can correspondingly implement all the contents of the aforementioned method embodiments. Specifically, as... Figure 8 As shown, it includes:

[0175] The flow data acquisition unit 81 can be used to acquire the gene data to be detected;

[0176] The apparatus 82 for constructing a gradient boosting decision tree (GBDT) model is used to obtain the GBDT model based on a sample dataset. The sample dataset used to train the GBDT model is a traffic dataset, where known malicious traffic data is positive sample data, and undetected traffic data is unlabeled sample data. In embodiments of the present invention, the apparatus 82 for constructing the GBDT model can specifically be as follows: Figure 3 or Figure 4 As shown;

[0177] The execution unit 83 can be used to perform the detection task of pathogenic genes using the gradient boosting decision tree GBDT model obtained by the device 82 for constructing the gradient boosting decision tree GBDT model.

[0178] Furthermore, embodiments of the present invention also provide a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, wherein the computer program, when executed by one or more computing devices, implements the above-described method for constructing a gradient boosting decision tree (GBDT) model.

[0179] In addition, embodiments of the present invention also provide a system including one or more computing devices and one or more storage devices, wherein a computer program is recorded on the one or more storage devices, and when the computer program is executed by the one or more computing devices, the one or more computing devices enable the one or more computing devices to implement the above-described method for constructing a gradient boosting decision tree (GBDT) model.

[0180] In summary, the method and apparatus for constructing a Gradient Boosting Decision Tree (GBDT) model proposed in this invention can acquire a sample dataset. Then, when training each regression tree of the GBDT model, a positive sample training subset is constructed based on the positive sample data in the sample dataset, and a negative sample training subset is constructed by sampling the unlabeled sample data in the sample dataset. The positive sample training subset and the multiple negative sample training subsets are combined to obtain the training set of the current regression tree. The current regression tree is trained based on the training set of the current regression tree, and a Gradient Boosting Decision Tree (GBDT) model is constructed based on each regression tree. Compared with the prior art, this invention can train the regression tree of each GBDT model using the positive and negative sample training subsets obtained from the sample dataset. Since the current regression tree training set used to train each regression tree is extracted from the sample training set, the differences between multiple trees can be guaranteed, avoiding the overfitting problem caused by existing training methods, thereby improving the accuracy of the trained Gradient Boosting Decision Tree (GBDT) model.

[0181] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0182] It is understood that the relevant features in the above-described methods and apparatus can be referenced interchangeably. Furthermore, the terms "first," "second," etc., in the above embodiments are used to distinguish between embodiments and do not represent the superiority or inferiority of any particular embodiment.

[0183] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0184] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0185] In addition, the memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0186] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0188] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0190] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0191] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0192] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0193] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0194] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0195] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for constructing a gradient boosting decision tree (GBDT) model, comprising: Obtain a sample dataset, which includes positively labeled positive sample data and unlabeled unlabeled sample data. The sample dataset is image / text data for image / text classification and is generated based on the corresponding application scenario. When training each regression tree of the GBDT model, a positive sample training subset is constructed based on the positive sample data in the sample dataset, and a negative sample training subset is constructed by sampling the unlabeled sample data in the sample dataset. The positive sample training subset and multiple negative sample training subsets are combined to obtain the training set of the current regression tree. The current regression tree is trained based on the training set of the current regression tree. Then, a gradient boosting decision tree GBDT model is constructed based on each regression tree. The step of training the current regression tree based on the training set includes: iterative training using the training set of the current regression tree and a preset GBDT algorithm to obtain each regression tree corresponding to each iteration of training. In a business scenario where the ratio of positive to negative samples is known, if the estimated ratio of negative sample data to positive sample data is x, then the data volume of the negative sample training subset is x times the data volume of the positive sample training subset. The step of iteratively training the current regression tree using the training set and a preset GBDT algorithm to obtain each regression tree for each iteration includes: obtaining a first training set from the sample dataset and training a first regression tree based on the first training set, the preset GBDT algorithm, and a first parameter, where the first parameter is the mean of the actual results of all sample data in the sample dataset; after obtaining the first regression tree, selecting a second training set from the sample training set and training a second regression tree based on the second training set, the preset GBDT algorithm, and a second parameter, where the second parameter is determined by the prediction results determined by the first regression tree and the actual results of the sample data in the second training set. Both the first and second training sets consist of a positive sample training subset and a negative sample training subset, and the negative sample training subset contained in the first training set is different from that in the second training set.

2. The method as described in claim 1, wherein, The step of constructing a positive sample training subset based on the positive sample data in the sample dataset includes: Take all positive sample data from the sample dataset to construct a positive sample training subset; or, A positive sample training subset is constructed by taking a portion of the positive sample data from the sample dataset.

3. The method as described in claim 1, wherein, In business scenarios where the ratio of positive to negative samples is unknown, the amount of data in the negative sample training subset should be 1 to 2 times the amount of data in the positive sample training subset.

4. The method of claim 1, wherein, Before selecting a second training set from the sample training set and training a second regression tree based on the second training set, a preset GBDT algorithm, and second parameters, the method further includes: The prediction results corresponding to the second training set are obtained by using the first regression tree to predict the second training set. Based on the actual results of the second training set and the prediction results corresponding to the second training set, the residual between the actual results and the prediction results is determined, and the residual is determined as the second parameter.

5. The method according to any one of claims 1-4, wherein, The method further includes: Multiple training sets were constructed based on the aforementioned sample dataset; Multiple candidate models are obtained by selecting from the set of machine learning algorithms, the set of hyperparameter combinations, and the multiple training sets. A candidate model is determined by a machine learning algorithm, a set of hyperparameters, and a training set. At least one GBDT model and the plurality of candidate models are evaluated respectively, and a plurality of models that meet the preset conditions are selected. The multiple models that meet the preset conditions are integrated to obtain the integrated composite model.

6. The method according to claim 5, wherein, Before evaluating at least one of the GBDT models and the plurality of candidate models respectively, the method further includes: A positive sample evaluation subset is constructed by sampling positive sample data in the sample dataset, and a negative sample evaluation subset is constructed by sampling unlabeled sample data in the sample dataset. The positive sample evaluation subset and the negative sample evaluation subset are combined to obtain the evaluation set. The step of evaluating at least one GBDT model and the plurality of candidate models respectively, and selecting a plurality of models that meet preset conditions, includes: The at least one GBDT model and the plurality of candidate models are evaluated according to the evaluation set to obtain the evaluation result for each model, and a plurality of models that meet the preset conditions are selected from the evaluation results.

7. The method according to claim 6, wherein, When the sample dataset is image / text data, the classified images / text data are positive sample data, and the unclassified images / text data are negative sample data.

8. A method for image / text classification, comprising: Obtain the image / text data to be predicted; According to the method described in any one of claims 1-7, the gradient boosting decision tree (GBDT) model is obtained; The resulting gradient boosting decision tree (GBDT) model is used to perform image / text classification tasks.

9. An apparatus for constructing a gradient boosting decision tree (GBDT) model, comprising: The acquisition unit is used to acquire a sample dataset, which includes positively labeled positive sample data and unlabeled unlabeled sample data. The sample dataset is image / text data and is generated based on the corresponding application scenario. A construction unit is used to construct a positive sample training subset based on the positive sample data in the sample dataset and to construct a negative sample training subset by sampling the unlabeled sample data in the sample dataset when training each regression tree of the GBDT model. The positive sample training subset and multiple negative sample training subsets are combined to obtain the training set of the current regression tree. The current regression tree is trained based on the training set of the current regression tree. Then, a gradient boosting decision tree GBDT model is constructed based on each regression tree. The construction unit includes a training module, which is used to perform iterative training using the training set of the current regression tree and a preset GBDT algorithm to obtain each regression tree corresponding to each iteration of training. The construction unit includes: a second construction module, used to make the data volume of the negative sample training subset x times the data volume of the positive sample training subset when the ratio of positive to negative samples is known in a business scenario; The training module includes: a first training submodule, used to obtain a first training set from the sample dataset and train a first regression tree based on the first training set, a preset GBDT algorithm, and a first parameter, wherein the first parameter is the mean of the actual results of all sample data in the sample dataset; and a second training submodule, used to select a second training set from the sample training set after the first regression tree is trained, and train a second regression tree based on the second training set, a preset GBDT algorithm, and a second parameter, wherein the second parameter is determined by the prediction results determined by the first regression tree based on the sample data in the second training set and the actual results of the sample data in the second training set. Both the first and second training sets consist of a positive sample training subset and a negative sample training subset, wherein the negative sample training subset contained in the first training set is different from that in the second training set.

10. The apparatus of claim 9, wherein, The building unit includes: The first construction module is used to construct a positive sample training subset by taking all positive sample data from the sample dataset; or, it is used to construct a positive sample training subset by taking a portion of the positive sample data from the sample dataset.

11. The apparatus of claim 9, wherein, The building unit includes: The second construction module is used to ensure that the amount of data in the negative sample training subset is 1 to 2 times the amount of data in the positive sample training subset when the ratio of positive to negative samples is unknown in business scenarios.

12. The apparatus of claim 9, wherein, The training module also includes: The prediction submodule is used to predict the second training set using the first regression tree to obtain the prediction result corresponding to the second training set. The determination submodule is used to determine the residual between the actual result and the prediction result based on the actual result of the second training set and the prediction result corresponding to the second training set, and to determine the residual as the second parameter.

13. The apparatus according to any one of claims 9-12, wherein, The device further includes: A training set construction unit is used to construct multiple training sets based on the sample dataset; The training unit is used to select from the set of machine learning algorithms, the set of hyperparameter combinations, and the multiple training sets to train multiple candidate models, wherein a machine learning algorithm, a set of hyperparameters, and a training set determine a candidate model. An evaluation unit is used to evaluate at least one GBDT model and the plurality of candidate models respectively, and select a plurality of models that meet preset conditions; An integration unit is used to integrate multiple models that meet preset conditions to obtain an integrated composite model.

14. The apparatus according to claim 13, wherein, The device further includes: The evaluation set construction unit is used to sample positive sample data in the sample dataset to construct a positive sample evaluation subset, sample unlabeled sample data in the sample dataset to construct a negative sample evaluation subset, and combine the positive sample evaluation subset and the negative sample evaluation subset to obtain the evaluation set. The evaluation unit is specifically used to evaluate the at least one GBDT model and the plurality of candidate models according to the evaluation set, obtain the evaluation result for each model, and select a plurality of models that meet the preset conditions from the evaluation results.

15. The apparatus according to claim 14, wherein, The sample dataset is the image / text data used in image / text classification. Wherein, when the sample dataset is image / text data, the classified image / text data is positive sample data, and the unclassified image / text data is negative sample data.

16. A system for image / text classification, comprising: Image / text data acquisition unit, used to acquire image / text data to be predicted; An apparatus for constructing a gradient boosting decision tree (GBDT) model, used to obtain a gradient boosting decision tree (GBDT) model according to any one of claims 1-7; The execution unit is used to perform image / text classification tasks using the obtained gradient boosting decision tree GBDT model.

17. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by one or more computing devices, implements the method of any one of claims 1-7.

18. A system comprising one or more computing devices and one or more storage devices, wherein a computer program is stored on the one or more storage devices, the computer program, when executed by the one or more computing devices, causes the one or more computing devices to perform the method as described in any one of claims 1-7.

Images