Prediction model optimization method, device and equipment based on improved gradient boosting tree
By eliminating the bad learner, incremental training and weight update optimization gradient enhancement tree model, the prediction accuracy reduction caused by data distribution changes is solved, and the model is adaptive and efficient prediction is achieved.
Patent Information
- Application Number
- CN202510392016.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-07-04
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing prediction models cannot adapt effectively when the data distribution changes, resulting in reduced prediction accuracy and increased computational complexity.
By determining the bad learners based on the preset statistical indicators and expelling them, performing incremental training until the number of learners returns to the preset level, each learner is weighted and weighted updates are performed according to the change in the data distribution, and the concept drift adaptive prediction model is optimized.
It improves the prediction accuracy and efficiency of the model, maintains the diversity and adaptability of the learner, and can maintain good prediction performance when the data distribution changes.
Smart Images
Figure CN120258178A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technologies, and in particular, to a method, device, and equipment for optimizing a prediction model based on an improved gradient boosting tree. Background Art
[0002] Ensemble learning methods are considered to be one of the most effective strategies for dealing with concept drift in data streams. In the aspect of tree ensembles, incremental gradient boosting trees, in an incremental form based on cards, continuously integrate weak decision trees while minimizing the loss function, thereby enhancing the drift adaptation ability of the prediction model.
[0003] In existing diverse ensemble methods, the accuracy-weighted diversity online boosting method generates instance weights using correct classification, misclassification, and expert accuracy.
[0004] However, since the type and scale of concept drift are often unknown, existing model adaptation methods may lead to a decrease in prediction accuracy and an increase in computational complexity. Summary of the Invention
[0005] This application provides a method for optimizing a prediction model based on an improved gradient boosting tree, which is used to solve the problem that the existing prediction model cannot adapt to the model when the data distribution changes to maintain the prediction accuracy rate.
[0006] In a first aspect, this application provides a method for optimizing a prediction model based on an improved gradient boosting tree. The prediction model includes multiple learners, and the method includes:
[0007] Determine the poor learners in the prediction model according to a preset statistical metric, and perform elimination processing on the poor learners;
[0008] Perform incremental training on the prediction model after eliminating the poor learners until the number of learners in the prediction model resumes to the preset level;
[0009] Assign weights to each learner in the prediction model, and update the weights of each learner in the prediction model according to the change in data distribution to obtain an optimized concept drift adaptive prediction model.
[0010] Optionally, the determining the poor learners in the prediction model according to a preset statistical metric and performing elimination processing on the poor learners includes:
[0011] Determine the cumulative loss of each learner in the prediction model, and determine the loss change rate between any two adjacent learners in the prediction model according to the cumulative loss;
[0012] Judge whether the subsequent learner is a bad learner according to the loss change rates of the two adjacent learners;
[0013] In the case where the subsequent learner is a bad learner, perform an elimination process on the bad learner.
[0014] Optionally, the judging whether the subsequent learner is a bad learner according to the loss change rates of the two adjacent learners includes:
[0015] Obtain the corresponding significance index of the loss change rates of two consecutive learners through a statistical test;
[0016] Judge whether the significance index is less than a preset significance threshold;
[0017] In the case where the significance index is less than the preset significance threshold, regard the subsequent learner as a bad learner.
[0018] Optionally, the incrementally training the prediction model after eliminating the bad learner until the number of learners in the prediction model resumes to a preset level includes:
[0019] Judge whether the number of learners in the prediction model after the elimination process of the bad learner is lower than a preset threshold;
[0020] If the number of learners in the prediction model after the elimination process of the bad learner is lower than the preset threshold, then incrementally train the prediction model until the number of learners in the prediction model reaches the preset threshold, and then add a preset number of learners to the prediction model;
[0021] If the number of learners in the prediction model after the elimination process of the bad learner is not lower than the preset threshold, then directly add a preset number of learners to the prediction model.
[0022] Optionally, the weight update process for each learner in the prediction model according to the data distribution change satisfies the following formula:
[0023]
[0024] where y is the true value, y i is the predicted value, n is the number of samples, and ε is a calibration parameter, generally set to 0.9 - 1.
[0025] Optionally, the method further includes: outputting a prediction result through the optimized concept drift adaptive prediction model, and the output prediction result can be expressed as:
[0026]
[0027] Among them, F represents a model set, and f i represents the i-th model, and w i represents the weight of the i-th model, f0 represents the incremental model for continuous learning, and w0 represents the weight of the incremental model for continuous learning.
[0028] In a second aspect, the present application provides a device, and the device includes:
[0029] A rejection processing module, configured to determine bad learners in the prediction model according to a preset statistical metric, and perform rejection processing on the bad learners;
[0030] An incremental training module, configured to perform incremental training on the prediction model after rejecting bad learners until the number of learners in the prediction model resumes to a preset level;
[0031] A weight assignment module, configured to assign weights to each learner in the prediction model, and perform weight update processing on each learner in the prediction model according to changes in data distribution, so as to obtain an optimized concept drift adaptive prediction model.
[0032] In a third aspect, the present application provides a device, including:
[0033] A memory;
[0034] A processor;
[0035] Among them, the memory stores computer execution instructions;
[0036] The processor executes the computer execution instructions stored in the memory to implement the prediction model optimization method based on the improved gradient boosting tree as described in the first aspect and various possible implementation manners of the first aspect.
[0037] In a fourth aspect, the present application provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the prediction model optimization method based on the improved gradient boosting tree as described in the first aspect and various possible implementation manners of the first aspect.
[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the prediction model optimization method based on the improved gradient boosting tree as described in the first aspect and various possible implementation manners of the first aspect.
[0039] The present application provides a method, apparatus, and device for optimizing a prediction model based on an improved gradient boosting tree. The prediction model includes multiple learners. The method determines bad learners in the prediction model according to a preset statistical metric, and eliminates the bad learners; performs incremental training on the prediction model after eliminating the bad learners until the number of learners in the prediction model resumes to the preset level; assigns weights to each learner in the prediction model, and updates the weights of each learner in the prediction model according to the change in data distribution, obtaining an optimized concept drift adaptive prediction model, which improves the accuracy and efficiency of model prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.
[0041] Figure 1 is a flowchart of the method for optimizing a prediction model based on an improved gradient boosting tree provided by an embodiment of the present application Figure 1 ;
[0042] Figure 2 is a flowchart of the method for optimizing a prediction model based on an improved gradient boosting tree provided by an embodiment of the present application Figure 2 ;
[0043] Figure 3 is a schematic structural diagram of the apparatus for optimizing a prediction model based on an improved gradient boosting tree provided by an embodiment of the present application;
[0044] Figure 4 is a schematic structural diagram of the device for optimizing a prediction model based on an improved gradient boosting tree provided by an embodiment of the present application.
[0045] Through the above accompanying drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and the written description are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0046] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below with reference to the accompanying drawings in the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present application belong to the scope of protection of the present application.
[0047] In the specification, claims and above-mentioned drawings of the present invention, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.
[0048] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.
[0049] Concept drift refers to the phenomenon that the statistical characteristics or data distribution of the target variable changes over time in machine learning or data mining tasks. This change causes the previously trained model to perform poorly on new data.
[0050] The prior art can quantify drift by observing whether the data distribution changes or by the classification effect of the prediction model.
[0051] In practical applications, since the types and scales of concept drift in data streams are mostly unknown, it makes the concept drift detection and adaptation of the model more difficult, resulting in a low accuracy of the output results of the prediction model.
[0052] To address the above problems, the present application proposes an optimization method for a prediction model based on an improved gradient boosting tree. This method evaluates the performance of the base learner by calculating the loss change rate of two consecutive learners, prunes the learners with poor performance, and dynamically updates the weights of each learner, so as to maintain a high accuracy while ensuring the diversity of the learners. This method can automatically adjust the optimal iteration according to different severities of concept drift while maintaining the diversity of the learners.
[0053] The following uses specific embodiments to elaborate in detail on the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems. These several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of the present application with reference to the drawings.
[0054] Figure 1 Flow schematic of an optimization method for a prediction model based on an improved gradient boosting tree provided for the embodiments of the present application Figure 1 . As Figure 1 shown, the optimization method for a prediction model based on an improved gradient boosting tree provided in this embodiment includes:
[0055] S101: Determine the poor learners in the prediction model according to the preset statistical metrics, and remove the poor learners.
[0056] Among them, the statistical metrics are numerical values used to measure and describe data characteristics. The prediction model is a mathematical model constructed based on historical data, aiming to predict future events or results. The model consists of multiple learners, and each learner can be regarded as a component of the model, learning the rules from the data respectively and collaborating to complete the prediction task. A poor learner refers to a learner that performs poorly in the prediction model, cannot effectively learn the data rules, and has a negative impact on the overall performance of the model.
[0057] By removing the learners determined to be poor from the prediction model, it is beneficial to optimize the model structure, improve the overall performance of the model, and avoid the interference of poor learners on the model prediction results.
[0058] S102: Perform incremental training on the prediction model after removing the poor learners until the number of learners in the prediction model resumes to the preset level.
[0059] Among them, incremental training is a way of gradual learning, which allows the model to continuously learn and update using new data based on the existing knowledge. Incremental training does not require all data to be input into the model at once, but can process new data in batches, and fine-tune the model parameters each time new data is processed, so that the model can continuously adapt to new data patterns and changes.
[0060] It can be understood that select a suitable incremental training algorithm according to the type and characteristics of the model, input the new training data into the prediction model after removing the poor learners in batches, train the model according to the selected incremental training algorithm, and continuously adjust the model parameters during the training process to enable the model to gradually adapt to the new data. After the incremental training is completed, check whether the number of learners in the model reaches the preset level. If not, continue the next round of incremental training until the number of learners resumes to the preset level.
[0061] By increasing the number of learners to the preset level through incremental training, the model can be restored to a state where it can fully express data characteristics, improving the model's adaptability to various situations and prediction accuracy.
[0062] S103: Assign weights to each learner in the prediction model, and update the weights of each learner in the prediction model according to the change in data distribution to obtain an optimized concept drift adaptive prediction model.
[0063] It is understandable that the weight assigned to each learner in the prediction model represents the importance of the learner in the final prediction result.
[0064] It is also understandable that data distribution change refers to the change of data characteristics and patterns over time, and this phenomenon is called concept drift. To enable the prediction model to adapt to this change, it is necessary to update the weights of each learner according to the change of data distribution. The prediction model obtained after the weight update process is the optimized concept drift adaptive prediction model, which can still maintain good prediction performance when the data distribution changes.
[0065] Specifically, evaluate the performance of each learner through historical data. According to the evaluation results, assign higher weights to learners with better performance and lower weights to learners with poorer performance, and monitor the change of data distribution through preset statistical indicators. When the preset statistical indicators change significantly, it indicates that the data distribution has changed and the weights of the learners need to be updated.
[0066] By updating the weights of the learners according to the change of data distribution, the prediction model can timely adjust the importance of each learner, thereby maintaining good adaptability to new data and improving the stability and reliability of the model.
[0067] An optimization method for a prediction model based on an improved gradient boosting tree provided by an embodiment of the present application. This method determines the bad learners in the prediction model according to preset statistical quantity indicators and performs elimination processing on the bad learners; performs incremental training on the prediction model after eliminating the bad learners until the number of learners in the prediction model resumes to the preset level; assigns weights to each learner in the prediction model and updates the weights of each learner in the prediction model according to the change of data distribution to obtain an optimized concept drift adaptive prediction model, improving the accuracy and efficiency of model prediction.
[0068] Figure 2 The flow of an optimization method for a prediction model based on an improved gradient boosting tree provided by an embodiment of the present application Figure 2 . This embodiment is based on Figure 1 the embodiment, and a possible implementation manner of the optimization method for the prediction model based on the improved gradient boosting tree is described in detail. As Figure 2 shown, the method includes:
[0069] S201: Determine the cumulative loss of each learner in the prediction model, and determine the loss change rate between any two adjacent learners in the prediction model according to the cumulative loss.
[0070] Among them, loss is an indicator that measures the difference between the prediction result of the learner and the true result, and the cumulative loss is the sum of the losses generated by each prediction during the process of the learner processing a series of data samples.
[0071] It can be understood that in the prediction model, multiple learners are arranged in a certain order. The loss change rate between any two adjacent learners refers to the change range between the cumulative losses of these two adjacent learners. The loss change rate can reveal the change trend of the performance between adjacent learners. If the loss change rate is positive, it indicates that the cumulative loss of the latter learner has increased significantly compared to the previous learner; if the loss change rate is negative, it means that the performance of the latter learner has been significantly improved. By analyzing the loss change rate, the mutual relationship between learners and the adaptability of learners at different data stages can be further understood.
[0072] Specifically, in this embodiment, in order to iteratively measure the loss change rate and find out the learners with poor performance, the loss change rate of using two consecutive learners is introduced as a criterion. For example, consider an initial model in the data block The loss change rate of two consecutive learners is calculated by the following formula for training on:
[0073]
[0074] where, r m is the loss change rate of two consecutive learners, m - 1 and m are the indices of two consecutive learners, h m (x) is the base learner, and l is the cumulative loss.
[0075] It can also be understood that due to the large number of data samples, the mean value of the loss is used in this embodiment to calculate l m . According to the law of large numbers, as the number of samples increases, the sample mean converges to the population mean (i.e., the expectation). If r m < 0, it means that the added learner h m (x) helps to reduce the loss. The smaller r m is, the greater the contribution of the learner h m (x) to reducing the loss function; if r m ≥0, the loss has not decreased, then the learner h m (x) is considered to work poorly.
[0076] S202: Obtain the corresponding significance index for the loss change rate of two consecutive learners through statistical tests.
[0077] It is understandable that statistical testing is a method based on probability theory and mathematical statistics, used to determine whether the overall characteristics represented by sample data have a certain specific property or relationship. The significance index is the result of statistical testing. If the significance index is less than the pre-set significance level, the null hypothesis is rejected, and it is considered that the loss change rate of two consecutive learners is statistically significant, that is, this change is not randomly generated, but reflects the true change in the performance of the learner; if the significance index is greater than the pre-set significance level, the null hypothesis is accepted, and it is considered that the loss change may be caused by random factors.
[0078] Specifically, in this embodiment, the loss change rate of adjacent learners obtained in the above steps is substituted into the k-s test to obtain the significance index of the loss change rate of adjacent learners. By comparing the significance index obtained by the k-s test with the significance threshold, where the significance threshold will be set to multiple parameters such as {0.001, 0.005, 0.01, 0.05} to select the optimal value through experiments.
[0079] S203: Determine whether the significance index is less than the preset significance threshold.
[0080] S204: In the case where the significance index is less than the preset significance threshold, regard the subsequent learner as a poor learner.
[0081] It is understandable that if the significance index obtained by the k-s test is less than the preset significance threshold, the null hypothesis is rejected, and the subsequent learner is regarded as a poor learner.
[0082] S205: In the case where the subsequent learner is a poor learner, perform a removal process on the poor learner.
[0083] S206: Determine whether the number of learners in the prediction model after the removal process of the poor learner is lower than the preset threshold; if so, execute step S207, if not, execute step S208.
[0084] S207: Incrementally train the prediction model until the number of learners in the prediction model reaches the preset threshold, and then add a preset number of learners to the prediction model.
[0085] It is understandable that since the pruning strategy prunes a part of the poor learners with relatively poor performance, in order to help the model adapt to the changes in the streaming data after pruning, this embodiment will perform incremental learning on the model to supplement the pruned learners. Incremental training can perform local adjustments and updates for new data or features, so that the model can better adapt to new concepts or data distributions. During this process, two incremental learning selections will be made after each pruning to ensure that the number of learners is not less than the initial number.
[0086] If the number of learners after pruning is lower than the preset threshold, the number of learners is incrementally trained to the preset threshold, and then a certain number of learners are gradually added. By incrementally training the model, the number of learners returns to the normal level, and at the same time, the model forgets the outdated old concepts and learns new knowledge.
[0087] S208: Directly add a preset number of learners to the prediction model.
[0088] It can be understood that if the number of learners after pruning is not lower than the preset threshold, a fixed number of learners are added.
[0089] S209: Assign weights to each learner in the prediction model, and update the weights of each learner in the prediction model according to the change in data distribution to obtain an optimized concept drift adaptive prediction model.
[0090] It can be understood that after statistical pruning, weights will be assigned to the learners, and by giving lower weights to the learners with poorer performance, their impact on the overall model is reduced.
[0091] Due to the change in data distribution, the prediction performance of the model decreases. The learners trained during concept drift may no longer adapt to the subsequent data distribution, resulting in large errors. Therefore, due to the continuous change in data distribution caused by concept drift, the weights of each learner also need to be continuously updated as the data distribution changes. The weights of each learner in the prediction model are updated according to the change in data distribution, satisfying the following formula:
[0092]
[0093] Where y is the true value, yi is the predicted value, n is the number of samples, and ε is a correction parameter, generally set to 0.9 - 1.
[0094] To retain the diversity of learners, the learners are not pruned, but only their weights are continuously adjusted to reduce their impact on the overall model prediction effect. The weights of the learners with better performance will be continuously increased to highlight their superior prediction ability.
[0095] The learners after weight assignment and update processing are combined to form the final concept drift adaptive prediction model. When making a prediction, the prediction results of each learner are weighted and summed according to their weights to obtain the final prediction result.
[0096] In an optional embodiment, the method further includes: outputting a prediction result through the optimized concept drift adaptive prediction model.
[0097] Specifically, the output of the prediction result can be expressed as:
[0098]
[0099] Among them, F represents a set of models, and f i represents the i-th model, and w i represents the weight of the i-th model, f0 represents the incremental model for continuous learning, w0 represents the weight of the incremental model for continuous learning, x is a feature, and softmax converts a set of arbitrary real values into a probability distribution, and the sum of all values is 1.
[0100] An optimization method for a prediction model based on an improved gradient boosting tree provided by an embodiment of the present application determines the cumulative loss of each learner in the prediction model, determines the loss change rate between any two adjacent learners in the prediction model according to the cumulative loss, obtains the corresponding significance index by statistical test for the loss change rate of two consecutive learners, and when the significance index is less than a preset significance threshold, regards the latter learner as a bad learner and performs an elimination process on the bad learner, determines whether the number of learners in the prediction model after the elimination process on the bad learner is lower than a preset threshold, if it is lower than the preset threshold, then performs incremental training on the prediction model until the number of learners reaches the preset threshold, and then adds a preset number of learners to the prediction model, if the number of learners in the prediction model after the elimination process on the bad learner is not lower than the preset threshold, then directly adds a preset number of learners to the prediction model, then performs weight assignment on each learner in the prediction model, and updates the weight of each learner in the prediction model according to the change in data distribution, so as to obtain an optimized concept drift adaptive prediction model, improving the accuracy and efficiency of model prediction.
[0101] Figure 3 It is a structural schematic diagram of an optimization device for a prediction model based on an improved gradient boosting tree provided by the present application. As Figure 3 shown, the optimization device 300 for a prediction model based on an improved gradient boosting tree provided in this embodiment includes:
[0102] An elimination processing module 301, configured to determine a bad learner in the prediction model according to a preset statistical index, and perform an elimination process on the bad learner;
[0103] An incremental training module 302, configured to perform incremental training on the prediction model after eliminating the bad learner until the number of learners in the prediction model resumes to a preset level;
[0104] A weight assignment module 303, configured to perform weight assignment on each learner in the prediction model, and update the weight of each learner in the prediction model according to the change in data distribution, so as to obtain an optimized concept drift adaptive prediction model.
[0105] An optimization device for a prediction model based on an improved gradient boosting tree provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effects are similar. Details are not described here in this embodiment.
[0106] Figure 4 The following is a schematic structural diagram of an optimization device for a prediction model based on an improved gradient boosting tree provided in this application. As Figure 4 shown, the optimization device for a prediction model based on an improved gradient boosting tree provided in this application, the optimization device 400 for a prediction model based on an improved gradient boosting tree includes: a receiver 401, a transmitter 402, a processor 403, and a memory 404.
[0107] The receiver 401 is configured to receive instructions and data;
[0108] The transmitter 402 is configured to transmit instructions and data;
[0109] The memory 404 is configured to store computer-executable instructions;
[0110] The processor 403 is configured to execute the computer-executable instructions stored in the memory 404 to implement each step executed by the optimization method for the prediction model based on the improved gradient boosting tree in the above embodiment. Specifically, reference can be made to the relevant descriptions in the foregoing embodiment of the optimization method for the prediction model based on the improved gradient boosting tree.
[0111] Optionally, the above-mentioned memory 404 can be either independent or integrated with the processor 403.
[0112] When the memory 404 is independently provided, the electronic device further includes a bus for connecting the memory 404 and the processor 403.
[0113] This application also provides a computer storage medium, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, the optimization method for the prediction model based on the improved gradient boosting tree executed by the above-mentioned optimization device for the prediction model based on the improved gradient boosting tree is implemented.
[0114] This application also provides a computer program product, including a computer program, which when executed by the processor implements the above-mentioned optimization method for the prediction model based on the improved gradient boosting tree.
[0115] Those of ordinary skill in the art will understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations. In the hardware implementation, the division between the functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be executed by several physical components in cooperation. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technologies, CD-ROM, digital versatile disk (DVD), or other optical disk storage, magnetic cassettes, tapes, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium typically contains computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.
[0116] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include the common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and examples are only illustrative, and the true scope and spirit of the present application are pointed out by the following claims.
[0117] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.
Claims
1. An optimization method for a prediction model based on an improved gradient boosting tree, characterized in that, The prediction model includes multiple learners, and the method includes: Determining the poor learners in the prediction model according to a preset statistical metric, and performing a rejection process on the poor learners; Performing incremental training on the prediction model after rejecting the poor learners until the number of learners in the prediction model resumes to the preset level; Assigning weights to each learner in the prediction model, and performing a weight update process on each learner in the prediction model according to the change in data distribution to obtain an optimized concept drift adaptive prediction model.
2. The method according to claim 1, wherein The determining the poor learners in the prediction model according to a preset statistical metric, and performing a rejection process on the poor learners includes: Determining the cumulative loss of each learner in the prediction model, and determining the loss change rate between any two adjacent learners in the prediction model according to the cumulative loss; Judging whether the latter learner is a poor learner according to the loss change rate of the two adjacent learners; In the case where the latter learner is a poor learner, performing a rejection process on the poor learner.
3. The method according to claim 2, wherein The judging whether the latter learner is a poor learner according to the loss change rate of the two adjacent learners includes: Obtaining a corresponding significance index by subjecting the loss change rates of two consecutive learners to a statistical test; Judging whether the significance index is less than a preset significance threshold; In the case where the significance index is less than the preset significance threshold, taking the latter learner as a poor learner.
4. The method according to claim 1, wherein The performing incremental training on the prediction model after rejecting the poor learners until the number of learners in the prediction model resumes to the preset level includes: Judging whether the number of learners in the prediction model after rejecting the poor learners is lower than a preset threshold; If the number of learners in the prediction model after rejecting the poor learners is lower than the preset threshold, performing incremental training on the prediction model until the number of learners in the prediction model reaches the preset threshold, and then adding a preset number of learners to the prediction model; If the number of learners in the prediction model after rejecting the poor learners is not lower than the preset threshold, directly adding a preset number of learners to the prediction model.
5. The method according to claim 1, characterized in that The performing a weight update process on each learner in the prediction model according to the change in data distribution satisfies the following formula: where y is the true value, yi is the predicted value, n is the number of samples, and ε is a correction parameter, generally set to 0.9 - 1.
6. The method according to claim 1, wherein The method further includes: outputting a prediction result through the optimized concept drift adaptive prediction model, and the output prediction result can be expressed as: Among them, F represents the model set, and f i represents the i-th model, and w i represents the weight of the i-th model, f0 represents the incremental model for continuous learning, and w0 represents the weight of the incremental model for continuous learning.
7. An optimization device for a prediction model based on an improved gradient boosting tree, characterized in that, The device includes: A rejection processing module, configured to determine the poor learners in the prediction model according to a preset statistical metric, and perform a rejection process on the poor learners; An incremental training module, configured to perform incremental training on the prediction model after rejecting the poor learners until the number of learners in the prediction model resumes to the preset level; A weight allocation module, configured to allocate weights to each learner in the prediction model, and update the weights of each learner in the prediction model according to changes in data distribution, so as to obtain an optimized concept drift adaptive prediction model.
8. An optimization device for a prediction model based on an improved gradient boosting tree, characterized in that, The device includes: A memory; A processor; Wherein, the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the prediction model optimization method based on the improved gradient boosting tree according to any one of claims 1-6.
9. A computer storage medium, characterized in that, Computer-executable instructions are stored in the computer storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement a prediction model optimization method based on the improved gradient boosting tree according to any one of claims 1-6.
10. A computer program product, characterized in that, It includes a computer program, which is used to implement a prediction model optimization method based on the improved gradient boosting tree according to any one of claims 1-6 when executed by a processor.