Time sequence prediction precision improvement method based on LightGBM algorithm
By using the LightGBM algorithm to model and predict the residuals in time series prediction, and combining the difference method and model parameter optimization, the problem of inefficient residual processing and parameter tuning in time series prediction is solved, and higher prediction accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202411510558.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-03-11
AI Technical Summary
In time series prediction, the prior art is difficult to effectively process residuals, resulting in low prediction accuracy. Especially when facing complex and variable data, the parameter tuning efficiency of traditional methods is low and it is difficult to find the best parameter combination.
The time series prediction method based on LightGBM algorithm is adopted to improve prediction accuracy by modeling and predicting residuals, combining the difference method and model parameter optimization. Specific steps include missing value processing, trend and seasonal judgment, differential method to remove trend and seasonality, white noise inspection, model parameter selection and parameter optimization of LightGBM algorithm.
It improves the accuracy and robustness of time series prediction, can capture hidden information and trends in the data more accurately, provides more reliable prediction results, and the model is robust and suitable for multiple use.
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of machine learning and time series analysis, and specifically to a method for improving the accuracy of time series prediction based on the LightGBM algorithm. Background Art
[0002] In time series forecasting, the prediction accuracy of the model is often affected by many factors, among which the processing of residuals is particularly critical. Residuals contain information and noise that are not captured by the model. Therefore, accurate prediction and processing of residuals can effectively improve the accuracy of time series forecasting.
[0003] As an efficient gradient boosting decision tree algorithm, the LightGBM algorithm has excellent ability to handle large-scale data and complex models. In time series prediction, the LightGBM algorithm can further capture the hidden information and trends in time series data by modeling and predicting residuals. This method can make full use of the advantages of the LightGBM algorithm and perform refined processing of residuals, thereby further improving the accuracy of time series prediction.
[0004] The LightGBM algorithm involves many parameters, such as learning rate, tree depth, number of leaf nodes, etc. The selection of these parameters has an important impact on the performance of the model. Different parameter combinations will cause the model to show completely different effects during training and prediction. In order to obtain the best prediction performance, the model needs to be optimized. Common parameter tuning methods include grid search, random search, and Bayesian optimization. However, these tuning methods will be less efficient when facing high-dimensional parameter space, making it more difficult to search for the best parameter combination.
[0005] When predicting time series models, although ACF diagram (autocorrelation function diagram) and PACF diagram (partial autocorrelation function diagram) are commonly used analysis tools, which provide autocorrelation and partial autocorrelation information about time series data, they cannot intuitively and accurately judge the parameters in the model. They can only roughly determine a parameter range, and the model needs to be tuned later.
[0006] In order to solve the above problems, a method to improve the accuracy of time series prediction based on the LightGBM algorithm is urgently needed. Summary of the invention
[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for improving the accuracy of time series prediction based on the LightGBM algorithm, comprising the following steps:
[0008] Step 1: Before starting time series analysis, missing values must be processed first;
[0009] Step 2: After data processing is completed, determine whether there is trend and seasonality in the time series;
[0010] Step 3: If the time series is not stationary, use the difference method to remove the trend and seasonality of the series, perform an ADF test on the results of each step, and determine the optimal number of difference steps through the ADF test statistic and p-value obtained after the test;
[0011] Step 4: After the sequence is stable, perform a white noise test on the sequence;
[0012] Step 5: After the sequence is a stationary non-white noise sequence, select a model based on the tailing and truncation of the time series diagram after difference. Multiple models can be selected, and finally the model performance is judged to determine the parameter autoregressive order (p), difference number (d), moving average order (q), and cycle length (s) in each model;
[0013] Step 6: After determining the model parameters, the next step is to select the optimal model. The performance of the model can usually be judged by the value of the Akaike Information Criterion, that is, the AIC value. The smaller the AIC value, the better the model performance.
[0014] Step 7: After the model selection is completed, the future values of the sequence are predicted and the model residuals are extracted;
[0015] Step 8: After the residual extraction is completed, the LightGBM algorithm is used to model the residual. If the difference is used in the model, the prediction results need to be processed by the necessary reverse difference.
[0016] Step 9: Add the preliminary results obtained by the model prediction to the residual prediction results of the LightGBM algorithm to get the final prediction results;
[0017] Step 10: Appropriate evaluation indicators such as mean square error (MSE), root mean square error (RMSE) or mean absolute error (MAE) can be used to evaluate the accuracy of the final prediction results.
[0018] Preferably, when processing missing values in step 1, any one of interpolation, eigenvalue filling and deletion of missing rows can be adopted.
[0019] Preferably, in step 2, an intuitive judgment can be made by drawing a time series diagram. When the trend is not obvious, an ADF test can be used to determine whether the time series is stable.
[0020] Preferably, the Ljung-Box method is used in step 4 to verify whether the useful information in the sequence is fully extracted. The white noise sequence is a random sequence with no analytical value and should be discarded.
[0021] Preferably, the LightGBM algorithm in step eight involves many parameters, and in order to obtain the best prediction performance, it needs to be optimized, and the optimal parameters can be determined by grid search and other methods. After completing the parameter selection, the optimized LightGBM algorithm is used for residual prediction.
[0022] The present invention provides a method for improving the accuracy of time series prediction based on the LightGBM algorithm. It has the following beneficial effects:
[0023] Compared with traditional methods, this method has higher prediction accuracy and stronger robustness, can cope with various complex and changeable data situations, provide us with more accurate and reliable prediction results, and help us better grasp the future development trend and change direction of the data. This method introduces a machine learning model based on the gradient boosting decision tree algorithm, which improves the accuracy of model prediction. The model trained by this method can be used multiple times and for a long time. The program is robust and the use of lightweight models has low requirements on the hardware environment. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0025] Embodiment, the present invention provides a method for improving the accuracy of time series prediction based on the LightGBM algorithm:
[0026] Step 1: Before starting time series analysis, missing values must be processed first, which can be done by interpolation, feature value filling, and deletion of missing rows.
[0027] Step 2: After data processing is completed, to determine whether the time series has a trend and seasonality, you can draw a time series diagram to make an intuitive judgment. When the trend is not obvious, you can also use the ADF (unit root) test to determine whether the time series is stable;
[0028] Step 3: If the time series is not stationary, use the difference method to remove the trend and seasonality of the series, perform an ADF test on the results of each step, and determine the optimal difference step number through the ADF test statistic and p-value obtained after the test. Draw a time series diagram after the difference and compare it with the time series diagram before the difference to check whether the trend has been removed;
[0029] Step 4: After the sequence is stationary, perform a white noise test on the sequence. The Ljung-Box method can be used to verify whether the useful information in the sequence is fully extracted. The white noise sequence is a random sequence with no analytical value and should be discarded.
[0030] Step 5: After the sequence is a stationary non-white noise sequence, select a model based on the tailing and truncation of the time series diagram after difference. Multiple models can be selected, and finally the model performance is judged to determine the autoregressive order (p), difference number (d), moving average order (q), and cycle length (s) of each model. The difference method in the previous step 3 can determine the value of parameter d, and then draw the ACF diagram and PACF diagram to determine the parameters p and q. Usually, there will be a certain judgment error in the values of parameters p and q, and a range can be roughly determined. Then, through hyperparameter search, traverse different combinations of p and q values, and use each traversed parameter value to fit the model to judge the model performance;
[0031] Step 6: As mentioned in step 4, multiple models can be selected. After determining the model parameters, the next step is to select the optimal model. The performance of the model can usually be judged by the AIC (Akaike Information Criterion) value. The smaller the AIC value, the better the model performance.
[0032] Step 7: After the model selection is completed, the future values of the sequence are predicted and the model residuals are extracted;
[0033] Step 8: After the residual extraction is completed, the LightGBM algorithm is used to model the residual. If the difference is used in the model, the prediction results need to be subjected to necessary reverse difference processing. The LightGBM algorithm involves many parameters. In order to obtain the best prediction performance, it needs to be optimized. Grid search and other methods can be used to determine the optimal parameters. After completing the parameter selection, the optimized LigthGBM algorithm is used for residual prediction.
[0034] Step 9: Add the preliminary results obtained by the model prediction to the prediction results of the residual by the LightGBM algorithm to obtain the final prediction results, which more fully extracts the effective information in the residual and improves the prediction accuracy.
[0035] Step 10: Appropriate evaluation indicators such as mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), etc. can be used to evaluate the accuracy of the final prediction results.
[0036] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention are implemented according to the conventional means in the field unless otherwise specified and limited.
Claims
1. A method for improving the accuracy of time series prediction based on the LightGBM algorithm, characterized in that: The steps include: Step 1: Before starting time series analysis, missing values must be processed first; Step 2: After data processing is completed, determine whether there is trend and seasonality in the time series; Step 3: If the time series is not stationary, use the difference method to remove the trend and seasonality of the series, perform an ADF test on the results of each step, and determine the optimal number of difference steps through the ADF test statistic and p-value obtained after the test; Step 4: After the sequence is stationary, perform a white noise test on the sequence; Step 5: After the sequence is a stationary non-white noise sequence, select a model based on the tailing and truncation of the time series diagram after difference. Multiple models can be selected, and finally the model performance is judged to determine the parameter autoregressive order (p), difference number (d), moving average order (q), and cycle length (s) in each model; Step 6: After determining the model parameters, the next step is to select the optimal model. The performance of the model can usually be judged by the value of the Akaike Information Criterion, that is, the AIC value. The smaller the AIC value, the better the model performance. Step 7: After the model selection is completed, the future values of the sequence are predicted and the model residuals are extracted; Step 8: After the residual extraction is completed, the LightGBM algorithm is used to model the residual. If the difference is used in the model, the prediction results need to be processed by the necessary reverse difference. Step 9: Add the preliminary results obtained by the model prediction to the residual prediction results of the LightGBM algorithm to get the final prediction results; Step 10: Appropriate evaluation indicators such as mean square error (MSE), root mean square error (RMSE) or mean absolute error (MAE) can be used to evaluate the accuracy of the final prediction results.
2. According to claim 1, a method for improving the time series prediction accuracy based on the LightGBM algorithm is characterized by: In the step 1, when processing missing values, any one of interpolation, eigenvalue filling and deletion of missing rows can be adopted.
3. According to claim 1, a method for improving the time series prediction accuracy based on the LightGBM algorithm is characterized by: In the step 2, an intuitive judgment can be made by drawing a time series diagram. When the trend is not obvious, an ADF test can be used to determine whether the time series is stable.
4. According to claim 1, a method for improving the time series prediction accuracy based on the LightGBM algorithm is characterized by: The Ljung-Box method is used in step 4 to verify whether the useful information in the sequence is fully extracted. The white noise sequence is a random sequence with no analytical value and should be discarded.
5. According to claim 1, a method for improving the time series prediction accuracy based on the LightGBM algorithm is characterized by: The LightGBM algorithm in step 8 involves many parameters. In order to obtain the best prediction performance, it needs to be optimized. The optimal parameters can be determined by grid search and other methods. After completing the parameter selection, the optimized LightGBM algorithm is used for residual prediction.