Wind speed combination prediction method and system based on reinforcement learning, storage medium and product
By introducing a multi-armed slot machine model of reinforcement learning and an ε-greedy strategy, and by rationally allocating the weights of the combined model, the problem of large wind speed prediction errors in existing technologies is solved, and more accurate ultra-short-term multi-step wind speed prediction is achieved.
Patent Information
- Application Number
- CN202511250313.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-03
- Publication Date
- 2026-01-23
AI Technical Summary
In existing wind speed prediction models, the weight allocation of the combined model is unreasonable, resulting in large prediction errors and making it difficult to accurately depict the wind speed variation patterns in complex environments.
A multi-armed slot machine model based on reinforcement learning is adopted. The action selection for each step is determined by an ε-greedy strategy. The reward value is determined based on the prediction error of the wind speed prediction model on the validation set. After completing a preset number of action selections, the weight of each wind speed prediction model is determined based on the average reward value, and then a weighted combination is performed to achieve wind speed prediction.
It improves the robustness and generalization ability of the model, significantly enhances the accuracy and stability of ultra-short-term wind speed prediction, and can adaptively adjust the weight distribution during iteration, reducing the bias and overfitting risk of a single model.
Smart Images
Figure CN121389700A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of wind speed prediction of wind power generation, and more particularly relates to a wind speed combination prediction method and system based on reinforcement learning, a storage medium and a product. BACKGROUND
[0002] High-precision wind speed prediction can improve power generation efficiency and reduce operation and maintenance costs, and plays an important role in wind energy resource development and utilization, wind farm operation and realization of the "double carbon" target, and provides effective information support for real-time safe and reliable operation and scheduling scheme formulation of power grids. Due to the synergistic coupling effect of periodic and fluctuating changes in climate, wind speed series presents the characteristics of high randomness, intermittency and volatility, making prediction difficult and affecting the production of renewable energy. At present, wind speed prediction models are mainly divided into statistical models, machine learning models and combination prediction models. Due to the strong nonlinear characteristics of wind speed, the prediction effect of statistical models is generally poor. Machine learning models can effectively extract complex nonlinear relationships from multiple feature inputs, but have problems such as slow convergence speed, difficulty in determining parameters, and overfitting, which limit further improvement of their precision. Single models may be more sensitive to certain data anomalies or model bias, and combination prediction models can offset prediction errors by reasonably combining multiple strong and heterogeneous models, reducing the impact of specific bias or overfitting of single models, and have stronger robustness, and have been widely used in ultra-short-term wind speed prediction. In the combination prediction modeling process, there is a problem of weight distribution. Common weight distribution methods include average weight method, linear regression method, intelligent optimization algorithm, etc. Compared with single prediction models, the prediction accuracy of combination models has been improved to some extent, but it is still insufficient to accurately depict the wind speed variation law in complex environments.
[0003] Therefore, it is necessary to develop research on combination model weight distribution. SUMMARY
[0004] In view of the above defects or improvement needs of the prior art, the present application provides a wind speed combination prediction method and system based on reinforcement learning, a storage medium and a product, which aims to solve the technical problems of unreasonable combination model weight distribution and large prediction error in the prior art when developing ultra-short-term wind speed prediction.
[0005] To achieve the above-mentioned purpose, according to the first aspect of the present application, a wind speed combination prediction method based on reinforcement learning is provided, comprising the following steps: A plurality of different neural network models are trained respectively by using a training set, and a plurality of wind speed prediction models are obtained after training; A multi-armed slot machine model from reinforcement learning is introduced, with each wind speed prediction model corresponding to one arm of the slot machine; an ε-greedy strategy is used to determine the action selection for each time, with the action corresponding to the wind speed prediction model, and the corresponding reward value is determined based on the prediction error of the wind speed prediction model on the validation set; the training set and validation set are constructed based on wind speed time series data. After completing the preset number of action selections, the corresponding weight is determined based on the average reward of each wind speed prediction model. Each wind speed prediction model makes wind speed predictions separately, and the prediction results of each wind speed prediction model are weighted and combined based on the weights to obtain the final wind speed prediction result.
[0006] As a further preferred embodiment, the neural network model employs an LSTM model and an XGBoost model.
[0007] As a further optimization, an ε-greedy strategy is used to determine the action selection for each step, as follows:
[0008]
[0009]
[0010] In the formula, It is time t The selected action; It is a value between 0 and 1, representing the probability of choosing a random action; It's about actions. The estimated reward function; It is the first n Action during the second selection The current estimated reward, It is an action The new estimated reward; Indicates the deadline t ,action The number of times it was selected; It is time t The actual reward observed after selecting an action. and They are time t The model's predicted values and the actual values in the validation set.
[0011] As a further optimization, the weights are determined based on the average reward value corresponding to each wind speed prediction model, specifically as follows: After the preset number of action selections are completed, an average reward value of each wind speed prediction model is obtained, the average reward value being calculated based on actual rewards; the absolute value of the average reward value is taken as an inverse, and a weight is distributed according to a relative proportion obtained; each weight is normalized so that the sum is 1, thereby determining the weight of each wind speed prediction model.
[0012] As a further optimization, the weight is determined according to the average reward value corresponding to each wind speed prediction model, specifically:
[0013] In the formula, , respectively represent the average reward value of the first k th wind speed prediction model, j th wind speed prediction model, is a positive constant to prevent the denominator from being zero, is the weight of the first k th wind speed prediction model, K is the number of wind speed prediction models.
[0014] As a further optimization, a rolling prediction method is used in wind speed prediction: the final wind speed prediction result at the current time is incorporated into the wind speed sequence to form a new wind speed sequence; each wind speed prediction model performs wind speed prediction based on the new wind speed sequence, and the prediction results of each wind speed prediction model are combined by weighting based on the weight, to obtain the final wind speed prediction result at the next time; through the above process, a plurality of time combined prediction results can be obtained, and multi-step combined prediction of wind speed can be realized.
[0015] As a further optimization, historical wind speed time series data of a weather station is collected to construct an original wind speed data set; a variational mode decomposition (VMD) algorithm is used to decompose the original wind speed data set into a plurality of intrinsic mode functions, and a denoised wind speed data set is reconstructed; then the denoised wind speed data set is divided into a training set, a validation set and a test set.
[0016] According to a second aspect of the present application, a wind speed combined prediction system based on reinforcement learning is provided, comprising a processor, the processor being configured to execute the wind speed combined prediction method based on reinforcement learning.
[0017] According to a third aspect of the present application, a computer readable storage medium is provided, having a computer program stored thereon, the computer program being executed by a processor to implement the wind speed combined prediction method based on reinforcement learning.
[0018] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program, the computer program being executed by a processor to implement the wind speed combined prediction method based on reinforcement learning.
[0019] Overall, compared with the prior art, the above technical solutions conceived by the present application mainly have the following technical advantages: 1. In view of the problem of unreasonable weight distribution and large prediction error in the existing combination model, the present application introduces the multi-armed bandit epsilon-greedy strategy in reinforcement learning, and reasonably distributes the weights according to the prediction performance of different prediction models. This method avoids the shortcomings of manually setting weights or relying on a single optimization algorithm, can adaptively adjust the weight distribution, improves the robustness and generalization ability of the model, and can provide more accurate iterative input values for multi-step prediction, providing a new idea and method for ultra-short-term wind speed prediction.
[0020] 2. In view of the problem that a single machine learning model is more sensitive to data anomalies or model bias, the present application constructs a combination prediction method, which combines two strong and heterogeneous models of LSTM and XGBoost: the LSTM model can effectively capture the time series dependence and long-term correlation of wind speed data, and the XGBoost model can fully utilize nonlinear features and has strong generalization performance; the two are complementary and can offset the prediction error to a certain extent, reduce the bias or overfitting risk of a single model, significantly improve the prediction accuracy, and realize more accurate multi-step prediction of ultra-short-term wind speed.
[0021] 3. Compared with traditional combination methods such as average weight method, linear regression method or intelligent optimization algorithm, the present application determines the weight through reinforcement learning (multi-armed bandit) strategy, which can continuously utilize prediction result feedback for optimization in the iteration process, has the advantages of strong adaptability, no need for complex hyperparameter optimization, high computational efficiency, etc., and thus shows more stable prediction performance under different prediction steps and different weather conditions. BRIEF DESCRIPTION OF DRAWINGS
[0022] Figure 1 Flow chart of the wind speed combination prediction method based on reinforcement learning of the embodiment of the present application; Figure 2 Figures (a) and (b) are schematic diagrams of the original wind speed data set and the denoised wind speed data set provided by the embodiment of the present application; Figure 3 Figures (a) and (b) are schematic diagrams of the original wind speed data set and the denoised wind speed data set provided by the embodiment of the present application; Figure 4 Figures (a) and (b) are schematic diagrams of the original wind speed data set and the denoised wind speed data set provided by the embodiment of the present application; Figure 5 Figures (a) and (b) are schematic diagrams of the original wind speed data set and the denoised wind speed data set provided by the embodiment of the present application; DETAILED DESCRIPTION
[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0024] This invention provides a wind speed combination prediction method based on reinforcement learning, such as... Figure 1 As shown, it includes the following steps: S1. Collect historical wind speed time-series data from meteorological stations to construct the original wind speed dataset. Use the VMD algorithm to reconstruct the noise-reduced wind speed dataset. Divide the noise-reduced wind speed dataset into training set, validation set, and test set.
[0025] Specifically, the Variational Mode Decomposition (VMD) algorithm is used to decompose the original wind speed dataset into multiple intrinsic mode functions, and then reconstruct the denoised wind speed dataset; the constrained variational problem is expressed as follows:
[0026] In the formula, The first after VMD decomposition k One eigenmode function component; For the first k The instantaneous frequency of each eigenmode function component; Hilbert transform; It is the Dirac function; This is the convolution symbol.
[0027] S2. Train several different neural network models using the training set to obtain several wind speed prediction models after training.
[0028] Specifically, the neural network model includes, but is not limited to, LSTM model, XGBoost model, CNN model, MLP model, GRU model, TCN model, Transformer model, and other neural network models capable of predicting wind speed time series data.
[0029] Furthermore, the neural network models used are LSTM and XGBoost. The formula for the LSTM deep neural network model is as follows:
[0030] In the formula, , , They are time t Forget gate, input gate, output gate; It is timet Input; It is time t The hidden state; It is time t The candidate cell state; It is time t The unit state; W and b These are weights and biases; This represents element-wise multiplication; It is the sigmoid function.
[0031] The XGBoost model formula is as follows:
[0032] In the formula, It is the first i One predicted value; It is the first k A decision tree; K It is the total number of trees; It is a regular term; It is a parameter that controls the number of leaf nodes in the tree; T It is the number of leaf nodes in the tree; It is the L2 regularization parameter; It is the first j The score for each leaf; It is the loss function.
[0033] Furthermore, grid optimization is used on the validation set to adjust the model hyperparameters.
[0034] S3. Introduce a multi-armed slot machine from reinforcement learning. Each wind speed prediction model corresponds to one arm of the slot machine. Use an ε-greedy strategy to select the action (i.e., the wind speed prediction model) and assign weights to it based on the prediction performance of the wind speed prediction model on the validation set.
[0035] Specifically, the action to be taken is selected using the ε-greedy strategy of a multi-armed slot machine:
[0036]
[0037]
[0038] In the formula, It is time t The chosen action, that is, in time t The selected prediction model; It is a value between 0 and 1, representing the probability of choosing a random action; It is an action the current estimated reward (average value based on past experience) of is the action a new estimated reward of is updated iteratively by combining ; denotes the deadline t , the number of times the action is selected; is the time t the actual reward observed after selecting the action, i.e., negative error, and are the predicted and measured values of the time t , respectively.
[0039] The weights are determined according to the average reward values of the respective wind speed prediction models, specifically:
[0040] In the formula, , denote the average reward values of the first k and the first j wind speed prediction models, respectively, and the average reward value is the average value of all actual rewards (negative errors) corresponding to the wind speed prediction model after the reinforcement learning ends; is a positive constant to prevent the denominator from being zero, is the weight of the first k wind speed prediction model, K is the number of wind speed prediction models.
[0041] S4, Wind speed prediction: input the obtained wind speed sequence of the previous period of time into each wind speed prediction model to obtain the prediction result of the next time t ; based on the weights of the respective wind speed prediction models, the prediction results of each model are combined by weighting to obtain the combined prediction result at time t .
[0042] The combined prediction result is incorporated into the wind speed sequence in a rolling prediction manner to form a new input window, and the prediction is continued to obtain the combined prediction result at time t +1; through the above process, the combined prediction results at multiple times can be obtained, thereby realizing multi-step combined prediction.
[0043] The following is a specific embodiment: (1) The wind speed at a height of 50 meters at the Guinan Meteorological Station in Qinghai Province (north latitude 35°35', east longitude 100°45', altitude 3120 meters) is taken as the research object, and the measured 10-minute wind speed data of the meteorological station from January 1, 2017 to December 31, 2017 is as follows: Figure 2In the middle (a). However, wind speed data is often affected by various factors, such as equipment error, environmental noise, etc., resulting in noise in the data. In order to more accurately analyze and predict wind speed, the VMD algorithm is used to decompose the original wind speed data set into multiple intrinsic mode functions, and the noise-reduced wind speed data set is reconstructed, as shown in Figure 2 In the middle (b).
[0044] (2) The processed noise-reduced wind speed data set is used as the model input, and the training set, validation set, and test set are divided in the ratio of 8:1:1. The training set is used for model training, and the validation set is used to calculate the prediction error and as the reinforcement learning reward basis. LSTM model and XGBoost model are constructed respectively, as shown in Figure 3 . The rolling prediction scheme is used on the validation set, with the wind speed at 36 consecutive time points as input to predict the wind speed at the next time point; the time window is shifted by one step and the same mode is used, and the cycle is rolled 5 times to get the prediction results of the future 10min, 20min, 30min, 40min, 50min, 60min, and the instantaneous reward value is calculated based on the prediction error to update the average reward of each model. After completing the preset iteration, the final weight of each model is determined. Then, the same rolling prediction scheme is used on the test set to obtain the prediction results of each model, and the prediction results of different models are combined by weighting based on the weights, thereby obtaining the multi-step combined prediction results.
[0045] The two models are trained on the training set, and the model hyperparameters are adjusted on the validation set using grid search. The LSTM model selects the number of neurons, batch size and learning rate as three hyperparameters, and the optimization ranges are [8, 16, 32, 64, 128], [16, 32, 64, 128, 256, 512] and [0.0001, 0.001, 0.01, 0.1] respectively, and the optimal result is [64, 256, 0.01]; the XGBoost model selects the maximum depth, the number of sub-trees and the learning rate as three hyperparameters, and the optimization ranges are [2, 3, 4, 5, 6, 7, 8], [100, 200, 400, 800, 1600, 3200] and [0.05, 0.1, 0.15, 0.2, 0.25, 0.3] respectively, and the optimal result is [3, 800, 0.25]. The optimal model is used to predict on the test set to obtain the multi-step prediction results of the two models.
[0046] (3) A combined model weight distribution method based on reinforcement learning is proposed. Specifically, the multi-armed bandit ε-greedy strategy in reinforcement learning is introduced to distribute weights to the two models according to their prediction performance, and the multi-step combined prediction process is as shown in Figure 4 .
[0047] In the epsilon-greedy strategy, the weight update of the action is based on the difference between the observed return and the current estimated value. In this experiment, the two-arm slot machine (predicted values on the validation set of the LSTM and XGBoost models) is set to 5252 times (the length of the validation set) of playing the slot machine, with an epsilon of 0.9, and the negative error is used as the reward. The average reward values of the LSTM and XGBoost models are -0.25952996 and -0.35993069, respectively, and the weights of the LSTM and XGBoost models are 0.581 and 0.419, respectively (rounded to three significant digits, the same below).
[0048] (4) The test set prediction results of the combined LSTM and XGBoost models.
[0049] To verify the prediction accuracy of the proposed method, the 50-meter height wind speed at Guinan Meteorological Station from January 1, 2017 to December 31, 2017 is used as an example for multi-step short-term wind speed prediction. The weight distribution results of the average weight method, locust optimization algorithm, ridge regression method, and reinforcement learning method of the LSTM and XGBoost models are [0.5, 0.5], [0.819, 0.181], [0.928, 0.072], and [0.581, 0.419]. The prediction results of the proposed method and existing methods are shown in Figure 5 The mean absolute error (MAE), root mean square error (RMSE), and R-square index (R 2 ) are selected as evaluation indicators. Figure 5 (a) to (f) are the prediction values of the single models and combined models on the test set for the prediction periods of 10min to 60min. Taking the 60min prediction period as an example, the MAEs of the LSTM model, XGBoost model, average weight combination model, locust optimization combination model, ridge regression combination model, and reinforcement learning combination model are 1.423, 1.399, 1.383, 1.399, 1.413, and 1.386, respectively. The RMSEs are 1.893, 1.892, 1.857, 1.867, 1.885, and 1.856, respectively. The R 2 s are 0.701, 0.701, 0.711, 0.709, 0.703, and 0.712, respectively. The MAE and RMSE of the proposed method are the smallest, and the R 2 square is the largest, with the best prediction accuracy. The results for other prediction periods are the same. Overall, the accuracy of the proposed prediction method is better than that of the existing single models and combined models, proving the effectiveness of the method.
[0050] Those skilled in the art will readily understand that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A wind speed combination forecasting method based on reinforcement learning, characterized in that, The method comprises the following steps: a plurality of different neural network models are trained respectively through a training set, and a plurality of wind speed prediction models are obtained after the training is completed; A multi-armed bandit model in reinforcement learning is introduced, and each wind speed prediction model corresponds to an arm of the bandit; an ε-greedy strategy is used to determine the action selection each time, the action corresponds to a wind speed prediction model, and the reward value is determined according to the prediction error of the wind speed prediction model on a verification set; the training set and the verification set are constructed based on wind speed time series data; After a preset number of action selections are completed, the corresponding weight is determined according to the average reward value of each wind speed prediction model; Each wind speed prediction model performs wind speed prediction, and the prediction results of each wind speed prediction model are combined by weighting based on the weight, to obtain a final wind speed prediction result. 2.The wind speed combination forecasting method based on reinforcement learning according to claim 1, wherein, The neural network model uses an LSTM model and an XGBoost model. 3.The wind speed combination forecasting method based on reinforcement learning according to claim 1, wherein, The ε-greedy strategy is used to determine the action selection each time, and the specific process is as follows: where is the time t of the selection of the action; is a value between 0 and 1 representing the probability of selecting a random action; is the estimated reward function for the action ; is the current estimate of the reward for the action n at the time of the selection; is the new estimate of the reward for the action ; represents the number of times the action t was selected up to time ; is the actual reward observed after selecting the action at time t , and are the model predicted and actual values in the validation set, respectively, at time t . 4.The wind speed combination forecasting method based on reinforcement learning according to claim 3, wherein, The weight is determined according to the average reward value of each wind speed prediction model, and the specific process is as follows: After a preset number of action selections are completed, the average reward value of each wind speed prediction model is obtained, the average reward value is calculated based on the actual reward; the absolute value of the average reward value is taken as an inverse, and the weight is distributed according to the relative proportion obtained; the weights are normalized so that the sum is 1, thereby determining the weight of each wind speed prediction model. 5.The wind speed combination forecasting method based on reinforcement learning according to claim 4, wherein, The weight is determined according to the average reward value of each wind speed prediction model, and the specific process is as follows: In the formula, , They represent the first k The average reward value of the j-th wind speed prediction model. To prevent positive constants with a denominator of zero, For the first k The weights of each wind speed prediction model K This represents the number of wind speed prediction models.
6. The wind speed combination prediction method based on reinforcement learning according to any one of claims 1-5, characterized in that, A rolling prediction method is used in wind speed prediction: the final wind speed prediction result at the current time is incorporated into the wind speed sequence to form a new wind speed sequence; each wind speed prediction model performs wind speed prediction based on the new wind speed sequence, and the prediction results of each wind speed prediction model are combined by weighting based on the weight, to obtain the final wind speed prediction result at the next time; Through the above process, the combined prediction result at multiple times can be obtained, and multi-step combined prediction of wind speed can be realized.
7. The wind speed combination forecasting method based on reinforcement learning according to any one of claims 1-5, characterized in that, Historical wind speed time series data of a weather station is collected to construct an original wind speed data set; a VMD algorithm is used to decompose the original wind speed data set into a plurality of intrinsic mode functions, and a denoised wind speed data set is obtained by reconstruction; then the denoised wind speed data set is divided into a training set, a verification set and a test set. 8.A wind speed combination forecasting system based on reinforcement learning, characterized in that, The processor is used to execute the wind speed combined prediction method based on reinforcement learning according to any one of claims 1-7.
9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the wind speed combined prediction method based on reinforcement learning according to any one of claims 1-7.
10. A computer program product, characterised in that, The computer program is executed by the processor to realize the wind speed combined prediction method based on reinforcement learning according to any one of claims 1-7.