Monthly runoff prediction method based on improved stacking integration strategy
By improving the stacking integration strategy, integrating multiple deep learning models and giving weights, the problem of traditional methods ignoring the model correlation relationship during K-fold cross-validation is solved, and the accuracy of monthly runoff prediction is improved.
Patent Information
- Application Number
- CN202510370087.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-27
- Publication Date
- 2025-04-25
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The traditional stacking ensemble method ignores the correlation between different models during K-fold cross-validation, limiting the prediction performance of the basic model on different folds, thereby affecting the prediction accuracy of the final ensemble model.
By improving the stacking integration strategy, multiple steps are used to integrate multiple deep learning models and weight different meta models to form the final integration model. During the K-fold cross-validation process, this method combines the verification sets and test set prediction results of multiple basic models on each fold to form a new training set and verification set for training metamodels, and assigns weights to each metamodel based on the evaluation index of the prediction results.
By weighted summing the prediction results of different basic models, the traditional method is compensated for the shortcomings of ignoring the correlation relationship of the model, deeply mining and integrating the prediction performance of different models, and improving the accuracy of monthly runoff prediction.
Smart Images
Figure CN119884762A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of runoff forecasting, and in particular to a monthly runoff forecasting method based on an improved stacking integration strategy. Background Art
[0002] In the field of monthly runoff prediction, with the development of deep learning technology, data-driven models that focus on building a mapping relationship between input features and prediction targets are gradually replacing traditional process-driven models. However, single data-driven models, such as support vector machines, long short-term memory networks, random forests, etc., often have simple model structures and fewer parameters, resulting in poor performance in monthly runoff prediction. Therefore, with the deepening of research, scholars have proposed to use a multi-model integration method based on the stacking strategy to integrate multiple single models to improve the overall prediction performance of the model.
[0003] The traditional stacking integration method is to use K-fold cross-validation for different basic models, first concatenate the sub-validation sets of each basic model to form a complete validation set, average the prediction results of each fold model on the test set to get the complete test set, and then merge the complete validation set of each model as the training set of the meta-model to train the meta-model. Finally, merge the complete test set of each model as the input of the trained meta-model to obtain the final integrated prediction result. The specific flow chart is as follows Figure 1 shown.
[0004] However, in the integration process of the traditional stacking integration method, the correlation between different models when facing the same training set when using K-fold cross-validation is ignored. This, to a certain extent, limits the prediction performance of the mining basic model on different folds, thereby affecting the prediction accuracy of the final integrated model. Summary of the invention
[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a monthly runoff prediction method based on an improved stacking integration strategy, which aims to solve the problem that the traditional stacking integration method lacks the construction of the correlation between different models during k-fold cross validation, thereby enabling the integrated model integrated using this technology to show better prediction performance in complex monthly runoff prediction and improve the accuracy of monthly runoff prediction.
[0006] To achieve the above objectives, according to one aspect of the present invention, a method for medium- and long-term monthly runoff prediction is provided, which improves the traditional stacking integration method to integrate multiple deep learning models to obtain an integrated model ultimately used for monthly runoff prediction.
[0007] A monthly runoff prediction method based on an improved stacking integration strategy includes the following steps: S1. Data processing: The monthly runoff and monthly rainfall data in the original training samples were normalized, and the normalized data were divided into the training set and the test set according to the first 75% and the last 25% respectively. The time series data were converted into feature samples and targets, so that each time the next monthly runoff value was predicted, the historical values of monthly runoff and monthly rainfall in the previous 12 months needed to be input into the model. S2. Select multiple deep learning models or machine learning models as basic models and meta-models; S3. Integrate multiple basic models with the meta-models obtained in multiple steps S2 according to the improved stacking integration method, assign weights to different meta-models, and use the weights as the final integrated model; after denormalizing the prediction results obtained by the integrated model, the final prediction results can be obtained; S4. After the prediction result obtained in step S3 is denormalized, the actual runoff prediction value is obtained.
[0008] Preferably, each training sample in step S1 contains historical data of runoff and rainfall in the previous 12 months.
[0009] Preferably, the normalized data in step S1 is scaled to the interval [0, 1] to avoid large differences between data.
[0010] Preferably, step S3 specifically includes the following steps: S31: Use K-fold cross validation on the original training sets of multiple basic models respectively, and divide the training sets into k folds, that is, each model has k sub-training sets and validation sets; S32: In different folds, the sub-training set obtained in step S31 is input into the basic model for training. After the training is completed, prediction is performed on the remaining validation set and test set to obtain the prediction results of the validation set and test set of different basic models in each fold; S33: horizontally merge the prediction results of the validation set and the test set of multiple basic models on each fold, use the result of the validation set as the training set of the meta-model, and use the result of the test set as the test set of the meta-model, and perform prediction after the meta-model is trained with the training set to obtain the prediction results of the meta-model training set and the test set; train k meta-models in total and obtain the prediction results of k groups of meta-models on their training set and test set; S34: assigning a weight to each meta-model according to the prediction results of the k meta-models on their training sets, thereby obtaining respective weights of the multiple meta-models; S35: Multiply the prediction results of the k meta-models on their test sets by their respective weights and then sum them up as the prediction result of the final model.
[0011] Preferably, in step S31, the proportion of each sub-training set and validation set in the original training set is (k-1):1, and they are not repeated; k is a positive integer and k≥3.
[0012] Preferably, the evaluation indicators used in step S34 to assign weights to each meta-model for prediction results include root mean square error RMSE, mean square error MSE, mean absolute error MAE, mean absolute percentage error MAPE and determination coefficient .
[0013] Preferably, the weight formula in step S34 is: ; ; ; ; ; ; Where: RMSE is root mean square error, MSE is mean square error, MAE is mean absolute error, MAPE is mean absolute percentage error, and R² is the coefficient of determination; and Respectively and Evaluation indicators of the meta-model; and Represent the actual value and predicted value of monthly runoff, respectively. For the The weight of the meta-model, .
[0014] As a preference, the prediction result in step S35 S for: ; Where: is the prediction result of the kth meta-model on the test set.
[0015] To achieve the above-mentioned purpose, according to another aspect of the present invention, the present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the monthly runoff prediction method based on the improved stacking integration strategy.
[0016] In general, the above technical scheme conceived by the present invention has the following beneficial effects compared with the prior art: 1. Compared with the traditional stacking integration method, by weighted summing up the integration results of different basic models when using K-fold cross validation to face the same training set, it makes up for the fact that the traditional stacking integration method ignores the correlation between different basic models when using K-fold cross validation to face the same training set, and deeply explores and integrates the prediction performance of different basic models when facing different training sets.
[0017] 2. The present invention divides the original data by using K-fold cross validation, and uses different basic models to train and predict the divided data, and then fuses them into new training sets and validation sets to input into the meta-model. This method is conducive to mining hidden information in limited data and improving data utilization.
[0018] 3. The present invention assigns different weights to each meta-model according to a selected evaluation index in the meta-model, which can fully consider the difference in the integrated prediction ability of each meta-model at each fold. Compared with the traditional stacking integration method, it avoids the limitations of single meta-model integration and makes the integration result more flexible and accurate.
[0019] 4. Applying the present invention to monthly runoff prediction can improve the accuracy of monthly runoff prediction. In addition, the present invention can also be extended to other time series prediction problems, and has strong adaptability and versatility. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is a flow chart of the traditional stacking integration method.
[0021] Figure 2 It is a flowchart of monthly runoff prediction based on the improved stacking ensemble method.
[0022] Figure 3 This is a comparison chart between the predicted value and the actual value of the monthly runoff at the Miaowei Hydropower Station on the Lancang River using the weighted stacking integration model constructed using the stacking integration method proposed in the present invention.
[0023] Figure 4 This is a comparison chart between the monthly runoff prediction and actual value of the weighted stacking integration model constructed by the stacking integration method proposed in the present invention and the integration model obtained by the traditional stacking integration method at the Miaowei Hydropower Station on the Lancang River. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0025] Embodiment 1: The present invention takes the historical monthly runoff of Miaowei Hydropower Station, the seventh-level hydropower station in the lower reaches of the Lancang River, as an example to verify the effect of the present invention. Figure 2 The monthly runoff prediction flow chart based on the improved stacking integration method is shown to perform monthly runoff prediction to reflect the effect achieved by the present invention.
[0026] S1. Data processing: The monthly runoff and monthly rainfall data in the original training samples were normalized, and the normalized data were divided into the training set and the test set according to the first 75% and the last 25% respectively. The time series data were converted into feature samples and targets, so that each time the next monthly runoff value was predicted, the historical values of monthly runoff and monthly rainfall in the previous 12 months needed to be input into the model. The data used are the historical runoff data of Miaowei Hydropower Station from 1953 to 2014 and the average total rainfall data of the Lancang River Basin (a total of 744 data of each type). The data from January 1953 to August 1995 were used as the training set for training the model, and the data from September 1995 to December 2014 were used as the test set for testing the model results.
[0027] The divided model is normalized to reduce all data proportionally to the interval [0,1], and then the time series data is divided into feature samples and targets, that is, the model will need to consider the number of past time steps when predicting the next value. Because the monthly runoff data shows a periodic fluctuation with a cycle of 12 months, each prediction of the next monthly runoff value requires the input of the historical values of monthly runoff and monthly rainfall in the previous 12 months to the model.
[0028] S2. Select multiple deep learning models or machine learning models as basic models and meta-models; The long short-term memory network (LSTM), gated recurrent unit (GRU) and temporal convolutional network (TCN) are used as the basic models of the integrated model, and the structure and hyperparameters of each model are as follows: Long Short-Term Memory Network (LSTM): The input data passes through the hidden layer and the fully connected layer of the long short-term memory network in sequence to obtain the prediction results. The network has a total of 6 hyperparameters, namely the feature size of the input model (set to 2, including historical runoff and historical rainfall), the number of hidden layers (set to 2), the number of neurons in each hidden layer (set to 18), the training cycle (set to 400), the learning rate (set to 0.006), and the feature size of the output (set to 1).
[0029] Gated Recurrent Unit (GRU): The input data passes through the hidden layer and the fully connected layer of the gated recurrent unit in sequence to obtain the prediction result. The network has a total of 6 hyperparameters, namely the feature size of the input model (set to 2, including historical runoff and historical rainfall), the number of hidden layers (set to 2), the number of neurons in each hidden layer (set to 6), the training cycle (set to 400), the learning rate (set to 0.006), and the feature size of the output (set to 1).
[0030] Temporal Convolutional Network (TCN): The input data passes through the TCN layer and the fully connected layer of the temporal convolutional network in sequence to obtain the prediction results. The network has a total of 6 hyperparameters, namely the feature size of the input model (set to 2, including historical runoff and historical rainfall), the number of channels of the convolution layer (set to [12,13], indicating that the first and second layers in TCN have 12 and 13 channels respectively), the size of the convolution kernel (set to 3), the training cycle (set to 400), the learning rate (set to 0.006), and the feature size of the output (set to 1).
[0031] The Lightweight Gradient Boosting Machine (LightGBM) model is used as the meta-model. The hyperparameters of each meta-model are set as follows: Meta-model 1 (Light Gradient Boosting Machine 1): It has a total of 6 hyperparameters, namely, model evaluation metric (set to mse), gradient boosting type (set to gbdt), minimum amount of data required for each leaf node (set to 5), maximum depth of decision tree (set to 3), learning rate (set to 0.058), training epochs (set to 157).
[0032] Meta-model 2 (Light Gradient Boosting Machine 2): has a total of 6 hyperparameters, namely, model evaluation metric (set to mse), gradient boosting type (set to gbdt), minimum amount of data required for each leaf node (set to 8), maximum depth of decision tree (set to 4), learning rate (set to 0.07), training period (set to 53).
[0033] Meta-model 3 (Light Gradient Boosting Machine 3): has a total of 6 hyperparameters, namely, model evaluation metric (set to mse), gradient boosting type (set to gbdt), minimum amount of data required for each leaf node (set to 7), maximum depth of decision tree (set to 3), learning rate (set to 0.03), training epochs (set to 168).
[0034] S3. Integrate multiple basic models with the meta-models obtained in multiple steps S2 according to the improved stacking integration method, assign weights to different meta-models, and use the weights as the final integrated model; after denormalizing the prediction results obtained by the integrated model, the final prediction results can be obtained; Step S3 specifically includes the following steps: S31: Use K-fold cross validation on the original training sets of multiple basic models respectively, and divide the training sets into k folds, that is, each model has k sub-training sets and validation sets; S32: In different folds, the sub-training set obtained in step S31 is input into the basic model for training. After the training is completed, prediction is performed on the remaining validation set and test set to obtain the prediction results of the validation set and test set of different basic models in each fold; S33: horizontally merge the prediction results of the validation set and the test set of multiple basic models on each fold, use the result of the validation set as the training set of the meta-model, and use the result of the test set as the test set of the meta-model, and perform prediction after the meta-model is trained with the training set to obtain the prediction results of the meta-model training set and the test set; train k meta-models in total and obtain the prediction results of k groups of meta-models on their training set and test set; S34: assigning a weight to each meta-model according to the prediction results of the k meta-models on their training sets, thereby obtaining respective weights of the multiple meta-models; S35: Multiply the prediction results of the k meta-models on their test sets by their respective weights and then sum them up as the prediction result of the final model.
[0035] Specifically, 3-fold cross validation is used on the original training sets of multiple basic models, and the training sets are divided into 3 folds, that is, each model has 3 sub-training sets and validation sets, such as Figure 2 The medium and long short-term memory network is shown in the column. The ratio of each sub-training set and validation set in the original training set is 2:1, and they are not repeated.
[0036] from Figure 2 As can be seen from the figure, in the same fold, the sub-training set and sub-validation set of each model are the same. In different folds, the obtained sub-training set is input into the model for training, and prediction is performed on the remaining validation set and test set. The prediction results of the validation set and test set of different basic models on each fold are obtained.
[0037] The prediction results of the validation set and test set of multiple basic models on each fold are merged horizontally and used as the training set and test set of the meta-model for training and prediction respectively. A total of 3 meta-models are trained and 3 groups of prediction results of the meta-models on their training set and test set are obtained.
[0038] The root mean square error (RMSE) of the prediction results of the three meta-models on their training sets is used to assign weights to each meta-model according to the following formula to obtain the weights of meta-model 1, meta-model 2, and meta-model 3: They are 0.3795, 0.3524 and 0.2680 respectively.
[0039] ; ;
[0040] Where: RMSE is the root mean square error; For the The root mean square error of each meta-model; For the The root mean square error of each meta-model; and Represent the actual value and predicted value of monthly runoff, respectively. For the The weight of the meta-model, .
[0041] In other embodiments, evaluation indicators including mean square error MSE, mean absolute error MAE, mean absolute percentage error MAPE and determination coefficient R² may also be used; The specific weight formula is: ; ; ; ; ; Where: RMSE is root mean square error, MSE is mean square error, MAE is mean absolute error, MAPE is mean absolute percentage error, and R² is the coefficient of determination; and Respectively and Evaluation indicators of the meta-model; and Represent the actual value and predicted value of monthly runoff, respectively. For the The weight of the meta-model, .
[0042] The prediction results of the three meta-models on their test sets are multiplied by their respective weights and then summed up as the prediction results of the final model. The prediction results of the weighted stacking ensemble model obtained by the improved stacking ensemble method are as follows: for: ; S4. The prediction result obtained in step S3 Perform the anti-normalization operation to obtain the final integrated prediction result.
[0043] like Figure 3As shown, a comparison chart of the monthly runoff prediction results of the Miaowei Hydropower Station in Example 1 based on the weighted stacked monthly runoff prediction integrated model obtained based on the proposed improved stacking integration method and the original value is shown. It can be seen from the figure that the weighted stacked integrated model can accurately predict the fluctuation cycle and peak and valley values of the original true value.
[0044] like Figure 4 As shown, it is a comparison chart of the monthly runoff prediction results and original values of the weighted stacking integrated model and the traditional stacking integrated model for the Miaowei Hydropower Station in Example 1. It can be seen from the figure that the traditional stacking integrated model has poor prediction effect on the peak value, and even has an abnormal situation of peak depression, but the weighted stacking integrated model has a more accurate peak prediction result than the traditional stacking integrated model.
[0045] The monthly runoff prediction results of the test set of the traditional stacking integration model and the weighted stacking integration model proposed in the present invention are evaluated by indicators, as shown in Table 1; .
[0046] It can also be seen from the prediction effect evaluation indicators of the weighted stacking integration model obtained based on the improved stacking integration strategy and the integration model obtained based on the traditional stacking strategy in Table 1 that whether it is the mean absolute error RMSE reflecting the absolute difference between the predicted value and the actual value, the Nash efficiency coefficient NSE measuring the degree of fit between the prediction results of the hydrological model and the observed value, or the correlation coefficient R reflecting the strength and direction of the linear relationship between the predicted value and the actual observed value, the weighted stacking integration model obtained by the improved stacking integration method proposed in the invention shows better prediction performance in monthly runoff prediction.
[0047] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A monthly runoff prediction method based on an improved stacking integration strategy, characterized in that: The following steps are involved: S1. Data processing: Normalize the monthly runoff and monthly rainfall data in the original training samples, divide the normalized data into training set and test set, and convert the time series data into feature samples and targets, so that when predicting the next monthly runoff value, the historical values of monthly runoff and monthly rainfall in the previous 12 months need to be input into the model; S2. Select multiple deep learning models or machine learning models as basic models and meta-models; S3. Integrate multiple basic models with the meta-models obtained in multiple steps S2 according to the improved stacking integration method, assign weights to different meta-models, and use the weights as the final integrated model; after denormalizing the prediction results obtained by the integrated model, the final prediction results can be obtained; S4. After the prediction result obtained in step S3 is denormalized, the actual runoff prediction value is obtained.
2. A monthly runoff prediction method based on an improved stacking integration strategy as claimed in claim 1, characterized in that: In step S1, the data is divided into a training set and a test set according to the first 75% and the last 25%.
3. The monthly runoff prediction method based on the improved stacking integration strategy according to claim 1, characterized in that: Each training sample in step S1 contains the historical data of runoff and rainfall in the previous 12 months.
4. The monthly runoff prediction method based on the improved stacking integration strategy according to claim 1, characterized in that: The normalized data in step S1 are scaled to the interval [0,1] to avoid large differences between data.
5. The monthly runoff prediction method based on the improved stacking integration strategy according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31: Use K-fold cross validation on the original training sets of multiple basic models respectively, and divide the training sets into k folds, that is, each model has k sub-training sets and validation sets; S32: In different folds, the sub-training set obtained in step S31 is input into the basic model for training. After the training is completed, prediction is performed on the remaining validation set and test set to obtain the prediction results of the validation set and test set of different basic models in each fold; S33: horizontally merge the prediction results of the validation set and the test set of multiple basic models on each fold, use the result of the validation set as the training set of the meta-model, and use the result of the test set as the test set of the meta-model, and perform prediction after the meta-model is trained with the training set to obtain the prediction results of the meta-model training set and the test set; train k meta-models in total and obtain the prediction results of k groups of meta-models on their training set and test set; S34: assigning a weight to each meta-model according to the prediction results of the k meta-models on their training sets, thereby obtaining respective weights of the multiple meta-models; S35: Multiply the prediction results of the k meta-models on their test sets by their respective weights and then sum them up as the prediction result of the final model.
6. A monthly runoff prediction method based on an improved stacking integration strategy as claimed in claim 5, characterized in that: In step S31, the proportion of each sub-training set and validation set in the original training set is (k-1):1, and they are not repeated; k is a positive integer and k≥3.
7. A monthly runoff prediction method based on an improved stacking integration strategy as claimed in claim 6, characterized in that: The evaluation indicators used in step S34 to assign weights to each meta-model for prediction results include root mean square error RMSE, mean square error MSE, mean absolute error MAE, mean absolute percentage error MAPE and determination coefficient R².
8. A monthly runoff prediction method based on an improved stacking integration strategy as claimed in claim 7, characterized in that: The weight formula in step S34 is: Where: RMSE is root mean square error, MSE is mean square error, MAE is mean absolute error, MAPE is mean absolute percentage error, and R² is the coefficient of determination; and Respectively and Evaluation indicators of the meta-model; and Represent the actual value and predicted value of monthly runoff, respectively. For the The weight of the meta-model, .
9. The monthly runoff prediction method based on the improved stacking integration strategy according to claim 5, characterized in that: The prediction result S in step S35 is: Where: is the prediction result of the kth meta-model on the test set.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer executes a monthly runoff prediction method based on an improved stacking integration strategy as described in any one of claims 1 to 9.