Multi-model optimal load forecasting method and device based on reinforcement learning
By employing a multi-model optimization method based on reinforcement learning, and utilizing Markov decision models and reward function feedback mechanisms, the problem of insufficient long-term stability and adaptability in power load forecasting is solved, achieving higher accuracy and more stable forecast results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 中能智新科技产业发展有限公司
- Filing Date
- 2025-11-20
- Publication Date
- 2026-05-19
AI Technical Summary
Existing power load forecasting methods have shortcomings in terms of long-term stability, adaptability, dynamic optimization, and forecast continuity. In particular, single models are difficult to cope with the influence of seasonal and meteorological factors, and existing fusion strategies lack adaptive adjustment capabilities and dynamic learning mechanisms.
A multi-model optimization method based on reinforcement learning is adopted, which uses Markov decision models for model selection and weight optimization, and combines sliding time windows, standardization processing and reward function feedback mechanisms to achieve dynamic selection and adaptive fusion of prediction models.
It improves the accuracy and stability of power load forecasting, enhances the system's adaptability, and avoids forecast discontinuity and instability caused by frequent model switching.
Smart Images
Figure CN121599204B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of power load forecasting technology, specifically to a multi-model optimization load forecasting method and apparatus based on reinforcement learning. Background Technology
[0002] Electricity load forecasting plays a crucial role in energy management and grid dispatching. Currently, traditional methods generally employ statistical models (such as ARIMA and Prophet), machine learning models (such as SVR, XGBoost, and LightGBM), and deep learning models (such as LSTM, GRU, and Transformer). In practical applications, to improve robustness and generalization ability, these models are often fused to combine their advantages. However, existing models and their fusion strategies still have some shortcomings. First, because electricity load is significantly affected by seasonality, holidays, and weather conditions, a single model struggles to maintain stable forecasting performance over the long term. Second, existing fusion strategies typically use fixed weighting, lacking the ability to adaptively adjust to changes in operating conditions. Third, existing models cannot update their strategies based on real-time forecast errors, lacking dynamic learning and optimization mechanisms. Fourth, existing fusion strategies simply select models based on error ranking, leading to frequent model switching and affecting forecast continuity and reliability.
[0003] Therefore, there is an urgent need for a load forecasting method that can achieve automatic learning and dynamic optimization in a multi-model forecasting system, so as to improve the accuracy, stability and adaptability of forecasting. Summary of the Invention
[0004] To overcome the shortcomings of the prior art, this application provides a multi-model optimization load forecasting method and apparatus based on reinforcement learning, specifically adopting the following technical solution:
[0005] A multi-model optimization load forecasting method based on reinforcement learning, the method comprising the following steps:
[0006] Data collection and feature construction are performed to obtain model input parameters;
[0007] Input the model input parameters into the candidate model library, and obtain the predicted values and uncertainty measures of different prediction models in the candidate model library respectively;
[0008] Model selection is performed on the prediction models in the candidate model library based on Markov decision models;
[0009] The selected prediction model is used to make predictions and obtain the corresponding predicted values;
[0010] The error assessment results are obtained by comparing the predicted values and the actual values.
[0011] The error assessment result is fed back to the reward function to generate a reward signal;
[0012] The Markov decision model is optimized based on the reward signal.
[0013] Optionally: The steps of data acquisition and feature construction to obtain model input parameters include:
[0014] Different types of raw data and model characteristic parameters are obtained respectively; the data types of the raw data include load dimension data, environmental dimension parameters and period dimension parameters, and the data types of the model characteristic parameters include historical error data and prediction uncertainty parameters of different prediction models.
[0015] The original data is preprocessed according to its data type to obtain preprocessed data; the preprocessing includes time alignment, missing value imputation, and anomaly correction.
[0016] Preprocessed data is processed using a sliding time window method to obtain corresponding statistical characteristics;
[0017] The parameter set is obtained by integrating preprocessed data, statistical features, and model characteristic parameters.
[0018] The different data in the parameter set are standardized separately to obtain the model input parameters.
[0019] Furthermore, the candidate model library includes multiple different types of prediction models, and the different types of prediction models run independently. The model input parameters are input into the corresponding prediction model, and the predicted value and uncertainty parameters of the corresponding model are output.
[0020] Furthermore: when selecting a prediction model from the candidate model library based on a Markov decision model, the action parameter is set to discrete selection or weighted selection.
[0021] Optionally: the discrete selection selects the best model from the candidate model library with the error evaluation result of a single prediction model as a reward; the weighted selection outputs the best model weight allocation result with the error evaluation result of the weighted prediction value and the true value of multiple prediction models as a reward.
[0022] Optionally, the method further includes the steps of: performing data distribution drift detection on the Markov decision model based on the error evaluation results; when a drift determination is triggered, pausing the reinforcement learning model prediction process and retraining or fine-tuning the weights, while updating the model input parameters and the candidate model library.
[0023] Optional: When optimizing the policy of a Markov decision model based on reward signals, a proximal policy optimization algorithm or a deep Q-learning algorithm can be used to implement policy learning.
[0024] Optional: The reward function is constructed by considering prediction accuracy, stability, risk control, uncertainty penalty, and model switching penalty; the reward function employs a loss-based reward structure.
[0025] ;
[0026] in For reward parameters; The mean absolute percentage error of the model's predicted values; For error terms Variance within the time window; Value at risk (VaR) for the model conditions; To predict the interval width; Let be the model weight vector at time t; Let be the model weight vector at time t-1; The Manhattan distance between the weight vectors of adjacent time steps; These are the weights of each part.
[0027] Optional: When performing data distribution drift detection on Markov decision models, use the CUSUM drift detection method or the ADWIN adaptive window detection method.
[0028] Furthermore, this application also discloses a multi-model optimization load forecasting device based on reinforcement learning, the device comprising:
[0029] The state parameter acquisition module is used for data acquisition and feature construction to obtain model input parameters;
[0030] The candidate model prediction module is used to input the model input parameters into the candidate model library and obtain the predicted values and uncertainty measures of different prediction models in the candidate model library respectively.
[0031] The reinforcement learning decision module is used to select a prediction model from the candidate model library based on the Markov decision model.
[0032] The prediction execution module is used to make predictions using the selected prediction model and obtain the corresponding predicted values.
[0033] The error assessment module is used to compare the predicted value and the actual value to obtain the error assessment result.
[0034] The reward generation module is used to feed back the error evaluation result to the reward function to generate a reward signal;
[0035] The strategy optimization module is used to optimize the strategy of the Markov decision model based on the reward signal.
[0036] The technical solution of this application achieves the following beneficial effects:
[0037] This application's multi-model optimization load forecasting method introduces a reinforcement learning decision-making mechanism to achieve dynamic selection and adaptive fusion of forecasting models, thereby overcoming the shortcomings of existing technologies in terms of operating condition adaptation, forecasting accuracy, and stability. Based on Markov decision-making, this method automatically selects the optimal model or weight combination according to the model's real-time characteristics and historical performance. Through a reward function and error feedback mechanism, continuous optimization of forecasting performance can be achieved. Simultaneously, the reward function considers risk-sensitive rewards and switching penalties, avoiding oscillations caused by frequent model switching. Attached Figure Description
[0038] Figure 1 This is a flowchart of the multi-model optimization load forecasting method based on reinforcement learning in the embodiments of this application.
[0039] Figure 2 This is a schematic diagram of the decision-making process based on the Markov decision model in the embodiments of this application.
[0040] Figure 3 This is a flowchart illustrating the reward function and feedback mechanism in the embodiments of this application.
[0041] Figure 4 This is a schematic diagram of the drift adaptive detection process in the embodiments of this application.
[0042] Figure 5 This is a structural diagram of the multi-model optimization load prediction device based on reinforcement learning in the embodiments of this application.
[0043] Figure 6 This is a structural diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0044] The present application will now be further described with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application and should not be construed as limiting the scope of protection of the present application. It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present application.
[0045] like Figure 1 As shown in the figure, this embodiment specifically discloses a multi-model optimization load forecasting method based on reinforcement learning, the method including the following steps:
[0046] First, data collection and feature construction are required to obtain the model input parameters.
[0047] Specifically, in this embodiment, different types of raw data can be collected in real time through the enterprise energy consumption monitoring system and external meteorological interface. Simultaneously, model characteristic parameters of the selected prediction model can be obtained. The data types of the raw data include load dimension data, environmental dimension parameters, and period dimension parameters, such as load, temperature, humidity, wind speed, and holiday types. The data types of the model characteristic parameters include historical error data and prediction uncertainty parameters of different prediction models.
[0048] In this embodiment, the original data can be preprocessed according to data type to obtain preprocessed data. The preprocessing includes time alignment, missing value imputation, and anomaly correction. Time alignment ensures that original data from different sources are synchronized in time, avoiding analysis errors caused by time misalignment. Missing value imputation fills in gaps in the original data, ensuring data integrity. Anomaly correction identifies and corrects abnormal data that deviates from the normal range, improving data reliability.
[0049] For the preprocessed raw data, a sliding time window approach can be used to obtain corresponding statistical features. These statistical features include trend, periodicity, volatility, and peak-to-trough ratio. The trend reflects the overall trend of data change within the time window; the periodicity captures the periodicity of the data; volatility measures the degree of fluctuation of the data within the time window; and the peak-to-trough ratio compares the peak and trough values of the data within the time window.
[0050] Finally, the preprocessed data, statistical features, and model characteristic parameters are integrated to obtain a parameter set; in this embodiment, the parameter set can be represented as:
[0051] ;
[0052] in It is represented as the state vector at time t; Representative load and its derived characteristics; Represents meteorological and external environmental characteristics; Represents time and periodic characteristics; Represents the historical error performance of each model; This represents the uncertainty characteristic of the model's predictions.
[0053] In addition, to eliminate the influence of dimensions, this embodiment also performs standardization processing on different data in the parameter set, such as standard score or robust scaler standardization, to obtain standardized model input parameters.
[0054] Then, the model input parameters are input into the candidate model library to obtain the predicted values and uncertainty measures of different prediction models in the candidate model library.
[0055] The candidate model library described in this embodiment includes multiple different types of prediction models. Specifically, it includes traditional statistical models, such as ARIMA prediction model and Prophet prediction model; machine learning models, such as SVR prediction model, XGBoost prediction model and LightGBM prediction model; and deep learning models, such as LSTM learning model and Transformer learning model.
[0056] Furthermore, the different types of prediction models mentioned above run independently. After inputting the model input parameters into the corresponding prediction model, the model will output the predicted value and uncertainty parameters of the corresponding model.
[0057] ;
[0058] ;
[0059] in Indicates that the i-th model is based on state. The predicted value; This represents the uncertainty in the model's predictions, calculated using methods such as quantile interval width, coherent variance, or Monte Carlo methods, and reflects the model's confidence in the prediction results; and K represents the total number of models.
[0060] Secondly, a model selection is performed on the prediction models in the candidate model library based on the Markov decision model.
[0061] like Figure 2 As shown, in this embodiment, the model optimization process is modeled as a Markov decision model, which includes four parts: state space, action space, reward parameters, and policy parameters.
[0062] The state space is based on the standardized state model input parameters output by the above data acquisition and feature construction, and it covers load characteristics, environmental characteristics, time characteristics, model historical error characteristics, model uncertainty characteristics, etc.
[0063] In this embodiment, the action space employs two modes: discrete selection and weighted selection. Discrete selection uses the error evaluation result of a single prediction model as a reward to select the best model from the candidate model library, and the predicted value is the result of the selected best model. Weighted selection, on the other hand, uses the error evaluation result between the weighted predicted values of multiple prediction models and the true value as a reward to output the best model weight allocation result, where the model weight can be expressed as:
[0064] ;
[0065] In addition, the above weights must meet the following requirements. .
[0066] Correspondingly, when a weighted selection method is used, the predicted value It is obtained by calculation using the following formula:
[0067] ;
[0068] in Let be the weights of the i-th model.
[0069] Furthermore, such as Figure 3 As shown, in this embodiment, the reward function of the Markov decision model is constructed by considering prediction accuracy, stability, risk control, uncertainty penalty, and model switching penalty. By balancing these objectives with preset coefficients, the reward value is maximized to achieve optimal decision-making. Specifically, this reward function preferably employs a loss-based reward mechanism.
[0070] ;
[0071] in For reward parameters; The mean absolute percentage error of the model's predicted values; For error terms Variance within the time window; Value at risk (VaR) for the model conditions; To predict the interval width; Let be the model weight vector at time t; Let be the model weight vector at time t-1; The Manhattan distance between the weight vectors of adjacent time steps; These are the weights of each part.
[0072] More detailed, This is used to represent the volatility of the prediction error over the most recent W steps, where the expression is:
[0073] ;
[0074] in Indicates the error term The variance within the time window [tW,t] reflects the stability of the model's predicted values; the smaller the fluctuation, the higher the reward. W is the sliding window length. This represents the error term at time k. This represents the sample mean of the error term within the time window.
[0075] The tail risk used to measure extreme errors is calculated using the following formula:
[0076] ;
[0077] ;
[0078] in F() The cumulative distribution function representing the error; F -1 (q) The quantile function represents the confidence level. q The maximum acceptable value for the lower error; VaR q Represents the value at risk, that is... q The probability error does not exceed this threshold; The representative condition is the risk value, indicating that it exceeds the risk value. VaR q The average error of this portion is used to characterize the average loss level under the worst-case scenario. It should be noted that... This constrains the risk robustness of the prediction model in this embodiment under extreme scenarios. A large value indicates that the system is unstable under high-risk conditions, therefore a higher penalty weight should be assigned to the reward function. γ, This is to avoid high-risk predictions by the model under extreme weather or peak load conditions.
[0079] also Used to penalize uncertainty caused by excessively wide prediction intervals:
[0080] ;
[0081] in The upper confidence bound for the prediction of the nth predicted sample; Let n be the lower confidence bound for the prediction of the nth predicted sample; N This represents the number of samples to be predicted. Generally, the confidence interval for a model is very narrow, meaning the prediction has small fluctuations and high confidence. The value is small; however, the model confidence interval is wide, meaning the model uncertainty is high. The value is relatively large.
[0082] In this embodiment, power load forecasting... Reflecting the confidence and reliability of the model: If An overly broad model, while safe, is too conservative and loses its practical predictive value; if... An overly narrow reward function indicates that the model underestimates uncertainty and may err under peak loads. Therefore, this embodiment requires balancing the reward function. The penalty term enables the model to automatically adjust the confidence interval width, achieving both accurate and stable predictions.
[0083] Furthermore, in this embodiment, a model switching penalty term is added to the reward function. It represents the Manhattan distance between the weight vectors at adjacent time points:
[0084] ;
[0085] The above formula is used to constrain the model switching frequency, suppressing prediction jumps and resource waste. Furthermore, weighting coefficients are used... The balance between the smoothness and adaptability of the weights can be controlled; generally, the system is more stable and changes less when the values are large.
[0086] Then, the selected prediction model is used to make a prediction and obtain the corresponding prediction value; the fundamental embodiment will call the corresponding prediction model or fuse the prediction values of multiple models according to the action output by the Markov decision model, and finally generate the prediction result.
[0087] Next, in this embodiment, the predicted results are compared with the actual load values to calculate error indicators, such as MAPE, RMSE, and CVaR, thereby obtaining error assessment results. Then, these error indicators are substituted into the reward function to generate reward signals, which are fed back to the Markov decision model, thus forming a closed-loop learning mechanism of prediction-error calculation-reward feedback-strategy optimization. This enables the system to automatically adjust its strategy based on historical performance, achieving dynamic optimization and adaptive operation.
[0088] Finally, this embodiment optimizes the Markov decision model based on the reward signal.
[0089] In this embodiment, the optimization objective can be defined as:
[0090] ;
[0091] The above formula shows that in the discount factor The expected summation of all future rewards is considered; this goal-driven strategy π is optimal in terms of long-term returns, not just in terms of single-step accuracy.
[0092] Furthermore, in this embodiment, a proximal policy optimization algorithm (PPO) or deep Q-learning (DQN) is used to implement policy learning, thereby improving stability and convergence speed. In this embodiment, the reinforcement learning agent continuously interacts with the environment to optimize the model selection strategy, enabling the system to possess self-learning and adaptive capabilities.
[0093] Furthermore, the load forecasting method in this embodiment may experience data distribution drift or model performance degradation during long-term operation. Therefore, to maintain forecast accuracy and robustness, the CUSUM drift detection method or the ADWIN adaptive window detection method can be preferably used to achieve drift detection and adaptive updating.
[0094] Specifically, the CUSUM drift detection method is an online monitoring algorithm based on deviation accumulation, which can be expressed as:
[0095] ;
[0096] in For the observations at time t, such as prediction error or model confidence index; v The expected mean; The cumulative deviation value at time t represents the degree of sustained error deviation. Let be the cumulative deviation value at time t-1.
[0097] By comparing the cumulative deviation value at time t with the drift detection threshold h ,when When a significant shift in the system state is detected, model retraining or parameter reset is triggered. This CUSUM drift detection method can quickly respond to sudden drifts and is suitable for real-time sensitive load scenarios.
[0098] More specifically, the ADWIN adaptive window detection method is an adaptive detection algorithm based on a sliding window, which can automatically adjust the size of the detection window in the data stream to determine whether the data distribution has changed. Its specific process is as follows:
[0099] First, set the current window as... W The system receives new samples x t Then, it will automatically be divided into two sub-windows, namely the left window. W 1 and the right window W 2, where the left window W 1 represents the old data segment, in the right window. W 2 represents a new data segment; the left window is defined respectively. W 1 and the right window W The mean of 2 is and Its corresponding length is n 1. n 2.
[0100] The ADWIN detection conditions are expressed as follows:
[0101] ;
[0102] in The significance level is generally set between 0.002 and 0.01. The dynamic confidence threshold is derived from the Hoeffding inequality.
[0103] Based on the above detection conditions, ADWIN can determine that the data distribution has changed when the conditions are met, and thus discard the data.W 1. And trigger model update.
[0104] It should be noted that, for power load forecasting applications, the CUSUM drift detection method in this embodiment is more suitable for real-time detection of sudden drifts, such as industrial load jumps, power outages, and sudden holiday load changes; the ADWIN adaptive window detection method is more suitable for detecting slow trend drifts, such as long-term load pattern changes caused by climate or economic cycle variations. In this embodiment, the choice can be flexible depending on the scenario: when the system has high requirements for response speed, the CUSUM drift detection method should be used first; when the system is more sensitive to robustness and long-term trend detection, the ADWIN adaptive window detection method should be used first.
[0105] Furthermore, such as Figure 4 As shown, when drift is detected using the above method, the reinforcement learning model's prediction process is paused, and a state drift signal is sent to the Markov decision model. Subsequently, the Markov decision model will retrain or fine-tune the model weights based on the degree of drift, while simultaneously updating the model input parameters and the candidate model library. This mechanism maintains the stability of the model's prediction performance even when external conditions change.
[0106] like Figure 5 As shown, this application also discloses a multi-model optimization load forecasting device based on reinforcement learning, the device comprising:
[0107] The state parameter acquisition module is used for data acquisition and feature construction to obtain model input parameters;
[0108] The candidate model prediction module is used to input the model input parameters into the candidate model library and obtain the predicted values and uncertainty measures of different prediction models in the candidate model library respectively.
[0109] The reinforcement learning decision module is used to select a prediction model from the candidate model library based on the Markov decision model.
[0110] The prediction execution module is used to make predictions using the selected prediction model and obtain the corresponding predicted values.
[0111] The error assessment module is used to compare the predicted value and the actual value to obtain the error assessment result.
[0112] The reward generation module is used to feed back the error evaluation result to the reward function to generate a reward signal;
[0113] The strategy optimization module is used to optimize the strategy of the Markov decision model based on the reward signal.
[0114] The apparatus provided in this application embodiment can achieve... Figure 1To avoid repetition, the various processes implemented in the method embodiments will not be described again here.
[0115] like Figure 6 As shown in the illustration, this application also provides an electronic device, including a processor and a memory, and a program or instructions stored in the memory and executable on the processor, which, when executed by the processor, implement as follows: Figure 1 The various processes of the method embodiments shown are all capable of achieving the same technical effect, and will not be described again here to avoid repetition.
[0116] This application embodiment also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the above-described functionality. Figure 1 The various processes described in the embodiments of the method described herein can achieve the same technical effect, and will not be repeated here to avoid repetition.
[0117] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes described in the embodiments of the method described herein can achieve the same technical effect, and will not be repeated here to avoid repetition.
[0118] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.
[0119] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0120] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another device, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0121] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.
[0122] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0123] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.
[0124] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a device (which may be a terminal or platform, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.
[0125] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A multi-model optimization load forecasting method based on reinforcement learning, characterized in that, The method includes the following steps: Data collection and feature construction are performed to obtain model input parameters; Input the model input parameters into the candidate model library, and obtain the predicted values and uncertainty measures of different prediction models in the candidate model library respectively; Model selection is performed on the prediction models in the candidate model library based on Markov decision models; The selected prediction model is used to make predictions and obtain the corresponding predicted values; The error assessment results are obtained by comparing the predicted values and the actual values. The error evaluation result is fed back to the reward function to generate a reward signal; the reward function is constructed by considering prediction accuracy, stability, risk control, uncertainty penalty, and model switching penalty, and adopts a loss-based reward mechanism. ; in For reward parameters; The mean absolute percentage error of the model's predicted values; Error term Variance within the time window; Value at risk (VaR) for the model conditions; To predict the interval width; Let be the model weight vector at time t; Let be the model weight vector at time t-1; The Manhattan distance between the weight vectors of adjacent time steps; These are the weights of each part; The Markov decision model is optimized based on the reward signal.
2. The multi-model optimal load forecasting method according to claim 1, characterized in that, The steps of data acquisition and feature construction to obtain model input parameters include: Different types of raw data and model characteristic parameters are obtained respectively; the data types of the raw data include load dimension data, environmental dimension parameters and period dimension parameters, and the data types of the model characteristic parameters include historical error data and prediction uncertainty parameters of different prediction models. The original data is preprocessed according to its data type to obtain preprocessed data; the preprocessing includes time alignment, missing value imputation, and anomaly correction. Preprocessed data is processed using a sliding time window method to obtain corresponding statistical characteristics; The parameter set is obtained by integrating preprocessed data, statistical features, and model characteristic parameters. The different data in the parameter set are standardized separately to obtain the model input parameters.
3. The multi-model optimization load forecasting method according to claim 1, characterized in that, The candidate model library includes multiple different types of prediction models, and the different types of prediction models run independently. The model input parameters are input into the corresponding prediction model, and the predicted value and uncertainty parameters of the corresponding model are output.
4. The multi-model optimization load forecasting method according to claim 1, characterized in that, When selecting a prediction model from the candidate model library based on a Markov decision model, the action parameter is set to discrete selection or weighted selection.
5. The multi-model optimization load forecasting method according to claim 4, characterized in that, The discrete selection uses the error evaluation result of a single prediction model as a reward to select the best model from the candidate model library; the weighted selection uses the error evaluation result of the weighted prediction value and the true value of multiple prediction models as a reward to output the weight allocation result of the best model.
6. The multi-model optimal load forecasting method according to claim 1, characterized in that, The method further includes the steps of: performing data distribution drift detection on the Markov decision model based on the error evaluation results; when a drift determination is triggered, pausing the reinforcement learning model prediction process and retraining or fine-tuning the weights, while updating the model input parameters and the candidate model library.
7. The multi-model optimization load forecasting method according to claim 1, characterized in that, When optimizing the policy of a Markov decision model based on reward signals, a proximal policy optimization algorithm or a deep Q-learning algorithm is used to achieve policy learning.
8. The multi-model optimal load forecasting method according to claim 1, characterized in that, When performing data distribution drift detection on Markov decision models, the CUSUM drift detection method or the ADWIN adaptive window detection method is used.
9. A multi-model optimization load forecasting device based on reinforcement learning, characterized in that, The device includes: The state parameter acquisition module is used for data acquisition and feature construction to obtain model input parameters; The candidate model prediction module is used to input the model input parameters into the candidate model library and obtain the predicted values and uncertainty measures of different prediction models in the candidate model library respectively. The reinforcement learning decision module is used to select a prediction model from the candidate model library based on the Markov decision model. The prediction execution module is used to make predictions using the selected prediction model and obtain the corresponding predicted values. The error assessment module is used to compare the predicted value and the actual value to obtain the error assessment result. The reward generation module is used to feed back the error evaluation result to the reward function to generate a reward signal. The reward function is constructed by considering prediction accuracy, stability, risk control, uncertainty penalty, and model switching penalty. The reward function adopts a loss-based reward. ; in For reward parameters; The mean absolute percentage error of the model's predicted values; Error term Variance within the time window; Value at risk (VaR) for the model conditions; To predict the interval width; Let be the model weight vector at time t; Let be the model weight vector at time t-1; The Manhattan distance between the weight vectors of adjacent time steps; These are the weights of each part; The strategy optimization module is used to optimize the strategy of the Markov decision model based on the reward signal.