A method and device for predicting load probability based on adaptive dynamic combination

Abstract

The present invention relates to an adaptive dynamic combined load probability forecasting method and device, belonging to the technical field of power system load forecasting, comprising: selecting a plurality of probability forecasting models as combined sub-models, optimizing and training each sub-model, improving the probability forecasting accuracy of each sub-model, and predicting the charge probability separately; performing Bayesian model averaging on each sub-model to obtain the overall forecasting result after combination; evaluating the forecasting result; constructing a deep deterministic policy gradient algorithm; constructing a short-term load forecasting model based on the deep deterministic policy gradient algorithm, and obtaining an adaptive dynamic forecasting result. The present invention overcomes the problem that a single forecasting model performs poorly under certain conditions, and realizes a dynamic mapping between the environment and the combined weight, online sensing of external environmental changes, and adaptive updating of the combined weight, thereby enhancing the adaptability of the model and effectively improving the stability and accuracy of the probability forecasting result.

Description

Technical Field

[0001] The present invention relates to a method and device for predicting load probability based on adaptive dynamic combination, belonging to the technical field of power system load prediction. Background Art

[0002] With the increase of uncertainty in the new power system, power load data shows significant non-stationarity. Complex environmental changes cause the distribution of sample data to change over time, resulting in data drift. Although the currently constructed deep learning prediction model is complex enough, the parameters are fixed once trained. The model with fixed parameters cannot flexibly adapt to the new data distribution, resulting in the model's error gradually increasing over time during the prediction process, and cannot accurately reflect changes in the complex environment.

[0003] At the same time, due to limitations in data accuracy and model limitations, deterministic predictions inevitably have prediction errors. Complex environmental changes lead to interdependent patterns in error distribution that are difficult to explore, and measurement of prediction uncertainty is challenging. Non-parametric probabilistic prediction methods based on deep learning have gradually become the mainstream probabilistic prediction methods. Although deep learning has significant advantages in feature mining, any single probability distribution model struggles to fully describe the coupling patterns between complex variables. Due to differences in the internal characteristics of the models, the advantages of different models under different conditions often differ. Therefore, integrating the prediction advantages of various models, dynamically combining models under different conditions, and updating relevant parameters online can avoid the limitations of a single prediction model while also overcoming the drawbacks of fixed-parameter models that make it difficult to track environmental changes online. Summary of the Invention

[0004] In response to the above problems, the present invention provides an adaptive dynamic combination load probability forecasting method and device. First, based on the Bayesian averaging principle, the advantages of probability forecasting models with different distribution types and different estimation principles such as quantile regression, sparse Bayesian learning, and random forest regression are integrated to establish a combined forecasting model of multiple types of probability distributions to overcome the problem that a single forecasting model performs poorly under certain conditions; then, a deep reinforcement learning framework based on deep deterministic policy gradient (DDPG) is constructed, and a feedback mechanism of environment-probability forecasting evaluation indicators is established to realize dynamic mapping between environment and combination weights, online perception of external environmental changes, and adaptive update of combination weights, thereby enhancing the adaptability of the model and effectively improving the stability and accuracy of probability forecasting results.

[0005] The technical solutions of the present invention are as follows:

[0006] An adaptive dynamic combined load probability forecasting method includes the following steps:

[0007] (1) Select multiple probability prediction models as combined sub-models, optimize and train each sub-model, improve the probability prediction accuracy of each sub-model, and use them to predict the charge probability separately;

[0008] (2) Perform Bayesian model averaging on each sub-model to obtain the combined overall prediction result;

[0009] (3) Evaluate the prediction results of step (2);

[0010] (4) Construct a feedback mechanism based on the Deep Deterministic Policy Gradient (DDPG) algorithm to establish an environment-probability prediction evaluation index, realize the dynamic and instant update of the combined weight of the load combination probability, and realize the dynamic mapping between the environment and the combined weight;

[0011] (5) Construct a short-term load forecasting model based on a deep deterministic policy gradient algorithm to obtain adaptive dynamic forecasting results.

[0012] Preferably, in step (1), a quantile regression model, a sparse Bayesian learning model, and a random forest regression model are selected as combined sub-models, wherein the quantile regression model approximates the conditional probability distribution of the random variable in the form of quantiles by more comprehensively describing the relationship between the target variable and the explanatory variable, and defines the random target variable y as the prediction object, F as the probability distribution function of the random target variable y with respect to the random explanatory vector x, and for the random target variable y at any time, its quantile Q τ is defined as:

[0013] Q τ =min{y|F(y)≥τ} (1)

[0014] where τ∈[0,1];

[0015] The quantile regression model is a linear model:

[0016] Q τ =α(τ) T ·x (2)

[0017] Among them, α(τ)=[α1,α2,…,α m ] T It's Q τ The regression coefficient vector under quantile conditions, m is the number of random variables;

[0018] Quantile Q τ It can be obtained by minimizing the error function, as shown in formula (3):

[0019] minE{ρτ (y,Q τ )} (3)

[0020] where ρ τ is the test function in the quantile regression algorithm. When it takes the minimum value, α(τ) is the optimal estimate;

[0021] For any τ∈(0,1):

[0022]

[0023] The optimal estimated value is obtained by calculation using formula (4), and the estimated value of the target variable under the quantile condition at any time is calculated. Then, the probability density function of the target variable is obtained by statistics using the Gaussian kernel function.

[0024] Preferably, the sparse Bayesian learning model first estimates parameters based on the posterior probability, and then introduces the sparsity prior. Let the target variable y and the explanatory variable x be a set of discrete time history series, and the corresponding relationship is expressed as

[0025] y i =f(x i ;w)+ε i (5)

[0026] Among them, w represents the weight vector; the error vector ε i The mean is 0 and the variance is σ 2 The normal distribution is expressed as N(0,σ 2 ); the target variable y of group i i The mean is f(x i ;ω), variance is σ 2 The distribution is normal, based on the kernel method, the basis function K(x,x i ) is applied to the prediction task, then

[0027]

[0028] Among them, φ(x) is the kernel function K(x i ,x) constructed vector; N is the number of kernel functions; w0 is the bias;

[0029] The probability of the target variable y is expressed as

[0030]

[0031] where Φ is expressed as [φ(x1),φ(x2),…,φ(x N )] T ,φ(x i ) is [1,K(x i ,x1),…,K(x i,x N )] T , N is the number of samples;

[0032] Will w i Assume that they have mean 0 and variance Normally distributed random variables, parameter σ 2 Obeying the gamma distribution with shape parameter and scale parameter both being 0, based on the Bayesian formula we get:

[0033]

[0034] p(w,α,σ 2 |y) represents the parameters w, α, σ 2 The posterior distribution of p(α,σ 2 |y) represents the parameters α, σ 2 The posterior distribution of p(w,|y,α,σ 2 ) indicates that the parameter w is in the hyperparameters α, σ 2 , the posterior probability distribution under the condition y;

[0035] At this point, the original problem can be transformed into solving the parameters α, σ 2 The posterior distribution of the parameter w is then solved under the hyperparameters α and σ 2 , posterior probability distribution under the condition of y; assuming α, σ 2 The posterior distribution of follows the Dirac distribution where α M 、 are the parameters α and σ respectively. 2 The maximum likelihood estimate of , so based on the Bayesian principle, the mean and variance of the weight w are

[0036]

[0037] in, N H Indicates the dimension of the parameter;

[0038] For the new explanatory variable x * , target variable y * Subject to the mean μ * =μ T φ(x * ), the variance is The sparse Bayesian learning model is used to characterize the target variable y, and the obtained probability density function f(y) is expressed as:

[0039]

[0040] Preferably, in the random forest regression model, the random forest regression tree prediction model of the target variable Y and the explanatory variable X is defined as {h(X,ξ k ),k=1,2,…,K}, use decision tree to convert numerical variables into predictive factors, so as to build random variable model; k ) is used to obtain the final predicted value using the simple averaging method, specifically:

[0041] Randomly extract some data from the sample set as the training set, and the mean square generalization error of the remaining predictor variables h(X) is E X,Y (Yh(X)), when the number of trees in the random forest is infinite, the mean squared generalization error will tend to stabilize and will not continue to decrease significantly. At this time, the mean squared generalization error is specifically:

[0042] E X,Y (Y-av k h(X,ξ k )) 2 →E X,Y (YE ξ h(X,ξ)) 2 (12)

[0043] Among them, av k h(X,ξ k ) is the average value of the decision tree output; in the actual regression model calculation, the K value is large enough, and the calculation formula of the random forest regression model is E ξ h(X,ξ), using Y=av k h(X,ξ k ) approximates the regression function and conducts a specific analysis of the tree error, then:

[0044] PE=E ξ E X,Y (YE ξ h(X,ξ)) 2 (13)

[0045] Where PE is the average generalization error value predicted by the random forest regression model;

[0046] By calculating the average generalization error of each tree, the prediction performance of a single regression tree is deeply evaluated. Finally, the expectation and variance obtained based on the random forest regression are statistically analyzed using a Gaussian distribution to obtain the predicted probability density function.

[0047] Preferably, in step (2), the Bayesian model average is performed by taking into account the posterior probability of each model and performing a weighted summation based on the performance of each model under the given data to obtain the overall prediction result;

[0048] Define the feature variable X corresponding to the target variable Y, then the average expression of the Bayesian model is

[0049]

[0050] Among them, M i is the i-th sub-model, i=1,2,3,…,K,p(Y|M i ,X) is composed of M i The probability of the target variable Y predicted by the model alone, p(M i |X) is model M i The posterior model probability is

[0051]

[0052] in

[0053] p(X|M i )=∫p(X|θ i ,M i )p(θ i |M i )dθ i (16)

[0054] θ i For model M i Parameters; p(M i ) is the model M i The prior probability distribution of p(X|θ i ,M i ) is the variable X with parameter θ i Model M i The posterior probability distribution under i |M i ) is the model M i The posterior probability distribution of ;

[0055] It can be seen that by weighted averaging the posterior distribution of variable Y obtained under each sub-model, the posterior probability distribution of the combined model of the target variable Y can be obtained. Therefore, the Bayesian model can be expressed as

[0056]

[0057] Among them, w i is the contribution rate of each sub-model.

[0058] Preferably, in step (3), the prediction results are evaluated by reliability, sensitivity and comprehensive evaluation indicators. Assuming that the data set contains n sample data, the confidence interval obtained by prediction at the confidence level of 1-α is [L1 α ,L2 α], the reliability of the prediction model is evaluated by the prediction interval coverage probability PICP (Prediction Interval Coverage Probability). The calculation method of PICP is as follows:

[0059]

[0060] Among them, if the observation value P of sample i i * Falls within the prediction interval Inside, then function 1(P i * ∈[L 1i α ,L 2i α ]) takes the value of 1, otherwise it takes 0; RACE is the reliability deviation, the smaller the reliability deviation, the higher the reliability;

[0061] Sensitivity reflects the ability of the model to obtain accurate prediction information. This indicator quantifies the imprecision of the probability prediction interval and is evaluated by the prediction interval bandwidth PINAW (Prediction Interval Normalized Average Width) indicator. PINAW is expressed as

[0062]

[0063] Where n is the number of samples, and They represent the lower and upper bounds of the prediction confidence interval respectively; the smaller the value of PINAW, the higher the sensitivity of the model;

[0064] The comprehensive evaluation index is the continuous ranked probability score CRPS (Continuous Ranked Probability Score). The smaller the CRPS value, the better the probability prediction performance of the model. The CRPS expression is as follows:

[0065]

[0066] Among them, n is the number of test samples, F i (y) is the cumulative distribution function prediction result of the i-th sample, P * i and P i are the observed and predicted values ​​of the load of the i-th sample respectively. When the predicted value is greater than or equal to the observed value, the indicative function 1(P i * ≤P i ) takes 1, otherwise takes 0.

[0067] Preferably, in step (4), the network structure of the deep deterministic policy gradient algorithm includes an Actor online network, an Actor target network, a Critic online network, a Critic target network, and an experience replay pool. The Actor online network and the Actor target network have the same structure, and the Critic online network and the Critic target network also have the same structure. The Actor network is responsible for receiving states and generating actions, while the Critic network evaluates the Q value of state-action pairs. The experience replay pool is used to store experience tuples generated by the interaction between the agent and the environment to facilitate offline learning and improve data efficiency.

[0068] Using the experience replay mechanism, samples obtained from the interaction between the agent and the environment are stored in the experience replay pool and used for neural network training in a random sampling manner;

[0069] The target network is introduced into deep reinforcement learning, and the target network parameters are updated in a soft way. In each iteration, they are updated according to formulas (21) and (22), that is,

[0070]

[0071] Among them, θ t is the parameter of the Actor online network at the current moment, θ′ t is the parameter of the Actor target network at the current moment, ω t is the parameter of the critic online network at the current moment, ω′ t is the parameter of the critic target network at the current moment, τ is called the soft update coefficient, and its value is much less than 1;

[0072] During the learning process, based on N samples (s j ,a j ,r j ,s j+1 ),j=1,…,N, the critic online network is updated first, and its loss function is expressed as follows:

[0073]

[0074] Among them, s j is the state learned in step j; a j is the action learned in step j; r j is the reward for learning at step j; Q(s j ,a j |w) is the reward value function of the jth step;

[0075] y j =r j +γQ′(s j+1,π′(s j+1 |θ′)|ω′) (24)

[0076] γ is the discount factor; Q'(s j+1 ,π'(s j+1 |θ')|w) is the value function under the j+1th step;

[0077] Then, the critic online network parameter ω is updated as shown below:

[0078]

[0079] Among them, α ω is the learning rate of the Critic online network, is the gradient of the loss function with respect to the network parameter ω (where ω t is the parameter of the critic online network at time t, where sampling ω is a general term);

[0080] Then, the Actor online network is updated according to the chain rule shown in Equation (26), which is about the network parameters θ (where θ t The loss gradient of the parameters of the Actor online network at time t (sampling θ here is a general term) is as follows

[0081]

[0082] θ is updated as follows:

[0083]

[0084] Among them, α θ Indicates the learning rate of the Actor online network.

[0085] Preferably, in step (5), the environment is defined as a combination mechanism of each sub-prediction model, and the function of the agent is to perceive the state from the external environment and output the combination weight of the combination model to the environment, and obtain corresponding rewards from the environment;

[0086] In load forecasting applications, the information needed for combined forecasting includes numerical weather forecast meteorological elements, measured loads, and predicted values ​​of sub-models. Therefore, the state set is defined as

[0087]

[0088] Among them, S1 is the forecast meteorological element, such as temperature, shortwave radiation, relative humidity, etc., and S2 is the measured load element. It is defined as the combined weights of the sub-models, including the quantile regression model, the sparse Bayesian learning model, and the random forest regression model, and the combined prediction error CRPS;

[0089] The action A of the agent is defined as the combined weight of the sub-models, that is

[0090] A={w 1 ,w 2 ,w 3} (29)

[0091] The reward r is set as a function of the probability evaluation index CRPS of the combined model and sub-model, that is,

[0092]

[0093] in e1, e2, and e3 are the probability evaluation indicators CRPS of the prediction results of the combined model and each sub-model respectively. This reward function requires that the probability prediction error of the combined model must be smaller than the minimum probability error of the three sub-models;

[0094] The Actor online network interacts with the environment to generate the weights of the combined model. Then the combined model obtains the load probability prediction result based on the weights and sub-model results. At the same time, as new data samples arrive, a reward is returned according to (30) to indicate the quality of the weights, and the upcoming meteorological data, load data, combined weights, and the combined model evaluation index CPRS are transferred to a new state. After this cycle, the new quadruple (S t ,A t ,r t ,S t+1 ) is pushed into the experience replay pool. If the experience replay pool is full, the oldest sample will be discarded;

[0095] In the initial state, the parameters of the quaternary are all default parameters, and the target network completely copies the parameters of the online network. First, the Actor online network accepts the current state s i The generated current policy π i (s i |θ) is provided to the Critic online network, and the Actor target network takes the next state s i+1 Generate strategy π i+1 (s i+1 |θ′) is provided to the Critic target network; the Critic target network is based on s i+1 and π i+1 (s i+1 |θ′) combined with formula (24) to obtain y i ; At the same time, the Critic online network is based on the current state s i and the current policy π i (s i |θ) calculate the current Q(s i ,ai |w), and then according to the target value y i and Q value, the Critic online network updates the parameter ω by minimizing the loss function of formula (23); then, the Actor online network updates the corresponding parameters and calculates the two gradients of the Acotr online network and Then, the two gradients are combined and the policy gradient is applied to update the parameter θ according to Equation (26); finally, the Actor target network and the Critic target network copy the parameters from the Actor online network and the Critic online network respectively in a soft update manner.

[0096] An adaptive dynamic combined load probability forecasting device includes a combined sub-model module, a Bayesian model averaging module, an evaluation module and a short-term load forecasting module based on a deep deterministic policy gradient algorithm, which is used to implement the above-mentioned adaptive dynamic combined load probability forecasting method.

[0097] Where the present invention is not exhaustive, please refer to the prior art.

[0098] The beneficial effects of the present invention are:

[0099] Based on the Bayesian average principle, the present invention integrates the advantages of probability prediction models with different distribution types and different estimation principles, such as quantile regression, sparse Bayesian learning, and random forest regression, to establish a combined prediction model of multiple types of probability distributions, overcoming the problem that a single prediction model performs poorly under certain conditions.

[0100] The present invention constructs a deep reinforcement learning framework based on deep deterministic policy gradient, establishes a feedback mechanism of environment-probability prediction evaluation indicators, realizes dynamic mapping between environment and combination weights, perceives changes in the external environment online, and adaptively updates the combination weights, thereby enhancing the adaptability of the model and effectively improving the stability and accuracy of probability prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] The drawings in the specification, which constitute a part of this application, are used to provide further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute improper limitations on this application.

[0102] Figure 1 This is the framework diagram of the combined probability prediction model;

[0103] Figure 2 Schematic diagram of the Markov decision process for reinforcement learning;

[0104] Figure 3 This is a schematic diagram of the network structure of the deep deterministic policy gradient algorithm;

[0105] Figure 4 Schematic diagram of load probability forecasting model based on deep deterministic gradient algorithm;

[0106] Figure 5 The probability density prediction results of the three methods QR, SBL and RF for a randomly selected day in January and February respectively;

[0107] Figure 6 The probability density functions of the models at four times: 0:00, 6:00, 12:00, and 18:00;

[0108] Figure 7 This is a diagram of the dynamic portfolio weights for a certain day from January to April;

[0109] Figure 8 The reliability evaluation results of the five models M1 to M5 at four confidence intervals of 80%, 85%, 90%, and 95%;

[0110] Figure 9 The sensitivity evaluation results of the five models M1 to M5 at four confidence intervals of 80%, 85%, 90%, and 95% are shown;

[0111] Figure 10 The monthly average CRPS index performance of the five models M1 to M5 from January to December. DETAILED DESCRIPTION

[0112] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0113] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0114] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0115] Example 1

[0116] An adaptive dynamic combined load probability forecasting method includes the following steps:

[0117] (1) Select multiple probability prediction models as combined sub-models, optimize and train each sub-model, improve the probability prediction accuracy of each sub-model, and use them to predict the charge probability separately;

[0118] (2) Perform Bayesian model averaging on each sub-model to obtain the combined overall prediction result;

[0119] (3) Evaluate the prediction results of step (2);

[0120] (4) Construct a feedback mechanism based on the Deep Deterministic Policy Gradient (DDPG) algorithm to establish an environment-probability prediction evaluation index, realize the dynamic and instant update of the combined weight of the load combination probability, and realize the dynamic mapping between the environment and the combined weight;

[0121] (5) Construct a short-term load forecasting model based on a deep deterministic policy gradient algorithm to obtain adaptive dynamic forecasting results.

[0122] Probabilistic forecasting models usually require certain presets. For example, in non-parametric probability forecasting models, transition parameters need to be preset, and parametric probability forecasting models need to make certain assumptions about data distribution functions. However, under normal circumstances, the distribution law and fluctuation characteristics of data samples are extremely uncertain. Therefore, a single probability forecasting model will inevitably be difficult to accurately and quantitatively describe the uncertainty of the forecast. It has been proven that the combined forecasting model can combine the advantages of each model and perform well in load forecasting. In the selection process of the combined forecasting sub-model, two goals need to be met. One is that the forecasting effect is as good as possible when the sub-model is used as an independent forecasting model. The other is that the difference between the sub-models is as large as possible. This ensures that the sub-models focus on different features and achieves the universality of the forecasting effect through combination. Therefore, this paper considers the advantages of parametric probability models and non-parametric probability models. Through optimal comparison, the quantile regression model, sparse Bayesian learning model and random forest regression model are selected as combined sub-models. At the same time, the Bayesian average principle is used to realize the free combination of various sub-models with different parametric and non-parametric distributions. The framework of the multi-type probability distribution combined probability forecasting method is as follows. Figure 1 shown.

[0123] Example 2

[0124] An adaptive dynamic combined load probability forecasting method is as described in Example 1, except that in step (1), a quantile regression model, a sparse Bayesian learning model, and a random forest regression model are selected as combined sub-models, wherein:

[0125] 1) Quantile Regression (QR) is an extension of the least squares method. In the least squares method, the error sum of squares function is defined as the loss function, while in quantile regression, the loss function is defined as the minimization error function based on the test function. The quantile regression algorithm is a non-parametric model that does not assume the probability distribution of random variables and can more sensitively reflect abnormal data in the sample. The quantile regression model approximates the conditional probability distribution of random variables in the form of quantiles by more comprehensively describing the relationship between the target variable and the explanatory variable. It defines the random target variable y as the prediction object, F as the probability distribution function of the random target variable y with respect to the random explanatory vector x, and for the random target variable y at any given moment, its quantile Q τ is defined as:

[0126] Q τ =min{y|F(y)≥τ} (1)

[0127] where τ∈[0,1];

[0128] The quantile regression model is a linear model:

[0129] Q τ =α(τ) T ·x (2)

[0130] Among them, α(τ)=[α1,α2,…,α m ] T It's Q τ The regression coefficient vector under quantile conditions, m is the number of random variables;

[0131] Quantile Q τ It can be obtained by minimizing the error function, as shown in formula (3):

[0132] minE{ρ τ (y,Q τ )} (3)

[0133] where ρ τ is the test function in the quantile regression algorithm. When it takes the minimum value, α(τ) is the optimal estimate;

[0134] For any τ∈(0,1):

[0135]

[0136] The optimal estimated value is obtained by calculation using formula (4), and the estimated value of the target variable under the quantile condition at any time is calculated. Then, the probability density function of the target variable is obtained by statistics using the Gaussian kernel function.

[0137] 2) The core idea of ​​sparse Bayesian Learning (SBL) is to constrain parameters by introducing sparsity priors, thereby achieving parameter sparsification and feature selection. In the Bayesian framework, the sparse Bayesian learning model first estimates parameters based on the posterior probability, usually using the maximum posterior distribution to estimate parameters. Then, a sparsity prior, such as the Laplace prior or the Lasso prior, is introduced to achieve parameter sparsification. Suppose the target variable y and the explanatory variable x are a set of discrete time history series, and the corresponding relationship is expressed as

[0138] y i =f(x i ;w)+ε i (5)

[0139] Among them, w represents the weight vector; the error vector ε i The mean is 0 and the variance is σ 2 The normal distribution is expressed as N(0,σ 2 ); the target variable y of group i i The mean is f(x i ;ω), variance is σ 2 The distribution is normal, based on the kernel method, the basis function K(x,x i ) is applied to the prediction task, then

[0140]

[0141] Among them, φ(x) is the kernel function K(x i ,x) constructed vector; N is the number of kernel functions; w0 is the bias;

[0142] Generally speaking, the kernel function K(x,x i ) is a highly nonlinear function, so sparse Bayesian learning has a certain nonlinear fitting ability. When the constructed sparse Bayesian learning model contains multiple kernel functions, it can handle nonlinear fitting problems more flexibly. Therefore, using sparse Bayesian learning to construct a prediction model has obvious advantages.

[0143] The probability of the target variable y is expressed as

[0144]

[0145] where Φ is expressed as [φ(x1),φ(x2),…,φ(x N )] T ,φ(x i ) is [1,K(x i ,x1),…,K(x i ,x N )]T , N is the number of samples;

[0146] For the parameter w i and σ 2 Generally, the maximum likelihood estimation is used to solve the problem. However, it is prone to overfitting, which leads to poor generalization ability of the model. In order to improve the generalization ability of the model and ensure the sparsity of the model, in the sparse Bayesian learning model, w i Assume that they have mean 0 and variance A normally distributed random variable with parameter σ 2 Obeying the gamma distribution with shape parameter and scale parameter both being 0, based on the Bayesian formula we get:

[0147] p(w,α,σ 2 |y)=p(w|y,α,σ 2 )p(α,σ 2 |y) (8)

[0148] p(w,α,σ 2 |y) represents the parameters w, α, σ 2 The posterior distribution of p(α,σ 2 |y) represents the parameters α, σ 2 The posterior distribution of p(w,|y,α,σ 2 ) indicates that the parameter w is in the hyperparameters α, σ 2 , the posterior probability distribution under the condition y;

[0149] At this point, the original problem can be transformed into solving the parameters α, σ 2 The posterior distribution of the parameter w is then solved under the hyperparameters α and σ 2 , posterior probability distribution under the condition of y; for the convenience of solution, assume that α, σ 2 The posterior distribution of follows the Dirac distribution where α M 、 are the parameters α and σ respectively. 2 The maximum likelihood estimate of , so based on the Bayesian principle, the mean and variance of the weight w are

[0150]

[0151] Among them, A=diag(α0,α1,…,α NH ), N H Indicates the dimension of the parameter;

[0152] For the new explanatory variable x * , target variable y * Subject to the mean μ * =μ T φ(x* ), the variance is The sparse Bayesian learning model is used to characterize the target variable y, and the obtained probability density function f(y) is expressed as:

[0153]

[0154] 3) Random Forest Regression (RF) is a machine learning method that builds an ensemble of decision trees. In the process of building individual decision trees, each node of the base decision tree is considered. Multiple decision trees are built, each trained on a different random subset, and the optimal attributes are selected to split the tree. Because the training data and features of each tree are randomly selected, there is a large difference between them, resulting in a smaller variance in the mean of the resulting decision tree.

[0155] In the random forest regression model, the random forest regression tree prediction model of the target variable Y and the explanatory variable X is defined as {h(X,ξ k ),k=1,2,…,K}, use decision tree to convert numerical variables into predictive factors, so as to build random variable model; k ) is used to obtain the final predicted value using the simple averaging method, specifically:

[0156] Randomly extract some data from the sample set as the training set, and the mean square generalization error of the remaining predictor variables h(X) is E X,Y (Yh(X)), when the number of trees in the random forest is infinite, the mean squared generalization error will tend to stabilize and will not continue to decrease significantly. At this time, the mean squared generalization error is specifically:

[0157] E X,Y (Y-av k h(X,ξ k )) 2 →E X,Y (YE ξ h(X,ξ)) 2 (12)

[0158] Among them, av k h(X,ξ k ) is the average value of the decision tree output; in the actual regression model calculation, the K value is large enough, and the calculation formula of the random forest regression model is E ξ h(X,ξ), using Y=av k h(X,ξ k ) approximates the regression function and conducts a specific analysis of the tree error, then:

[0159] PE=E ξ EX,Y (YE ξ h(X,ξ)) 2 (13)

[0160] Where PE is the average generalization error value predicted by the random forest regression model;

[0161] By calculating the average generalization error of each tree, the prediction performance of a single regression tree is deeply evaluated, which in turn reflects the prediction accuracy level of a single regression tree. The random forest model effectively reduces the overall generalization error and improves the robustness and prediction accuracy of the model by integrating the prediction results of multiple trees.

[0162] If for all ξ, E(Y)=E X h(X,ξ), then when building a random forest regression model, we can ensure that the prediction expectation of each decision tree is consistent with the prediction expectation of the overall model, thereby further improving the performance of the random forest model. Finally, the expectation and variance obtained from random forest regression are statistically analyzed using a Gaussian distribution to obtain the predicted probability density function.

[0163] Example 3

[0164] An adaptive dynamic combined load probability forecasting method, as described in Example 2, except that, in step (2), Bayesian model averaging (BMA) is a model fusion technology based on the Bayesian method, which is used to combine the prediction results of multiple models to obtain more accurate and robust predictions. Different models are regarded as different hypotheses or perspectives on the same problem, and each model has its own unique parameters and characteristics. Bayesian model averaging considers the posterior probability of each model and performs a weighted summation based on the performance of each model under given data to obtain an overall prediction result; Bayesian model averaging can effectively utilize the differences between different models, avoid overfitting and improve the accuracy of prediction.

[0165] Define the feature variable X corresponding to the target variable Y, then the average expression of the Bayesian model is

[0166]

[0167] Among them, M i is the i-th sub-model, i=1,2,3,…,K,p(Y|M i ,X) is composed of M i The probability of the target variable Y predicted by the model alone, p(M i |X) is model M i The posterior model probability is

[0168]

[0169] in

[0170] p(X|M i )=∫p(X|θ i ,M i )p(θ i |M i )dθ i (16)

[0171] θ i For model M i Parameters; p(M i ) is the model M i The prior probability distribution of p(X|θ i ,M i ) is the variable X with parameter θ i Model M i The posterior probability distribution under i |M i ) is the model M i The posterior probability distribution of ;

[0172] It can be seen that by weighted averaging the posterior distribution of variable Y obtained under each sub-model, the posterior probability distribution of the combined model of the target variable Y can be obtained. Therefore, the Bayesian model can be expressed as

[0173]

[0174] Among them, w i is the contribution rate of each sub-model.

[0175] Example 4

[0176] An adaptive dynamic combined load probability forecasting method, as described in Example 3, except that, in step (3), the forecast result is evaluated by reliability, sensitivity and comprehensive evaluation indicators;

[0177] 1) Reliability reflects the degree to which the model obtains effective prediction results. Assuming that the data set contains n sample data, the confidence interval obtained by prediction at the confidence level of 1-α is [L1 α ,L2 α ], the reliability of the prediction model is evaluated by the prediction interval coverage probability PICP. The calculation method of PICP is as follows:

[0178]

[0179] Among them, if the observation value P of sample i i* Falls within the prediction interval Inside, the function The value is 1, otherwise it is 0; RACE is the reliability deviation, the smaller the reliability deviation, the higher the reliability;

[0180] 2) Sensitivity reflects the model's ability to obtain accurate prediction information. This indicator quantifies the degree of imprecision of the probability prediction interval and is evaluated by the prediction interval bandwidth PINAW (Prediction Interval Normalized Average Width) indicator. PINAW is expressed as

[0181]

[0182] Where n is the number of samples, and Respectively represent the lower and upper bounds of the prediction confidence interval; if the prediction interval width is wider, it provides less effective prediction information. Conversely, a narrower prediction interval can provide more prediction information, so the smaller the value of PINAW, the higher the sensitivity of the model;

[0183] 3) The comprehensive evaluation index is the continuous ranked probability score CRPS (Continuous Ranked Probability Score). The smaller the CRPS value, the better the probability prediction performance of the model. The CRPS expression is as follows:

[0184]

[0185] Among them, n is the number of test samples, F i (y) is the cumulative distribution function prediction result of the i-th sample, P * i and P i are the observed and predicted values ​​of the load of the i-th sample respectively. When the predicted value is greater than or equal to the observed value, the indicative function 1(P i * ≤P i ) takes 1, otherwise takes 0.

[0186] Example 5

[0187] An adaptive dynamic combined load probability forecasting method, as described in Example 4, is different from the above-mentioned multi-type probability distribution combined probability forecasting method, that is, firstly optimizing and training each sub-model to improve the probability forecasting accuracy of each sub-model, and then performing Bayesian model averaging on each type of sub-model to obtain a combined probability forecasting method. Assume that the sample set is X n，n=1,2,3,...,N, then its combined probability prediction model is

[0188]

[0189] Among them, w n,1 ,w n,2 ,w n,3 are the contribution rates of sub-models M1, M2, and M3 in the nth sample case respectively.

[0190] Traditional methods mostly use statistical and learning algorithms such as maximum likelihood estimation and particle swarm optimization to optimize and train weight parameters. However, fixed parameters make it difficult to dynamically track the ever-changing external environment, resulting in poor generalizability of the prediction model. This paper combines current reinforcement learning methods to construct an environmental and probabilistic error feedback mechanism to dynamically and instantly update model parameters and improve the stability of prediction results.

[0191] Reinforcement learning is a method that instantly learns the relationship between environmental states and behaviors. Its core goal is to maximize the long-term cumulative reward of an agent in the environment. Unlike supervised and unsupervised learning, reinforcement learning relies primarily on immediate feedback from the environment regarding the agent's actions. This feedback is used to evaluate the effectiveness of the actions and guide future action choices based on these evaluations, thereby strengthening the action strategy. Reinforcement learning methods are based on the Markov decision process (MDP), a process in which the agent continuously interacts with the environment through trial and error over time. Therefore, applying reinforcement learning to load forecasting allows the model to instantly perceive environmental changes and make corresponding adjustments. This avoids the problem of fixed-parameter forecasting models that grow in error over time due to long periods of inactivity. This enhances the automation of the forecasting model and reduces manual maintenance and intervention.

[0192] The Markov decision process mainly includes two components: environment and agent, and three basic elements: state, action, and reward. The agent first perceives the state of the environment and selects the corresponding action to influence the environment. Then the environment receives the agent's action and returns reward information to the agent according to the pre-set reward mechanism to feedback the agent's behavior effect. Finally, the current state is converted to a new environment state using a certain state transition probability. This process constitutes a continuous interactive cycle between the agent and the environment. The entire decision-making process of MDP is shown in the figure below. Figure 2 .

[0193] Therefore, the learning process of reinforcement learning can be described as follows. In each interaction with the environment, the agent first obtains the current state of the environment s t, and then obtain the action a made during this interaction through the strategy model t And feedback to the environment, and then the environment according to the current state of the agent s t and action a t Infer the next state s of the agent t+1 , the agent obtains the reward function r(t) for the current state based on its interaction with the environment. Reinforcement learning attempts to find a better action strategy through continuous trial and error exploration to maximize the cumulative reward R(t), as shown in the following formula:

[0194]

[0195] Among them, r(t) is the reward function at time t, γ is the discount factor, and its value range is [0,1]. The discount factor reflects the degree of importance attached to rewards at future moments, that is, the agent pays more attention to the rewards of the current state and gradually ignores the rewards of future states.

[0196] To achieve the desired goal, reinforcement learning defines a value function to quantitatively evaluate the superiority of the strategy. According to the Bellman equation, the Q-value function is as follows:

[0197] Q(s,a)=E[R t |s=s t ,a=a t ]

[0198] =E(r(s,a)+γQ(s′,a′))

[0199] Where s′ and a′ are the subsequent state and action, a′=π(s′), and π(s′) is the strategy in state s′.

[0200] Traditional reinforcement learning algorithms, such as Q-learning, typically employ an iterative optimization strategy based on a two-dimensional state-action table, meaning that the states and actions are discrete. However, in load forecasting, the state and action spaces are typically high-dimensional and continuous, posing a challenge to traditional approaches. High-dimensional continuous state spaces result in a vast number of state combinations, making it difficult for traditional approaches to effectively store and update the values ​​corresponding to all possible states and actions. To address this issue, deep reinforcement learning has emerged. Deep reinforcement learning incorporates deep learning algorithm fitting to enable modeling and optimization of high-dimensional continuous state spaces. The powerful feature extraction and abstraction capabilities of neural networks facilitate processing high-dimensional data, enabling perception of raw data input and effective control of system outputs.

[0201] In deep reinforcement learning (DRL), neural networks are often used to approximate value functions or policy functions. By optimizing neural network parameters, a continuous modeling state space can be constructed and effective policies can be learned. The neural network's input is the environment state, and its output is the agent's action. By optimizing the neural network and updating the policy, algorithm efficiency and performance are improved.

[0202] The Deep Q-network (DQN) is a classic deep reinforcement learning (DRL) algorithm used to learn state-action value functions. It is an improvement on Q-Learning. In deep learning, samples are usually assumed to be independent and identically distributed, but in reinforcement learning, samples are obtained through the interaction between the agent and the environment, and there is correlation in the time series.

[0203] DQN uses a target network model to calculate Q values. The original network is an online network, and the target network has the same structure as the online network. The network parameters are copied from the online network, but with a certain delay, typically using soft updates or fixed-step updates. During training, the online network continuously updates its own parameters, while the target network's parameters are periodically copied from the online network, a delayed update. The introduction of the target network in DQN improves the stability and convergence of network training.

[0204] During training, the online network interacts with the environment. The target network is used to calculate the target value, while the online network calculates the estimated value. By comparing these two values, the model parameters can be updated. The update formula is as follows:

[0205]

[0206] The stochastic gradient descent method is used to update the network weights, which can be expressed as:

[0207]

[0208] Among them, Q(s,a,w) is the value function approximation; w is the parameter of neural network training; s is the initial state of the random environment in DQN; γ is the discount factor; α is the learning rate. The larger the learning rate, the less the effect of previous training is retained.

[0209] Using the above formula, we can use the gradient descent method to update the network parameters. After completing a certain number of iterations, the parameters of the online network are passed to the target network, which uses a soft update method to reduce model fluctuations. The specific implementation method is shown in formula (22). The DQN algorithm can generate a Q-value function based on the neural network to process continuous states. This requires enumerating countless actions to determine the maximum Q-value. Therefore, DQN has difficulty in handling continuous action problems, and this method has certain limitations.

[0210] Taking into account the limitations of DRL based on value functions, researchers changed their thinking and proposed policy-based DRL, which mainly includes the Deep Deterministic Policy Gradient (DDPG) algorithm, the Proximal Policy Optimization (PPO) algorithm, etc. The present invention adopts the Deep Deterministic Policy Gradient algorithm to realize the dynamic and instant update of the combined weights of the load combination probability. DDPG is a reinforcement learning algorithm based on the Actor-Critic architecture, which is specifically used to solve reinforcement learning problems in continuous action spaces. In the DDPG algorithm framework, the Actor network is a deterministic policy network responsible for outputting deterministic actions and updating policy parameters through gradient ascent. The Critic network estimates the action value function and updates the value function parameters by minimizing the error. Both are fitted using deep learning networks. Therefore, the core idea of ​​DDPG is to simultaneously learn the value function and the deterministic policy, guide policy learning by maximizing the value function, and thus achieve accurate exploration and optimization of the continuous action space.

[0211] In step (4), the network structure of the deep deterministic policy gradient algorithm (DDPG) includes the Actor online network, the Actor target network, the Critic online network, the Critic target network and the experience replay pool, such as Figure 3 As shown in the figure, the actor online network and the actor target network have the same structure. The critic online network and the critic target network also have the same structure. The actor network is responsible for receiving states and generating actions, while the critic network evaluates the Q value of state-action pairs. The parameters of the target network are updated at a certain rate during training to improve stability and convergence. The experience replay pool is used to store experience tuples generated by the interaction between the agent and the environment, facilitating offline learning and improving data efficiency.

[0212] 1) Experience Replay Pool

[0213] Using the experience replay mechanism, samples generated by the agent's interactions with the environment are stored in an experience replay pool and randomly sampled for neural network training. Specifically, interaction samples are stored in the experience replay pool, and a certain number of these samples are randomly selected during network training. Typically, the experience replay pool is set large enough to store a long period of experience samples. When the pool exceeds its capacity, new experience overwrites the oldest experience. In reinforcement learning, experience replay typically uniformly randomly samples from the pool to break down correlations between samples and reduce fluctuations within a single sequence. Furthermore, the same sample can be used multiple times, improving sample utilization. The key benefit of experience replay lies in its effective destruction of correlations between samples, reducing the interference of correlations on training, and significantly improving training stability. Through experience replay, the model can more fully learn the dynamic characteristics of the environment, avoid being trapped in local optima, and improve the agent's decision-making performance in complex environments.

[0214] 2) Actor and Critic target network parameter update

[0215] In deep reinforcement learning, when a nonlinear function is used to approximate the Q value, its update is prone to oscillation. To ensure the stability of learning, a target network is introduced into deep reinforcement learning. The target network parameters are updated in a soft update manner. In each iteration, they are updated according to formulas (21) and (22), that is,

[0216]

[0217]

[0218] Among them, θ t is the parameter of the Actor online network at the current moment, θ′ t is the parameter of the Actor target network at the current moment, ω t is the parameter of the critic online network at the current moment, ω′ t is the parameter of the critic target network at the current moment, τ is called the soft update coefficient, and its value is much smaller than 1. Therefore, when the online network parameters change slightly, the target network parameters will be relatively stable. Only when the online network parameters are updated significantly will there be a significant update, thus avoiding the occurrence of Q value oscillation.

[0219] 3) Actor and Critic online network parameter update

[0220] DDPG introduces the experience replay pool to break the correlation of sample data, and adds noise to the output actions during the learning process to improve the generalization learning ability of the model. During the learning process, based on N samples (s randomly drawn from the experience replay pool), j ,aj ,r j ,s j+1 ),j=1,…,N, the critic online network is updated first, and its loss function is expressed as follows:

[0221]

[0222] Among them, s j is the state learned in step j; a j is the action learned in step j; r j is the reward for learning at step j; Q(s j ,a j |w) is the reward value function of the jth step;

[0223] y j =r j +γQ′(s j+1 ,π′(s j+1 |θ′)|ω′) (24)

[0224] γ is the discount factor; Q'(s j+1 ,π'(s j+1 |θ')|w) is the value function under the j+1th step;

[0225] Then, the critic online network parameter ω is updated as shown below:

[0226]

[0227] Among them, α ω is the learning rate of the Critic online network, is the gradient of the loss function with respect to the network parameter ω (where ω t is the parameter of the critic online network at time t, where sampling ω is a general term);

[0228] Then, the Actor online network is updated according to the chain rule shown in Equation (26), which is about the network parameters θ (where θ t The loss gradient of the parameters of the Actor online network at time t (sampling θ here is a general term) is as follows

[0229]

[0230] θ is updated as follows:

[0231]

[0232] Among them, α θ Indicates the learning rate of the Actor online network.

[0233] Example 6

[0234] An adaptive dynamic combined load probability forecasting method, as described in Example 5, except that, in step (5), the environment is defined as a combination mechanism of the sub-prediction models, and the function of the intelligent agent is to perceive the state from the external environment and output the combined weight of the combined model to the environment, and obtain corresponding rewards from the environment;

[0235] In load forecasting applications, the information needed for combined forecasting includes numerical weather forecast meteorological elements, measured loads, and predicted values ​​of sub-models. Therefore, the state set is defined as

[0236]

[0237] Among them, S1 is the forecast meteorological element, such as temperature, shortwave radiation, relative humidity, etc., and S2 is the measured load element. It is defined as the combined weights of the sub-models, including the quantile regression model, the sparse Bayesian learning model, and the random forest regression model, and the combined prediction error CRPS;

[0238] The action A of the agent is defined as the combined weight of the sub-models, that is

[0239] A={w 1 ,w 2 ,w 3} (29)

[0240] The reward r is set as a function of the probability evaluation index CRPS of the combined model and sub-model, that is,

[0241]

[0242] in e1, e2, and e3 are the probability evaluation indicators CRPS of the prediction results of the combined model and each sub-model respectively. This reward function requires that the probability prediction error of the combined model must be smaller than the minimum probability error of the three sub-models;

[0243] Load probability prediction model based on deep deterministic gradient algorithm Figure 4 As shown, in Figure 4In the above example, the upper part can be regarded as the external environment. The lower part is the Actor-Critic framework, which includes the Actor online network, the Actor target network, the Critic online network, and the Critic target network. The Actor online network interacts with the environment to generate the weights of the combined model. Then the combined model obtains the load probability prediction results based on the weights and sub-model results. At the same time, as new data samples arrive, a reward is returned according to (30) to indicate the quality of the weights, and the upcoming meteorological data, load data, combined weights, and the combined model evaluation index CPRS are transferred to the new state; after this cycle, the new quadruple (S t ,A t ,r t ,S t+1 ) is pushed into the experience replay pool. If the experience replay pool is full, the oldest sample will be discarded;

[0244] In the initial state, the parameters of the quaternary are all default parameters, and the target network completely copies the parameters of the online network. First, the Actor online network accepts the current state s i The generated current policy π i (s i |θ) is provided to the Critic online network, and the Actor target network takes the next state s i+1 Generate strategy π i+1 (s i+1 |θ′) is provided to the Critic target network; the Critic target network is based on s i+1 and π i+1 (s i+1 |θ′) combined with formula (24) to obtain y i ; At the same time, the Critic online network is based on the current state s i and the current policy π i (s i |θ) calculate the current Q(s i ,a i |w), and then according to the target value y i and Q value, the Critic online network updates the parameter ω by minimizing the loss function of formula (23); then, the Actor online network updates the corresponding parameters and calculates the two gradients of the Acotr online network and Then, the two gradients are combined and the policy gradient is applied to update the parameter θ according to Equation (26); finally, the Actor target network and the Critic target network copy the parameters from the Actor online network and the Critic online network respectively in a soft update manner.

[0245] Example 7

[0246] An adaptive dynamic combined load probability forecasting device includes a combined sub-model module, a Bayesian model averaging module, an evaluation module and a short-term load forecasting module based on a deep deterministic policy gradient algorithm, which is used to implement the above-mentioned adaptive dynamic combined load probability forecasting method.

[0247] Case analysis:

[0248] Data selection and model testing environment:

[0249] This paper conducts case analysis based on load data from a certain region in China. The dataset contains numerical weather forecast data and measured load data for the region every 15 minutes from 2019 to 2020. All data are divided into two sets: the relevant data from January 1, 2019 to December 31, 2019 is used for initial training of reinforcement learning, and the other datasets are used for testing and learning. This paper uses the Jupyter integrated development environment in Anaconda software and the Tensorflow deep learning framework to build a 24-hour lead-time short-term load forecasting model. The model runs in Windows 11, Intelli7-10875H@2.30GHz, NVIDIA RTX2060, and 32GB of RAM.

[0250] Verification of the validity of sub-model probability prediction results:

[0251] This paper first comprehensively evaluates the prediction results of the three proposed models, namely quantile regression (QR), sparse Bayesian learning (SBL), and random forest regression (RF), from January 1, 2020 to December 31, 2020. Figure 5The probability density forecast results for the QR, SBL, and RF methods for a randomly selected day in January and February are shown. The figure shows that the probability density functions of the three models cover the measured load well, and the measured load is mostly distributed near the forecast model mean. Therefore, all three models perform well. It can also be seen that the forecast performance of different forecast models varies at different times. Therefore, a combination method can be used to integrate the forecast advantages of each model to improve forecast accuracy. Table 1 summarizes the RMSE and MAE of the forecast mean for the selected day of the three models, as well as the CRPS (Criteria for Probabilistic Forecasting) evaluation index. The table shows that all three models perform well in the forecast. It can be clearly seen from Table 1 that the performance of the three models is inconsistent at different times. From the analysis of the CRPS index of the probability prediction results, the three models are ranked from best to worst in January as RF, SBL, and QR, and the three models are ranked from best to worst in February as RF, QR, and SBL. However, the three models are ranked from best to worst in December as SBL, RF, and QR. This proves that different models perform inconsistently in different scenarios. It is difficult to effectively measure the uncertainty of the prediction using only a single probability prediction model. Obviously, if multiple models are reasonably combined and the advantages of different models in different scenarios are explored, the prediction effect can be effectively improved.

[0252] Table 1 Statistical results of prediction indicators of the three models QR, SBL and RF on the selected day

[0253]

[0254] Verification of the effectiveness of adaptive dynamic prediction results:

[0255] To verify the superiority of the adaptive dynamic combination method proposed in this paper, we used sub-models QR (M1), SBL (M2), RF (M3), and a fixed combination prediction model (M4) to compare and verify with the DDPG combined probability prediction (M5) proposed in this paper. The sub-models used training sets to optimize parameters; the fixed combination model performed a weighted combination of the sub-model prediction results, and the combination weights were optimized using a particle swarm optimization algorithm. The combination weights of the adaptive dynamic prediction method proposed in this paper were obtained and updated in real time using DDPG deep reinforcement learning.

[0256] (1) Analysis of the key components of the proposed method

[0257] The present invention selects the prediction results of the probability density function (PDF) of models M1, M2, M3, M4, and M5 at four times: 0:00, 6:00, 12:00, and 18:00 on a certain day in August. Among them, M4 is obtained by weighted combination of the individual PDFs of the sub-models, and its combination weight is fixed. M5 is also obtained by weighted combination of the individual PDFs like M4, but its weight changes with the external environment. Figure 6 It can be seen that the true value is well covered by the PDF and is close to the mean position of the PDF of each model.

[0258] The combined weight of the model reflects the contribution of each sub-model to the prediction result. The combined weight of model M4 is constant. The present invention uses particle swarm optimization to optimize the QR, SBL, and RF weights to be (0.39, 0.55, 0.06). One day was randomly selected from January to April for analysis. Figure 7 The combined weights of 96 points on the selected day were calculated, and the combined weights changed over time.

[0259] (2) Verify the probability prediction results

[0260] The present invention uses the above reliability deviation index RACE to evaluate the reliability of the proposed model. Figure 8 Reliability evaluation results for the five models are presented at four confidence intervals of 80%, 85%, 90%, and 95%, with a statistical timeframe of one year. The reliability deviations for model M5 at the four confidence intervals were 1.04%, 1.15%, 1.05%, and 1.28%, respectively. The deviations for model M4 at the four confidence intervals were 1.82%, 2.02%, 2.01%, and 1.99%, respectively. This indicates that model M5 significantly outperforms model M4 in predictive reliability. Models M1, M2, and M3, however, had reliability deviations exceeding 2.3%, with some even exceeding 3%, indicating their predictive performance was significantly inferior to models M4 and M5. These results demonstrate that combined models provide more reliable probabilistic forecasts, with the adaptive dynamic combined model exhibiting superior reliability.

[0261] The present invention uses the interval bandwidth index PINAW proposed above to evaluate the sensitivity of the proposed model. The smaller the interval bandwidth, the higher the sensitivity and the better the model performance. Figure 9The sensitivity evaluation results of the five models are presented at four confidence intervals of 80%, 85%, 90%, and 95%, with a statistical time range of one year. The bandwidths of the four confidence intervals for model M5 are 2.31%, 2.20%, 2.13%, and 2.00%, respectively, while those for model M4 are 3.59%, 3.18%, 2.76%, and 2.51%, respectively. This indicates that model M5 significantly outperforms model M4 in predictive sensitivity. However, the sensitivity deviations of models M1, M2, and M3 exceed 3%, and some even exceed 4%, making their predictive performance far inferior to models M4 and M5. The results indicate that combined models can provide more accurate probabilistic forecasts, and the adaptive dynamic combined model exhibits superior sensitivity, with narrower interval widths and more information contained in its predictions.

[0262] Figure 10 Comparisons of the 12-month monthly average CRPS indicators for the five models are presented. The results show that the CRPS value of model M5 is lower than that of model M4, which in turn is significantly lower than that of the other three sub-models. The CRPS results demonstrate that the proposed combined probability model has a more accurate prediction distribution than individual models, and that the adaptive dynamic combined prediction model outperforms the fixed-parameter combined prediction model.

[0263] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An adaptive dynamic combined load probability forecasting method, characterized in that: The steps include: (1) Select multiple probability prediction models as combined sub-models, optimize and train each sub-model, improve the probability prediction accuracy of each sub-model, and use them to predict the charge probability separately; (2) Perform Bayesian model averaging on each sub-model to obtain the combined overall prediction result; (3) Evaluate the prediction results of step (2); (4) Construct an environment-probability prediction feedback mechanism based on a deep deterministic policy gradient algorithm to achieve dynamic and instant updates of the combined weights of load combination probabilities; (5) Construct a short-term load forecasting model based on a deep deterministic policy gradient algorithm to obtain adaptive dynamic forecasting results; In step (1), the quantile regression model, the sparse Bayesian learning model and the random forest regression model are selected as the combined sub-models.

2. The adaptive dynamic combined load probability forecasting method according to claim 1, characterized in that: The quantile regression model describes the relationship between the target variable and the explanatory variable, approximating the conditional probability distribution of the random variable in the form of quantiles. The random target variable y is defined as the prediction object, F is the probability distribution function of the random target variable y with respect to the random explanatory vector x, and for any random target variable y at any time, its quantile Q τ is defined as: Q τ =min{y|F(y)≥τ} (1) where τ∈[0,1]; The quantile regression model is a linear model: Q τ =a(t) T ·x (2) Among them, α(τ)=[α1,α2,L,α m ] T It's Q τ The regression coefficient vector under quantile conditions, m is the number of random variables; Quantile Q τ It is obtained by minimizing the error function, as shown in formula (3): minE{ρ τ (y,Q τ )}(3) where ρ τ is the test function in the quantile regression algorithm. When it takes the minimum value, α(τ) is the optimal estimate; For any τ∈(0,1): The optimal estimated value is obtained by calculation using formula (4), and the estimated value of the target variable under the quantile condition at any time is calculated. Then, the probability density function of the target variable is obtained by statistics using the Gaussian kernel function.

3. The adaptive dynamic combined load probability forecasting method according to claim 2, characterized in that: The sparse Bayesian learning model first estimates parameters based on the posterior probability, and then introduces the sparsity prior. The target variable y and the explanatory variable x are set as a set of discrete time history series. The corresponding relationship is expressed as y i =f(x i ;w)+ε i (5) Among them, w represents the weight vector; the error vector ε i The mean is 0 and the variance is σ 2 The normal distribution is expressed as N(0,σ 2 ); the target variable y of group i i The mean is f(x i ;ω), variance is σ 2 The distribution is normal, based on the kernel method, the basis function K(x,x i ) is applied to the prediction task, then in, ϕ (x) is the kernel function K(x i ,x) constructed vector; N is the number of kernel functions; w0 is the bias; The probability of the target variable y is expressed as where Φ is expressed as [φ(x1),φ(x2),…,φ(x N )] T , ϕ (x i ) is [1,K(x i ,x1),…,K(x i ,x N )] T , N is the number of samples; Will w i Assume that they have mean 0 and variance A normally distributed random variable with parameter σ 2 Obeying the gamma distribution with shape parameter and scale parameter both being 0, based on the Bayesian formula we get: p(w,a,s 2 |y)=p(w|y,α,σ 2 )p(a,s 2 |y) (8) p(w,α,σ 2 |y) represents the parameters w, α, σ 2 The posterior distribution of p(α,σ 2 |y) represents the parameters α, σ 2 The posterior distribution of p(w,|y,α,σ 2 ) indicates that the parameter w is in the hyperparameters α, σ 2 , the posterior probability distribution under the condition y; At this point, the original problem can be transformed into solving the parameters α, σ 2 The posterior distribution of the parameter w is then solved under the hyperparameters α and σ 2 , posterior probability distribution under the condition of y; assuming α, σ 2 The posterior distribution of follows the Dirac distribution where α M 、 are the parameters α and σ respectively. 2 The maximum likelihood estimate of , so based on the Bayesian principle, the mean and variance of the weight w are in, N H Indicates the dimension of the parameter; For the new explanatory variable x * , target variable y * Subject to the mean μ * =μ T φ(x * ), the variance is The sparse Bayesian learning model is used to characterize the target variable y, and the obtained probability density function f(y) is expressed as:

4. The adaptive dynamic combined load probability forecasting method according to claim 3, characterized in that: In the random forest regression model, the random forest regression tree prediction model of the target variable Y and the explanatory variable X is defined as {h(X,ξ k ),k=1,2,…,K}, use decision tree to convert numerical variables into predictive factors, so as to build random variable model; k ) is used to obtain the final predicted value using the simple averaging method, specifically: Randomly extract some data from the sample set as the training set, and the mean square generalization error of the remaining predictor variables h(X) is E X,Y (Yh(X)), when the number of trees in the random forest is infinite, the mean squared generalization error will tend to stabilize, and the mean squared generalization error is specifically: E X,Y (Y-off k h(X,ξ k )) 2 →E X,Y (YE ξ h(X,ξ)) 2 (12) Among them, av k h(X,ξ k ) is the average value of the decision tree output; in the actual regression model calculation, the K value is large enough, and the calculation formula of the random forest regression model is E ξ h(X,ξ), using Y=av k h(X,ξ k ) approximates the regression function and conducts a specific analysis of the tree error, then: PE=E ξ BY X,Y (YE ξ h(X,ξ)) 2 (13) Where PE is the average generalization error value predicted by the random forest regression model; By calculating the average generalization error of each tree, the prediction performance of a single regression tree is deeply evaluated. Finally, the expectation and variance obtained based on the random forest regression are statistically analyzed using a Gaussian distribution to obtain the predicted probability density function.

5. The adaptive dynamic combined load probability forecasting method according to claim 1, characterized in that: In step (2), the Bayesian model average considers the posterior probability of each model and performs a weighted sum based on the performance of each model under the given data to obtain the overall prediction result; Define the feature variable X corresponding to the target variable Y, then the average expression of the Bayesian model is Among them, M i is the i-th sub-model, i=1,2,3,…,K,p(Y|M i ,X) is composed of M i The probability of the target variable Y predicted by the model alone, p(M i |X) is model M i The posterior model probability is in p(X|M i )=∫p(X|θ i ,M i )p(θ i |M i )dθ i (16) θ i For model M i Parameters; p(M i ) is the model M i The prior probability distribution of p(X|θ i ,M i ) is the variable X with parameter θ i Model M i The posterior probability distribution under i |M i ) is the model M i The posterior probability distribution of ; The posterior distribution of variable Y obtained under each sub-model is weighted averaged to obtain the posterior probability distribution of the combined model of the target variable Y. Thus, the Bayesian model can be expressed as Among them, w i is the contribution rate of each sub-model.

6. The adaptive dynamic combined load probability forecasting method according to claim 5, characterized in that: In step (3), the prediction results are evaluated by reliability, sensitivity and comprehensive evaluation indicators. Assuming that the data set contains n sample data, the confidence interval obtained by prediction at the confidence level of 1-α is [L1 α ,L2 α ], the reliability of the prediction model is evaluated by the prediction interval coverage ratio PICP, and the calculation method of PICP is as follows: Among them, if the observation value P of sample i i * Falls within the prediction interval Inside, the function The value is 1, otherwise it is 0; RACE is the reliability deviation, the smaller the reliability deviation, the higher the reliability; Sensitivity reflects the ability of the model to obtain accurate prediction information. Sensitivity quantifies the imprecision of the probability prediction interval and is evaluated by the prediction interval bandwidth PINAW indicator. PINAW is expressed as Where n is the number of samples, and They represent the lower and upper bounds of the prediction confidence interval respectively; the smaller the value of PINAW, the higher the sensitivity of the model; The comprehensive evaluation index is the continuous ranking probability score CRPS. The smaller the CRPS value, the better the probability prediction performance of the model. The CRPS expression is as follows: Among them, n is the number of test samples, F i (y) is the cumulative distribution function prediction result of the i-th sample, P * i and P i are the observed and predicted values ​​of the load of the i-th sample respectively. When the predicted value is greater than or equal to the observed value, the indicative function 1(P i * ≤P i ) takes 1, otherwise takes 0.

7. The adaptive dynamic combined load probability forecasting method according to claim 6, characterized in that: In step (4), the network structure of the deep deterministic policy gradient algorithm includes an Actor online network, an Actor target network, a Critic online network, a Critic target network, and an experience replay pool. The Actor online network and the Actor target network have the same structure, and the Critic online network and the Critic target network also have the same structure. The Actor network is responsible for receiving states and generating actions, while the Critic network evaluates the Q value of the state-action pair. The experience replay pool is used to store experience tuples generated by the interaction between the agent and the environment to facilitate offline learning and improve data efficiency. Using the experience replay mechanism, samples obtained from the interaction between the agent and the environment are stored in the experience replay pool and used for neural network training in a random sampling manner; The target network is introduced into deep reinforcement learning, and the target network parameters are updated in a soft way. In each iteration, they are updated according to formulas (21) and (22), that is, Among them, θ t is the parameter of the Actor online network at the current moment, θ t ′ is the parameter of the Actor target network at the current moment, ω t is the parameter of the critic online network at the current moment, ω′ t is the parameter of the critic target network at the current moment, τ is called the soft update coefficient, and its value is much less than 1; During the learning process, based on N samples (s j ,a j ,r j ,s j+1 ),j=1,…,N, the critic online network is updated first, and its loss function is expressed as follows: Among them, s j is the state learned in step j; a j is the action learned in step j; r j is the reward for learning at step j; Q(s j ,a j |w) is the reward value function of the jth step; y j =r j +γQ′(s j+1 ,π′(s j+1 |θ′)|ω′) (24) γ is the discount factor; Q'(s j+1 ,π'(s j+1 |θ')|w) is the value function under the j+1th step; Then, the critic online network parameter ω is updated as shown below: Among them, α ω is the learning rate of the Critic online network, is the gradient of the loss function with respect to the network parameter ω; Next, the Actor online network is updated according to the chain rule shown in Equation (26), and its loss gradient with respect to the network parameter θ is as follows θ is updated as follows: Among them, α θ Indicates the learning rate of the Actor online network.

8. The adaptive dynamic combined load probability forecasting method according to claim 7, characterized in that: In step (5), the environment is defined as the combination mechanism of each sub-prediction model. The function of the agent is to perceive the state from the external environment and output the combined weight of the combined model to the environment, and obtain corresponding rewards from the environment. In load forecasting applications, the information needed for combined forecasting includes numerical weather forecast meteorological elements, measured loads, and predicted values ​​of sub-models. Therefore, the state set is defined as Among them, S1 is the forecast meteorological element, S2 is the measured load element, It is defined as the combined weights of the sub-models, including the quantile regression model, the sparse Bayesian learning model, and the random forest regression model, and the combined prediction error CRPS; The action A of the agent is defined as the combined weight of the sub-models, that is A={w 1 ,w 2 ,w 3 } (29) The reward r is set as a function of the probability evaluation index CRPS of the combined model and sub-model, that is, in e1, e2, and e3 are the probability evaluation indexes CRPS of the prediction results of the combined model and each sub-model respectively; The Actor online network interacts with the environment to generate the weights of the combined model. Then the combined model obtains the load probability prediction result based on the weights and sub-model results. At the same time, as new data samples arrive, a reward is returned according to (30) to indicate the quality of the weights, and the upcoming meteorological data, load data, combined weights, and the combined model evaluation index CPRS are transferred to a new state. After this cycle, the new quadruple (S t ,A t ,r t ,S t+1 ) is pushed into the experience replay pool. If the experience replay pool is full, the oldest sample will be discarded; First, the Actor online network receives the current state s i The generated current policy π i (s i |θ) is provided to the Critic online network, and the Actor target network takes the next state s i+1 Generate strategy π i+1 (s i+1 |θ′) is provided to the Critic target network; the Critic target network is based on s i+1 and π i+1 (s i+1 |θ′) combined with formula (24) to obtain y i ; At the same time, the Critic online network is based on the current state s i and the current policy π i (s i |θ) calculate the current Q(s i ,a i |w), and then according to the target value y i and Q value, the Critic online network updates the parameter ω by minimizing the loss function of formula (23); then, the Actor online network updates the parameters and calculates the two gradients of the Acotr online network and Then, the two gradients are combined and the policy gradient is applied to update the parameter θ according to Equation (26); finally, the Actor target network and the Critic target network copy the parameters from the Actor online network and the Critic online network respectively in a soft update manner.

9. An adaptive dynamic combined load probability forecasting device, characterized in that: It includes a combined sub-model module, a Bayesian model averaging module, an evaluation module and a short-term load forecasting module based on a deep deterministic policy gradient algorithm, and is used to implement the adaptive dynamic combined load probability forecasting method described in any one of claims 1 to 8.

Images