CNN-BiGRU-Attention-based power load interval prediction method

Abstract

The invention belongs to the technical field of power system load prediction, and particularly relates to a CNN-BiGRU-Attention-based power load interval prediction method. Comprising the following steps: S1, extracting local features of a power load time sequence by using a convolutional neural network (CNN) to capture a local mode in time sequence data; s2, capturing forward time dependence and backward time dependence through a bidirectional gating cycle unit BiGRU; s3, applying an attention mechanism to carry out weight distribution on the features output by the BiGRU; and S4, generating prediction intervals under different confidence coefficients by using a quantile regression layer. According to the method, the problems of complexity and uncertainty in power load time sequence data processing are solved through a deep learning technology, and the precision and reliability of power load prediction can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power system load forecasting, and in particular relates to a power load interval forecasting method based on CNN-BiGRU-Attention. Background Art

[0002] With the continuous development of power systems and the complexity of power demand, power load forecasting plays a vital role in the planning, scheduling and operation of power systems. Accurate power load forecasting can help the power grid achieve more efficient energy scheduling and optimization, thereby reducing operating costs and improving the stability of the power system. However, since power load data usually exhibits obvious nonlinearity, time-varying and seasonal characteristics, traditional forecasting methods often have problems such as insufficient accuracy and limited generalization ability when dealing with these complexities.

[0003] In recent years, deep learning technology has made significant progress in time series forecasting, among which convolutional neural networks (CNN) and bidirectional gated recurrent neural networks (BiGRU) have been widely used for feature extraction and temporal dependency capture. However, most existing methods are aimed at point forecasting and cannot effectively deal with the uncertainty problem in power load forecasting. Interval forecasting can not only provide the forecast value, but also give the upper and lower intervals of the forecast value, which is more suitable for the fluctuation and uncertainty of actual power load.

[0004] Therefore, there is an urgent need for an algorithm that can effectively handle the complexity of power load time series data, which can not only accurately predict the load point value, but also give a reasonable prediction interval, thereby improving the intelligence level and adaptability of power system scheduling. Based on this, the present invention proposes a power load interval prediction algorithm based on CNN-BiGRU-Attention, and applies the algorithm to power load interval prediction for the first time, solving the shortcomings of the existing technology. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a power load interval prediction method based on CNN-BiGRU-Attention, which can effectively deal with the complexity and uncertainty of power load time series data to improve the accuracy and robustness of power load prediction.

[0006] The present invention is achieved in this way.

[0007] A method for predicting electric load intervals based on CNN-BiGRU-Attention, the method comprising:

[0008] S1 uses convolutional neural network (CNN) to extract local features of power load time series to capture local patterns in time series data;

[0009] S2 captures the forward and backward temporal dependencies through the bidirectional gated recurrent unit BiGRU;

[0010] S3 applies the attention mechanism to weight the features output by the bidirectional gated recurrent unit BiGRU;

[0011] S4 uses a quantile regression layer to generate prediction intervals at different confidence levels.

[0012] Furthermore, in S1, in the convolutional neural network CNN, the specific operation of the convolutional layer is:

[0013]

[0014] in Represents the feature map matrix of the l-th layer input, represents the weight matrix of the convolution kernel of the lth layer, represents the bias term of the lth layer, i is the label of the convolution kernel, j is the label of the convolution layer, and the specific two-dimensional form of the convolution operation is expressed as:

[0015]

[0016] Where M and N are the sizes of the convolution kernel, i and j are the steps in the convolution operation, X represents the pixel value at that location in the feature map during the convolution operation, W represents the weight at that location, and m and n represent the offset of the row and column that the convolution kernel moves on the feature map, respectively.

[0017] Furthermore, the formula of the pooling layer is as follows:

[0018]

[0019] Where P(i, j) is the result of pooling, and the maximum pooling operation selects the maximum value in the local area.

[0020] Furthermore, for the bidirectional gated recurrent unit BiGRU, the forward and backward GRUs are expanded as follows: Reset gate:

[0021]

[0022]

[0023] Update Gate:

[0024]

[0025]

[0026] Candidate hidden states:

[0027]

[0028]

[0029] The forward and backward hidden state outputs are:

[0030]

[0031]

[0032] The final bidirectional gated recurrent unit BiGRU hidden state output is:

[0033]

[0034] Among them, σ represents the sigmoid function, tanh represents the hyperbolic tangent activation function, W, U represent weight matrices, x represents the input of the time step, h represents the input of the hidden state, b represents the bias, the variable superscript forward represents the forward direction, and backward represents the reverse direction. The variable subscripts r, z, and h represent the reset gate, update gate, and candidate hidden state, respectively, and the subscript t represents the time point.

[0035] Furthermore, the attention mechanism Attention is applied to calculate the attention weight according to the hidden state of the input. The formula is described as follows:

[0036] Attention score:

[0037] e t =v T tanh(W h h t +b h ) (12)

[0038] Among them, W h is a trainable weight matrix, b h is the bias, v is the attention vector. h t Meaning t This is the final hidden state output result of the previous step, that is, the bidirectional gated recurrent unit BiGRU;

[0039] Attention weights:

[0040]

[0041] Weighted sum:

[0042]

[0043] Furthermore, the quantile regression layer in S4 assumes that: is the predicted value of the model at quantile T, the true value is y, and the loss function of quantile regression is expressed as:

[0044]

[0045] When τ is close to 0, the weight of the underestimated loss is larger; when τ is close to 1, the weight of the overestimated loss is larger. By minimizing the loss function, the model converts the specified quantile τ into the corresponding predicted value.

[0046] Furthermore, the loss function of the quantile regression is:

[0047] τ∈(0,1) represents the quantile of the target, τ=0.025 represents the lower 2.5% quantile prediction value, and τ=0.975 represents the upper 97.5% quantile prediction value.

[0048] when When , the loss is τ times the error when When , the loss is (1-τ) times the error

[0049] The upper and lower prediction interval calculation formula is:

[0050] Assuming that the lower quantile of the target is τ1 and the upper quantile is τ2, the prediction interval is expressed as:

[0051] .

[0053] Furthermore, quantile regression optimizes the model parameters by minimizing the weighted absolute error. Given a set of observations ((x i ,y i ) and the corresponding quantile τ, the quantile regression problem is as follows:

[0054]

[0055] in is the linear regression prediction of the model.

[0056] Compared with the prior art, the beneficial effects are:

[0057] 1 The present invention combines CNN, BiGRU and attention mechanism, which can not only extract local features in the power load time series, but also capture the previous and next dependencies, and highlight the key features through the attention mechanism, effectively improving the prediction accuracy;

[0058] 2 Compared with the traditional point prediction method, the present invention can generate interval prediction results of load, which can adapt to the volatility and uncertainty of power load;

[0059] 3 The present invention uses deep learning technology to perform automatic optimization, reduces human intervention, and has better adaptability and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of the method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0063] See also Figure 1 As shown, a method for predicting power load intervals based on CNN-BiGRU-Attention includes the following steps:

[0064] S1 uses convolutional neural network (CNN) to extract local features of power load time series to capture local patterns in time series data;

[0065] In the convolutional neural network CNN, the detailed process of convolution operation and pooling layer can be expanded as follows:

[0066] The specific operations of the convolutional layer are:

[0067] The specific operations of the convolutional layer are:

[0068]

[0069] in Represents the feature map matrix of the l-th layer input, represents the weight matrix of the convolution kernel of the lth layer, represents the bias term of the lth layer, i is the label of the convolution kernel, j is the label of the convolution layer, and the convolution operation can be expressed in a specific two-dimensional form:

[0070]

[0071] Where M and N are the sizes of the convolution kernel, i and j are the steps in the convolution operation, X represents the pixel value at that location in the feature map during the convolution operation, W represents the weight at that location, and m and n represent the offset of the row and column that the convolution kernel moves on the feature map, respectively.

[0072] The formula for a pooling layer (such as a max pooling layer) is as follows:

[0073]

[0074] Where P(i, j) is the result of pooling, and the maximum pooling operation selects the maximum value in the local area.

[0075] (2) The forward and backward temporal dependencies are captured by the bidirectional gated recurrent unit (BiGRU), improving the global feature capture capability of power load time series data.

[0076] For the bidirectional GRU model, the detailed formulas of the forward and backward GRU can be expanded as follows:

[0077] Reset the gate:

[0078]

[0079]

[0080] Update Gate:

[0081]

[0082]

[0083] Candidate hidden states:

[0084]

[0085]

[0086] The forward and backward hidden state outputs are:

[0087]

[0088]

[0089] The final BiGRU hidden state output is:

[0090]

[0091] Among them, σ represents the sigmoid function, tanh represents the hyperbolic tangent activation function, W, U represent the weight matrix, x represents the input of the time step, h represents the input of the hidden state, and b represents the bias. The variable superscript forward represents the forward direction, and backward represents the reverse direction. The variable subscripts r, z, and h represent the reset gate, update gate, and candidate hidden state, respectively, and the subscript t represents the time point.

[0092] (3) Apply the attention mechanism to weight the features output by BiGRU, highlight key features, and optimize the prediction results;

[0093] The Attention mechanism calculates the attention weight according to the hidden state of the input. The formula can be described in more detail as follows: Attention score:

[0094] e t =v T tanh(Wh h t +b h ) (12)

[0095] Among them, W h is a trainable weight matrix, b h is the bias, v is the attention vector. h t Meaning t This is the previous step, that is, the final hidden state output result of BiGRU.

[0096] Attention weights:

[0097]

[0098] Weighted sum:

[0099]

[0100] (4) Use the quantile regression layer to generate prediction intervals at different confidence levels:

[0101] Quantile regression is used in interval prediction to estimate the target variable value at different quantiles, thereby generating prediction intervals. Its mathematical principle is based on minimizing the quantile loss function. Assumptions: is the predicted value of the model at quantile T, the true value is y, and the loss function of quantile regression can be expressed as:

[0102]

[0103] The core idea of ​​this loss function is to weight the overestimated and underestimated losses according to different quantiles. When τ is small (close to 0), the underestimated loss weight is larger; and when τ is large (close to 1), the overestimated loss weight is larger. By minimizing this loss function, the model can generate the corresponding prediction value for the specified quantile τ.

[0104] Explanation of the loss function for quantile regression:

[0105] τ∈(0, 1) represents the quantile of the target, for example, τ=0.025 represents the lower 2.5% quantile prediction value, and τ=0.975 represents the upper 97.5% quantile prediction value.

[0106] when When , the loss is τ times the error ( when When , the loss is ((1-τ) times the error (

[0107] The upper and lower prediction interval calculation formula is:

[0108] Assuming that the lower quantile of the target is τ1 and the upper quantile is τ2, the prediction interval can be expressed as:

[0109]

[0110] For example, for a 95% confidence interval prediction, we usually set τ1 = 0.025 and τ2 = 0.975, and the resulting prediction interval is

[0111] Through quantile regression, the model can simultaneously generate multiple prediction intervals with different confidence levels to quantify the uncertainty of the prediction.

[0112] Optimization problem for quantile regression:

[0113] Quantile regression optimizes model parameters by minimizing the weighted absolute error. Given a set of observations ((x i ,y i ) and the corresponding quantile τ, the quantile regression problem can be written as follows:

[0114]

[0115] in is the linear regression prediction of the model.

[0116] This optimization method ensures that the model can generate corresponding prediction results at different quantiles, thereby constructing a complete prediction interval.

[0117] Loss function formula

[0118] To train for interval predictions, the loss function can include mean absolute percentage error (MAPE) and interval loss. The formula for MAPE is:

[0119]

[0120] For interval loss, the interval coverage (PICP) and interval width (MPIW) can be defined:

[0121] Interval coverage:

[0122]

[0123] Interval width:

[0124]

[0125] The comprehensive loss function can be designed as:

[0126]

[0127] where λ is a hyperparameter that controls the trade-off between interval width and coverage.

[0128]

[0129]

[0130] Uncertainty interval Δ t It can be given by an estimate of the model's variance or by predicting the model's uncertainty.

[0131] In summary, the structure of this power load forecasting method is:

[0132] First, the convolutional neural network (CNN) adopts a multi-layer convolution structure, which can extract multi-dimensional features of power load time series from different scales to improve the richness of feature extraction.

[0133] Secondly, the bidirectional gated recurrent unit (BiGRU) simultaneously captures time dependency through two time paths, forward and backward, to improve the accuracy of capturing load fluctuations.

[0134] Then, the attention mechanism dynamically assigns weights according to the importance of each feature, thereby improving the accuracy and robustness of power load interval prediction.

[0135] Example: Power load interval prediction method based on CNN-BiGRU-Attention

[0136] This embodiment uses the power load data of Belgium in 2018 as an example. The data has a step length of 15 minutes and 96 time steps per day. The interval prediction of power load is achieved through the following steps.

[0137] 1. Data preprocessing

[0138] By normalizing Belgium's 2018 annual power load data, the convergence of the model training process is ensured and the errors caused by data scale differences are reduced.

[0139] 2. CNN feature extraction

[0140] The preprocessed time series data is fed into CNN to extract local temporal features. CNN captures the changing trend of daily power load by learning local patterns in power load data.

[0141] 3. BiGRU captures temporal dependencies

[0142] The extracted features are input into BiGRU to capture the forward and backward temporal dependencies of the power load data. Through the processing of BiGRU, the temporal nature of the power load can be better modeled and the global perception of the data can be improved.

[0143] 4. Attention mechanism enhances features

[0144] Based on the BiGRU output, the Attention mechanism is used to assign weights to different time features, emphasizing the most critical time period for power load forecasting, thereby improving the prediction accuracy.

[0145] 5. Interval prediction

[0146] Unlike the traditional method of predicting a predicted value at a point in time, interval prediction can give multiple predicted values ​​corresponding to a time within a confidence interval (set to 0.95 in the present invention). The denser the intervals between these values, the more likely it is that the true value of the power load is within this range. Its significance lies in allowing the power load forecast to obtain a probability distribution, which allows users to intuitively understand the reliability of the prediction result in terms of probability. A model with an Attention mechanism can be used to generate interval prediction values ​​for power load, including point prediction values ​​and prediction intervals of upper and lower limits. In order to generate interval prediction results, quantile regression technology is used to estimate the upper and lower limits of the point prediction value at different times. By setting different quantiles (such as 0.05 and 0.95), the upper and lower limits within the 95% confidence interval of the power load are generated respectively.

[0147] 6. Result Verification

[0148] The Belgian power load data for the whole year of 2018 was used for verification, and the results are as follows:

[0149] (1) Training set performance:

[0150] MAPE: 0.094719;

[0151] The interval coverage is 0.96806 and the average interval width percentage is 0.41806.

[0152] (2) Test set performance:

[0153] MAPE: 0.07941;

[0154] The interval coverage is 1 and the average interval width percentage is 0.3293.

[0155] (3) Results analysis

[0156] The results of the training set and the test set show that the model has a certain fitting ability on both the training set and the test set. And the interval coverage rate is good, indicating that the model can generate reasonable prediction intervals.

[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for predicting power load intervals based on CNN-BiGRU-Attention, characterized in that: The method includes: S1 uses convolutional neural network (CNN) to extract local features of power load time series to capture local patterns in time series data; S2 captures the forward and backward temporal dependencies through the bidirectional gated recurrent unit BiGRU; S3 applies the attention mechanism to weight the features output by the bidirectional gated recurrent unit BiGRU; S4 uses a quantile regression layer to generate prediction intervals at different confidence levels.

2. The power load interval prediction method based on CNN-BiGRU-Attention according to claim 1 is characterized in that: In S1, in the convolutional neural network CNN, the specific operation of the convolution layer is: in Represents the feature map matrix of the l-th layer input, represents the weight matrix of the convolution kernel of the lth layer, represents the bias term of the lth layer, i is the label of the convolution kernel, j is the label of the convolution layer, and the specific two-dimensional form of the convolution operation is expressed as: Where M and N are the sizes of the convolution kernel, i and j are the steps in the convolution operation, X represents the pixel value at that location in the feature map during the convolution operation, W represents the weight at that location, and m and n represent the offset of the row and column that the convolution kernel moves on the feature map, respectively.

3. The power load interval prediction method based on CNN-BiGRU-Attention according to claim 1 is characterized in that: The formula for the pooling layer is as follows: Where P(i, j) is the result of pooling, and the maximum pooling operation selects the maximum value in the local area.

4. The power load interval prediction method based on CNN-BiGRU-Attention according to claim 1 is characterized in that: For the bidirectional gated recurrent unit BiGRU, the forward and backward GRU expansions are: Reset the gate: Update Gate: Candidate hidden states: The forward and backward hidden state outputs are: The final bidirectional gated recurrent unit BiGRU hidden state output is: Among them, σ represents the sigmoid function, tanh represents the hyperbolic tangent activation function, W, U represent weight matrices, x represents the input of the time step, h represents the input of the hidden state, b represents the bias, the variable superscript forward represents the forward direction, and backward represents the reverse direction. The variable subscripts r, z, and h represent the reset gate, update gate, and candidate hidden state, respectively, and the subscript t represents the time point.

5. The power load interval prediction method based on CNN-BiGRU-Attention according to claim 1 is characterized in that: The attention mechanism Attention is used to calculate the attention weight according to the hidden state of the input. The formula is described as follows: Attention score: e t =v T tanh(W h h t +b h ) (12) Among them, W h is a trainable weight matrix, b h is the bias, v is the attention vector. t Meaning t This is the final hidden state output result of the previous step, that is, the bidirectional gated recurrent unit BiGRU; Attention weights: Weighted sum:

6. The power load interval prediction method based on CNN-BiGRU-Attention according to claim 1 is characterized in that: The quantile regression layer in S4 assumes that: is the predicted value of the model at quantile T, the true value is y, and the loss function of quantile regression is expressed as: When τ is close to 0, the weight of the underestimated loss is larger; when τ is close to 1, the weight of the overestimated loss is larger. By minimizing the loss function, the model converts the specified quantile τ into the corresponding predicted value.

7. The method for predicting power load intervals based on CNN-BiGRU-Attention according to claim 6, characterized in that: The loss function of the quantile regression is: τ∈(0,1) represents the quantile of the target, τ=0.025 represents the lower 2.5% quantile prediction value, and τ=0.975 represents the upper 97.5% quantile prediction value. when When , the loss is τ times the error when When , the loss is (1-τ) times the error The upper and lower prediction interval calculation formula is: Assuming that the lower quantile of the target is τ1 and the upper quantile is τ2, the prediction interval is expressed as:

8. The method for predicting power load intervals based on CNN-BiGRU-Attention according to claim 7, characterized in that: Quantile regression optimizes model parameters by minimizing the weighted absolute error. Given a set of observations ((x i ,y i ) and the corresponding quantile τ, the quantile regression problem is as follows: in is the linear regression prediction of the model.

Images