Day-ahead electricity price prediction method based on parallel multi-dimensional attention mechanism

Abstract

The invention discloses a day-ahead electricity price prediction method based on a parallel multi-dimensional attention mechanism, and belongs to the technical field of electricity market day-ahead electricity price prediction. The method comprises the following steps: firstly, selecting a variable having a certain correlation with the electricity price by adopting a maximum mutual information coefficient to assist in predicting future electricity price data, and secondly, decomposing an original electricity price signal by adopting a decomposition algorithm; then, the electricity price sub-components and the electricity price related variables obtained through decomposition are input into an MBI-IPMDA-PBISA deep learning model, future electricity price prediction is carried out, and therefore the MBI-IPMDA-PBISA deep learning model can fully learn the complex incidence relation between the electricity price sub-components and the electricity price related variables and the future electricity price; the problem of error superposition caused by the fact that the electricity price sub-components are independently predicted and then superposed to obtain the future electricity price is avoided, the problem that an existing attention mechanism is not high in prediction precision is solved, and prediction precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of day-ahead electricity price prediction in the power market, and specifically relates to a day-ahead electricity price prediction method based on a parallel multi-dimensional attention mechanism. Background Art

[0002] Since clean energy such as wind power and photovoltaic power generation has been added to the power market in large quantities, the electricity price in the power market has fluctuated sharply, which has increased the difficulty of electricity price forecasting. Therefore, how to make accurate day-ahead electricity price forecasts when a high proportion of new energy has been added to the power market has become a major issue that domestic and foreign researchers are paying close attention to.

[0003] With the development of deep learning and data processing technology, many studies in recent years have begun to combine multiple algorithms to form a complex prediction framework. There are many feature information related to future electricity price data in electricity price data and electricity price related variable data. How to mine useful deep features to predict future electricity price data is very important.

[0004] The attention mechanism is a mechanism that imitates the different levels of attention that humans pay to things. It pays more attention to important features to select more critical information and improve the accuracy of the model. In 2017, Google proposed the self-attention mechanism (SA), which captures the key information of the input sequence itself by imitating the attention selectivity of biological vision. It has strong adaptive capabilities and is suitable for processing long-term dependencies of input sequences. In 2018, Jie Hu et al. proposed the squeeze-and-excitation networks (SE-Net), which used the channel attention mechanism to selectively adjust the importance of different feature channels, so that the network pays more attention to useful feature channels in the feature map to improve the performance of the entire network model; in 2018, Sanghyun WOO et al. proposed the convolutional block attention module (CBAM), which uses the channel attention module CAM (channel attention module, CAM) and the spatial attention module SAM (spartial attention module, SAM) to more comprehensively improve the attention of important features; in 2021, Qibin Hou et al. proposed the coordinate attention mechanism (CA), which integrates the feature map position information into the channel attention mechanism to increase the attention of more important feature points in the feature map, instead of only focusing on the important feature areas of the feature map.

[0005] However, since the SE attention mechanism processes the information in the channel globally, it only pays attention to the importance of each channel, but ignores the information interaction in space. CBAM pays attention to the information interaction between channels and space by connecting the channel attention module and the spatial attention module in series, while the spatial attention model first performs a pooling operation on the channel and then uses a 7×7 convolution to extract spatial features. It can only capture local information, which will cause some long-range dependencies to be ignored. The above method cannot be considered comprehensively, resulting in low accuracy of the existing attention mechanism in prediction. Summary of the invention

[0006] The purpose of the present invention is to address the problem of low prediction accuracy caused by incomplete consideration of existing attention mechanisms when making predictions, and propose a day-ahead electricity price prediction method based on a parallel multi-dimensional attention mechanism. The maximum mutual information coefficient is used to select variables with a certain correlation with electricity prices to assist in predicting future electricity price data. Secondly, a decomposition algorithm is used to decompose the original electricity price signal; then, the decomposed electricity price sub-components and electricity price-related variables are input into the MBI-IPMDA-PBISA deep learning model to predict future electricity prices. In this way, the prediction model can fully learn the complex correlation between electricity price sub-components and electricity price-related variables and future electricity prices, and avoid the error superposition problem caused by separately predicting electricity price sub-components and then superimposing them to obtain future electricity prices. This solves the problem of low prediction accuracy of the existing attention mechanism and improves the prediction accuracy.

[0007] Compared with the prior art, the present invention comprehensively considers the different deep features contained in the input electricity price and electricity price-related variables, adopts different feature extraction modules to mine their feature information, and finally proposes the MBI-IPMDA-PBISA deep learning model.

[0008] In order to solve the above technical problems, the present invention provides a method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism, comprising:

[0009] S1. Collect electricity price data and electricity price related variables, perform data preprocessing on the collected data, and construct an electricity price prediction data set;

[0010] S2. Based on the electricity price prediction data set, calculate the maximum mutual information coefficient between the electricity price data and electricity price-related variables, and screen out electricity price-related variables that have a certain correlation with the electricity price through the maximum mutual information coefficient;

[0011] S3, using a decomposition algorithm to decompose the electricity price data to obtain electricity price sub-components;

[0012] S4, build the MBI-IPMDA-PBISA deep learning model;

[0013] S5, dividing the electricity price subcomponents and the electricity price related variables screened in S2 into input and output to obtain a data set sample, each sample containing the electricity price subcomponents, electricity price related variables and the true value of the future electricity price, and merging the electricity price subcomponents and electricity price related variables in each sample to obtain an electricity price subcomponent input matrix and an electricity price related variable input matrix, and finally each sample contains the electricity price subcomponent input matrix, the electricity price related variable input matrix and the true value of the future electricity price;

[0014] S6. Divide the samples into a training set and a test set. First, input the training set into the MBI-IPMDA-PBISA deep learning model for training, and then input the test set into the trained MBI-IPMDA-PBISA deep learning model for testing to predict future electricity prices.

[0015] Preferably, the data preprocessing includes processing abnormal values ​​in the electricity price prediction data set and standardizing the electricity price prediction data set.

[0016] Preferably, the abnormal value processing includes: for data missing values, using the sliding window filling method to supplement; for the same time nodes and repeated data values, directly using one of the data, for the same time nodes but different data values, taking the average of the two data for subsequent calculation; for data abnormal values, using the 3δ criterion or the Grubbs test method to test the electricity price data and electricity price related variable data, if a certain point is determined to be abnormal value data, it will be deleted and then treated as a missing data value.

[0017] Preferably, the construction of the MBI-IPMDA-PBISA deep learning model comprises:

[0018] Sa. For the input branch of the electricity price sub-component, use two layers of Inception modules in series to extract multi-scale spatial features of the electricity price sub-component to obtain several multi-scale spatial feature matrices, and stack the extracted multi-scale spatial feature matrices in the third dimension to obtain the multi-scale spatial features of the electricity price sub-component;

[0019] Sb, adjusting the attention weights of each dimension in the multi-scale spatial features of the valence sub-component by serially parallelizing the multi-dimensional attention mechanism;

[0020] Sc. For the input branch of electricity price related variables, the structure of Inception module and BiLSTM module in parallel is used to simultaneously extract the spatial features and temporal features of electricity price related variables, and the spatial and temporal features are fused to obtain the spatiotemporal features of electricity price related variables;

[0021] Sd, adjust the internal attention weights in the spatiotemporal features of electricity price-related variables through the cascaded self-attention mechanism;

[0022] Se, the multi-scale spatial characteristics of the electricity price sub-components after adjusting the attention weights on each dimension and the spatiotemporal characteristics of the electricity price-related variables after adjusting the internal attention weights are fused and further integrated through the fully connected layer and the regression layer to obtain the output of the MBI-IPMDA-PBISA deep learning model, that is, the output of the future electricity price forecast value.

[0023] Preferably, the Sa includes: calculating the first Inception module of the electricity price sub-component input branch, and the calculation process of the second Inception module under this branch and the Inception module in the electricity price related variable input branch is the same, and the multi-scale spatial feature matrix extracted by the second Inception module of the electricity price sub-component input branch is input into the serial parallel multi-dimensional attention mechanism; wherein, the calculation process of the first Inception module of the electricity price sub-component input branch is as shown in the following formula:

[0024]

[0025] Where: F p is the Danish electricity price decomposition matrix, is the weight matrix of the i-th convolution kernel of the 1×1 convolution branch; They are the weight matrices of the i-th convolution kernel of the first layer 3×3, 5×5, and 7×7 convolutions respectively; They are the weight matrices of the i-th convolution kernel of the second layer 3×3, 5×5, and 7×7 convolutions respectively; is the bias of the i-th convolution kernel of the 1×1 convolution; They are the biases of the i-th convolution kernel of the first layer 3×3, 5×5, and 7×7 convolutions respectively; They are the biases of the i-th convolution kernel of the second layer 3×3, 5×5, and 7×7 convolutions respectively; Output feature of 1×1 convolution branch; They are the output features of the first layer 3×3, 5×5, and 7×7 convolutions respectively; They are the output features of the second layer 3×3, 5×5, and 7×7 convolutions respectively; δr is the Relu activation function; the feature matrices extracted from the four convolution branches are concat-connected to obtain the output multi-scale spatial feature matrix Similarly, the multi-scale spatial feature matrix output by the second Inception module in the electricity price subcomponent input branch and the Inception module in the electricity price related variable input branch is and

[0026] Preferably, the Sb comprises:

[0027] Sb1, multi-scale spatial feature matrix through global maximum pooling and global average pooling The spatial information of the image is compressed and aggregated to obtain a feature map with an output size of C×1×1. Two layers of 1×1 convolution are set for scaling learning and activated by the Sigmoid function to obtain the attention weight on the first dimension. The calculation process is shown in the following formula:

[0028]

[0029] Where: GMP and GAP are global maximum pooling and global average pooling operations; is the vector dot product; and is the weight of the i-th 1×1 convolution kernel in the first and second layers in the C dimension; and is the bias of the i-th 1×1 convolution kernel in the first and second layers; It is the feature output after global maximum pooling and global average pooling dot product processing; and f CA is the output after feature learning through the first and second layers of 1×1 convolution, σ is the Sigmoid activation function;

[0030] Sb2, use two one-dimensional maximum pooling and one-dimensional average pooling to compress and aggregate the vertical spatial information and horizontal spatial information of the multi-scale spatial feature matrix respectively, and obtain the feature maps with output of C×H×1 and C×1×W. The calculation process is shown in the following formula:

[0031]

[0032]

[0033] Where: and for The output of the jth channel in the H dimension after average pooling and maximum pooling; and for The output of the jth channel in dimension W after average pooling and maximum pooling; τ represents the maximum pooling and average pooling operations of different spatial ranges for each channel;

[0034] Two layers of 1×1 convolution are set for scaling learning and activated by the Sigmoid function to obtain the vertical and horizontal attention weights. The calculation process is shown in the following formula:

[0035]

[0036] Where: and is the weight of the i-th 1×1 convolution kernel in the first and second layers in the H dimension; and is the bias of the i-th 1×1 convolution kernel in the first and second layers in the H dimension; and is the weight of the i-th 1×1 convolution kernel in the first and second layers in dimension W; and is the bias of the i-th 1×1 convolution kernel in the first and second layers in the W dimension; and f HA The output after learning the H-dimensional features through the first and second layers of 1×1 convolution; and f WA The output after learning the W-dimensional features through the first and second layers of 1×1 convolution;

[0037] Sb3, multiply the attention weights obtained in different dimensions by the original multi-scale spatial feature matrix to obtain the multi-dimensional attention feature matrix f DA , as shown below:

[0038]

[0039] Preferably, the Sc comprises:

[0040] For the input branch of electricity price related variables, the Inception and BiLSTM parallel structures are used to simultaneously extract the spatial features and time series features of electricity price related variables; the input matrix of electricity price related variables is transformed into a 1-dimensional long vector through the flat layer and input into the BiLSTM module. The connection between historical data and future data is strengthened through the forward and reverse propagation laws, and the time series features of electricity price related variables are extracted. The calculation process is shown in the following formula;

[0041]

[0042]

[0043] Where: and are the outputs of the forward hidden layer and the backward hidden layer at time t respectively; and are the weight matrices that map the input layer to the forward hidden layer and the backward hidden layer respectively; and are the weight matrices that map the output of the forward hidden layer and the backward hidden layer from the previous moment to the current moment; b (1) and b (2)are the biases of the forward hidden layer and the backward hidden layer respectively; W1 and W2 are the weight matrices mapping the output of the forward hidden layer and the backward hidden layer to the output layer respectively; δt is the Tanh activation function; It is the output of the electricity price related variable matrix after the flattening layer; Represents the output of the output layer. Similarly, this time series feature f v,1 After inputting into the second BiLSTM network of this branch, the time series features are obtained as follows: Subsequent use represent

[0044] The spatial features extracted by the Inception module The time series features extracted by the BiLSTM module after being flattened by the flat layer Perform feature fusion to obtain the spatiotemporal characteristics of electricity price related variable data f v , the calculation process is shown as follows:

[0045] in, Represents concatenation from the first dimension.

[0046] Preferably, the Sd comprises:

[0047] Through the self-attention mechanism, the different position information in the spatiotemporal characteristics of the electricity price-related variable data are associated and interacted with each other, and the dependencies therein are learned more comprehensively, focusing on the important information therein. The calculation process is shown in the following formula:

[0048]

[0049] Randomly initialize 4 sets of weight matrices W i Q , W i K , W i V , the spatiotemporal characteristics of electricity price related variable data f v Multiplying these 4 sets of weight matrices, we get 4 sets of Q i , K i 、V i (i∈0,1,2,3); the matrix Q i With K i The inner product is normalized by the softmax function to obtain the correlation weight matrix, which is then combined with the matrix V i Perform weighted summation to obtain the weight coefficient Z; finally, multiply Z by the initialization matrix W0 to obtain the spatiotemporal feature f containing the self-attention mechanism vSA .

[0050] Where: W iQ , W i K , W i V represents the initialized weight matrix, Q i , K i 、V i denote the query vector, key vector and value vector respectively, d k represents the dimension of the key vector, Z represents the weight coefficient, W 0 represents the weight matrix.

[0051] Preferably, the data set samples are divided into a training set and a test set. The training set is first input into the MBI-IPMDA-PBISA deep learning model for training to obtain a trained MBI-IPMDA-PBISA deep learning model. The test set data is then input into the trained MBI-IPMDA-PBISA deep learning model to predict future electricity prices. The prediction results are compared with the true values ​​to obtain the prediction accuracy and prediction error of the test set. In engineering applications, historical data is input into the trained MBI-IPMDA-PBISA deep learning model to obtain the future electricity price data to be predicted.

[0052] Beneficial effects of the present invention:

[0053] 1. This scheme proposes a parallel multi-dimensional attention mechanism to avoid the problem that the SE attention mechanism only focuses on the importance of each channel and ignores the information interaction in space, and the problem that CBAM can only capture local information by focusing on the information interaction between channels and space through the series connection of channel attention module and spatial attention module. The parallel multi-dimensional attention mechanism uses the idea of ​​"two one-dimensional global pooling operations to extract feature information in the vertical and horizontal directions respectively, and the two independent direction-aware feature maps obtained can capture the long-range dependencies in different directions on the input feature map" and integrates the advantages of CBAM using average pooling and maximum pooling to aggregate feature information at the same time, comprehensively considering different feature extraction modules to mine feature information, and improving the prediction accuracy of the MBI-IPMDA-PBISA model for predicted data.

[0054] 2. This scheme assists in predicting future electricity prices by considering electricity price-related variables, and uses the maximum household information coefficient to effectively eliminate related variables with low correlation with electricity prices, thereby reducing the computational complexity of the deep learning prediction model.

[0055] 3. This scheme uses the proposed MBI-IPMDA-PBISA deep learning model to first extract the deep features of electricity prices and electricity price-related variables respectively, and then perform feature fusion to predict future electricity prices. This allows the prediction model to fully learn the complex correlation between electricity price sub-components, electricity price-related variables and future electricity prices, and avoids the error superposition problem caused by predicting electricity price sub-components separately and then superimposing them to obtain future electricity prices.

[0056] 4. This solution uses three convolution layers with different convolution window sizes of 1×1, 3×3 and 5×5 and a maximum pooling layer to extract the spatial features of the input data. The receptive fields of different convolution window sizes are different, and the extracted spatial features are different. The features extracted by the pooling layer are different from the features extracted by the convolution layer. Combining them into a multi-channel spatial feature matrix can complement the feature extraction advantages of different convolution layers and pooling layers, and enhance their feature extraction performance. At the same time, this solution adopts the idea that different convolution window sizes in the Inception deep learning model extract different spatial features, and continues to use the three convolution window sizes of 1×1, 3×3 and 5×5 in the Inception basic model to increase feature diversity. The maximum pooling layer will ignore some element values ​​when extracting features, which is more likely to cause overfitting problems, and will excessively improve the accuracy of the training set, resulting in a decrease in test accuracy. Therefore, the Inception deep learning model used in this invention uses a 7×7 convolution module to replace the maximum pooling layer, and in order to expand the receptive field of the convolution and increase the network depth, the 3×3, 5×5 and 7×7 convolutions are set to two layers; furthermore, since the LSTM network can only process data in one direction, it often ignores the information hidden in the future. The BiLSTM network is composed of forward and backward LSTM networks respectively. While obtaining past information, it can also make good use of future information, which can further improve the prediction accuracy of time-series data. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are only for the purpose of illustrating preferred embodiments and are not to be considered as limiting the present invention. Also, the same reference symbols are used throughout the drawings to represent the same parts.

[0058] Figure 1 Schematic diagram of the parallel multi-dimensional attention mechanism of the present invention;

[0059] Figure 2 It is a schematic diagram of the structure of the MBI-IPMDA-PBISA deep learning model of the present invention;

[0060] Figure 3It is a schematic diagram of the change curve of data containing missing values ​​of the present invention;

[0061] Figure 4 The normalized electricity price and its related variable data diagram of the present invention;

[0062] Figure 5 It is a maximum mutual information coefficient value diagram between the DK1 electricity price and other related variables of the present invention;

[0063] Figure 6 This is a schematic diagram of the decomposition result of the DK1 electricity price data group of the present invention;

[0064] Figure 7 A schematic diagram of input and output of a sliding window construction model of the present invention;

[0065] Figure 8 It is a schematic diagram of the prediction results of the MBI-IPMDA-PBISA deep learning model of the present invention;

[0066] Fig. 9 Schematic diagram of the day-ahead electricity price prediction error of existing technologies with different attention mechanism models. DETAILED DESCRIPTION

[0067] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation method described herein is only an optimal embodiment of the present invention, which is only used to explain the present invention and does not limit the scope of protection of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0068] Example 1: Figure 1 As shown in the figure, the day-ahead electricity price prediction method based on the parallel multi-dimensional attention mechanism includes:

[0069] Step 1: Collect electricity price data P and electricity price related variables such as power generation, power consumption, power transaction volume and other data Vm, where m represents the number of electricity price related variables.

[0070] Step 2: Process outliers in all data.

[0071] For missing data values, the sliding window filling method is used to supplement them (such as the data processing process in the implementation example)

[0072] Duplicate data is usually caused by recording errors. If the data values ​​are exactly the same, just use one of the data. If the data values ​​are different, take the average of the two data for subsequent calculations. (Duplicate data is rare. Generally, duplicate data with different data values ​​may be due to special circumstances. The electricity price has changed during a period of time when it should have remained unchanged. The average method can be used to make it consistent with other data formats.)

[0073] For data outliers (a point in the data suddenly undergoes a large distortion due to recording errors), the 3δ criterion or Grubbs test method can be used to test electricity price data and electricity price-related variable data. If a point is determined to be an outlier, it will be deleted and then treated as a missing value.

[0074] Step 3: Standardize all data to unify the dimensions of electricity prices and electricity price-related variables.

[0075] Step 4: Calculate the maximum mutual information coefficient between electricity prices and electricity price-related variables, and select variables that have a certain correlation with electricity prices to assist in predicting future electricity price data.

[0076] The value range of the maximum mutual information coefficient is [0,1]. The closer the value is to 1, the greater the correlation between the current electricity price related variables and the historical electricity price. The closer the value is to 0, the smaller the correlation between the current electricity price related variables and the historical electricity price. How to set the threshold has a great impact on the subsequent overall experiment. If the threshold is set too large, too few electricity price related variables will be selected, and some important feature information contained in other electricity price related variables will be ignored. If the threshold is set too small, too much redundant information of electricity price related variables will be input into the deep learning model, resulting in increased calculation and prediction difficulty of the deep learning model.

[0077] The present invention refers to other relevant studies and comprehensive considerations when selecting electricity price-related variables in the implementation example, and ultimately determines that the threshold is 0.2.

[0078] Step 5: Decompose the electricity price signal using a signal decomposition algorithm to obtain electricity price subcomponents IMF1 to IMFn.

[0079] Step 6: While constructing the MBI-IPMDA-PBISA deep learning model, the input and output of the MBI-IPMDA-PBISA deep learning model are divided for the electricity price sub-components and electricity price-related variables to obtain the electricity price sub-component input matrix F p And related variable input matrix Fv, where p and v represent the number of samples of the electricity price sub-component input matrix and the related variable input matrix respectively.

[0080] Step 7: Divide all samples into training set and test set.

[0081] Step 8: Input the training set data into the MBI-IPMDA-PBISA deep learning model to train the model.

[0082] like Figure 2 As shown, it can be understood that for the MBI-IPMDA-PBISA deep learning model, for the electricity price sub-component input branch, since its historical electricity price data is decomposed to obtain multiple electricity price sub-components, and the temporal correlation information of the original electricity price data has changed, and there is spatial correlation information between the electricity price sub-components, a two-layer Inception module is used in series to extract multi-scale spatial features. Convolution kernels of different sizes can extract features of different scales. The extracted features are stacked to obtain multi-scale electricity price spatial features with richer feature information, so as to enhance the feature extraction ability of the model, and then the PMDA module (parallel multi-dimensional attention mechanism) is connected in series to adjust the attention weights of each dimension in the multi-scale electricity price spatial features in the electricity price sub-component extracted by the Inception module.

[0083] For the input branch of electricity price-related variables, since each of its related variables contains time series correlation information and there is spatial correlation information between the related variables, a parallel structure of the Inception module and the BiLSTM module is used to simultaneously extract the spatial and time series features of the electricity price-related variables. The spatial and time series features are fused to obtain the spatiotemporal features of the electricity price-related variables, and then the SA module (self-attention mechanism) is connected in series to adjust the internal attention weight of this spatiotemporal feature.

[0084] The multi-scale spatial characteristics of electricity prices after weight adjustment by the PMDA module and the spatiotemporal characteristics of electricity price-related variables after weight adjustment by the SA module are fused, and then input into the fully connected layer for further integration to output the future electricity price forecast value.

[0085] The Inception basic model was proposed by the Google team in the 2014 ImageNet competition. It uses convolution windows of different sizes and pooling to extract spatial feature matrices with different feature regularities in the input image, and then stacks them and gives them to the next module. This well preserves the different regularities of each spatial feature matrix, increases the width of the network, and has a stronger spatial feature extraction capability.

[0086] The network uses three convolutional layers with different convolutional window sizes of 1×1, 3×3 and 5×5 and a maximum pooling layer to extract the spatial features of the input data. The receptive fields of different convolutional window sizes are different, and the extracted spatial features are different. The features extracted by the pooling layer are different from the features extracted by the convolutional layer. Combining them into a multi-channel spatial feature matrix can complement the feature extraction advantages of different convolutional layers and pooling layers, thereby enhancing their feature extraction performance.

[0087] The MBI-IPMDA-PBISA deep learning model proposed in the present invention adopts the idea that the spatial features extracted by convolutions of different convolution window sizes in the Inception deep learning model are different, and the convolutions of three different convolution window sizes of 1×1, 3×3 and 5×5 in the Inception basic model are used to increase feature diversity. However, the maximum pooling layer will ignore some element values ​​when extracting features, which is more prone to overfitting problems, and will excessively improve the accuracy of the training set, thereby causing a decrease in test accuracy. Therefore, the Inception deep learning model adopted by the present invention uses a 7×7 convolution module to replace the maximum pooling layer, and in order to expand the receptive field of the convolution and increase the network depth, 3×3, 5×5 and 7×7 convolutions are set to two layers.

[0088] Since the LSTM network can only process data in one direction, it often ignores the hidden information in the future. The BiLSTM network is composed of forward and backward LSTM networks. While obtaining past information, it can also make good use of future information, which can further improve the prediction accuracy of time-series data.

[0089] The BiLSTM network consists of an input layer, an output layer, a forward hidden layer, and a backward hidden layer. The input data of the input layer is sent to the forward hidden layer and the backward hidden layer at the same time, and the data of the output layer is composed of the forward hidden layer and the backward hidden layer.

[0090] The calculation process of the MBI-IPMDA-PBISA deep learning model is as follows: for the electricity price sub-component matrix in input 1, first use two Inception modules to extract multi-scale features. Convolution kernels of different sizes can extract features of different scales. The extracted features are stacked to obtain electricity price spatial features with richer feature information, thereby enhancing the model's feature extraction capability. The calculation process of the first Inception module in the electricity price input branch is shown in formula (1), and the calculation process of the second Inception module and the Inception module in the electricity price-related variable input branch is the same.

[0091]

[0092] Where: F p is the Danish electricity price decomposition matrix, is the weight matrix of the i-th convolution kernel of the 1×1 convolution branch; They are the weight matrices of the i-th convolution kernel of the first layer 3×3, 5×5, and 7×7 convolutions respectively; They are the weight matrices of the i-th convolution kernel of the second layer 3×3, 5×5, and 7×7 convolutions respectively; is the bias of the i-th convolution kernel of the 1×1 convolution; They are the biases of the i-th convolution kernel of the first layer 3×3, 5×5, and 7×7 convolutions respectively; They are the biases of the i-th convolution kernel of the second layer 3×3, 5×5, and 7×7 convolutions respectively; Output feature of 1×1 convolution branch; They are the output features of the first layer 3×3, 5×5, and 7×7 convolutions respectively; They are the output features of the second layer 3×3, 5×5, and 7×7 convolutions respectively; δr is the Relu activation function; the feature matrices extracted from the four convolution branches are concat-connected to obtain the output multi-scale spatial feature matrix Similarly, the multi-scale spatial feature matrix output by the second Inception module in the electricity price decomposition matrix input branch and the Inception module in the electricity price related variable input branch is and

[0093] Subsequently, the multi-scale spatial feature matrix extracted by the second Inception module of the electricity price decomposition matrix input branch is input into the PMDA module, which first compresses and aggregates the spatial information of the multi-scale spatial feature matrix through global maximum pooling and global average pooling to obtain a feature map with an output size of C×1×1. Then, two layers of 1×1 convolution are set for scaling learning and activated by the Sigmoid function to obtain the attention weight on the first dimension (channel). The calculation process is shown in formula (2).

[0094]

[0095] Where: GMP and GAP are global maximum pooling and global average pooling operations; is the vector dot product; and is the weight of the i-th 1×1 convolution kernel in the first and second layers in the C dimension; and is the bias of the i-th 1×1 convolution kernel in the first and second layers; It is the feature output after global maximum pooling and global average pooling dot product processing; and f CAis the output after feature learning through the first and second layers of 1×1 convolution, and σ is the Sigmoid activation function. Subsequently, two one-dimensional maximum pooling and one-dimensional average pooling are used to compress and aggregate the vertical spatial information and horizontal spatial information of the multi-scale spatial feature matrix respectively, and the output feature maps are C×H×1 and C×1×W. The calculation process is shown in equations (3) and (4). Subsequently, two layers of 1×1 convolution are set for scaling learning and activated by the Sigmoid function to obtain the attention weights in the second dimension (vertical) and the third dimension (horizontal). The calculation process is shown in equation (5).

[0096]

[0097] Where: and for The output of the jth channel in the H dimension after average pooling and maximum pooling; and for The output of the jth channel in dimension W after average pooling and maximum pooling; τ represents the maximum pooling and average pooling operations of different spatial ranges for each channel; and is the weight of the i-th 1×1 convolution kernel in the first and second layers in the H dimension; and is the bias of the i-th 1×1 convolution kernel in the first and second layers in the H dimension; and is the weight of the i-th 1×1 convolution kernel in the first and second layers in dimension W; and is the bias of the i-th 1×1 convolution kernel in the first and second layers in the W dimension; and f HA The output after learning the H-dimensional features through the first and second layers of 1×1 convolution; and f WA It is the output after learning the W-dimensional features through the first and second layers of 1×1 convolution.

[0098] Multiply the attention weights obtained in different dimensions with the original multi-scale spatial feature matrix to obtain the multi-dimensional attention feature matrix f DA , as shown in formula (6).

[0099]

[0100] For the input branch of electricity price related variables, the parallel structure of Inception and BiLSTM is used to simultaneously extract the spatial features and time series features of electricity price related variables. The data of electricity price related variables are transformed into a 1-dimensional long vector through the flat layer and input into the BiLSTM module. The connection between historical data and future data is strengthened through the forward and backward propagation rules, and the time series features of electricity price related variables are extracted. The calculation process is shown in equations (7) and (8).

[0101]

[0102]

[0103] Where: and are the outputs of the forward hidden layer and the backward hidden layer at time t respectively; and are the weight matrices that map the input layer to the forward hidden layer and the backward hidden layer respectively; and are the weight matrices that map the output of the forward hidden layer and the backward hidden layer from the previous moment to the current moment; b (1) and b (2) are the biases of the forward hidden layer and the backward hidden layer respectively; W1 and W2 are the weight matrices mapping the output of the forward hidden layer and the backward hidden layer to the output layer respectively; δt is the Tanh activation function; It is the output of the electricity price related variable matrix after the flattening layer; Represents the output of the output layer. Similarly, this time series feature f v,1 After inputting into the second BiLSTM network of this branch, the time series features are obtained as follows: Subsequent use represent

[0104] The multi-scale spatial features extracted by the Inception module The time series features extracted by the BiLSTM module after being flattened by the flat layer Perform feature fusion to obtain the spatiotemporal characteristics of electricity price related variable data f v , and then the self-attention mechanism is used to associate and interact with each other in the sequence, so as to learn the dependencies in the sequence more comprehensively and focus on the important information in the sequence. The calculation process is shown in equations (9) and (10).

[0105]

[0106] Randomly initialize 4 sets of weight matrices W i Q , W i K , Wi V , the spatiotemporal characteristics of electricity price related variable data f v Multiplying these 4 sets of weight matrices, we get 4 sets of Q i , K i 、V i (i∈0,1,2,3); the matrix Q i With K i The inner product is normalized by the softmax function to obtain the correlation weight matrix, which is then combined with the matrix V i Perform weighted summation to obtain the weight coefficient Z; finally, use Z and the initialization matrix W 0 Multiply them together to obtain the spatiotemporal feature information f containing the attention mechanism vSA .

[0107] Finally, the attention feature information obtained from the electricity price decomposition matrix input branch and the electricity price related variable input branch are fused, and the electricity price prediction value Y is output after the fully connected layer.

[0108] Step 9: Input the test set data into the trained MBI-IPMDA-PBISA deep learning model to obtain the future electricity price forecast.

[0109] This embodiment uses the maximum mutual information coefficient to select variables that have a certain correlation with electricity prices to assist in predicting future electricity price data. Secondly, a decomposition algorithm is used to decompose the original electricity price signal; then, the decomposed electricity price sub-components and electricity price-related variables are input into the MBI-IPMDA-PBISA deep learning model to predict future electricity prices. In this way, the prediction model can fully learn the complex correlation between the electricity price sub-components and electricity price-related variables and future electricity prices, and avoid the error superposition problem caused by predicting the electricity price sub-components separately and then superimposing them to obtain the future electricity price. This solves the problem of low accuracy of the existing attention mechanism in prediction and improves the prediction accuracy.

[0110] Example 2: Take a DK1 area provided by a power exchange as an example:

[0111] Step 1: Collect electricity price data for a total of 365 days from December 22, 2021 to December 21, 2022 provided by a certain power exchange, and 27 related variable data such as power generation, wind power generation, power consumption, and various power trading volumes within and outside a certain region.

[0112] Step 2: Use the sliding average window filling method to fill in the missing values ​​in all data, where the sliding window length is set to 7, so the filled data value is equal to the average of the sum of the first three data points and the last three data points. In order to intuitively show the comparison effect before and after missing value filling, select the DK1 electricity price data on March 27, 2022 and use the sliding window filling method for its missing values. The data change curve before and after filling is as follows Figure 3 shown.

[0113] like Figure 3 It can be seen that there is a missing value at 03:00 on March 27, 2022, indicating that the data in the DK1 area in the time period of 02:00-03:00 is missing. The sliding window filling method is used to fill the data, where the sliding window is 7, so the filled data value is equal to the average of the sum of the first three data points and the last three data points. The data after sliding window filling is more complete, eliminating the problem of affecting the entire experimental research process due to a missing value.

[0114] Step 3: Normalize all data to unify the dimensions of electricity prices and electricity price-related variables.

[0115] Min-Max standardization is used to normalize the electricity price and its related variables after outlier processing. Assuming a vector is X, Min-Max standardization can be expressed as

[0116]

[0117] Where: X norm is the value after Min-Max normalization, X i is the i-th value of X, X max is the maximum value of X, X min is the maximum value of X. The normalized data of a certain electricity price and its related variables are as follows: Figure 4 As shown in Figure 2, there are too many variables related to electricity prices. Figure 4 Only some of the variables are shown.

[0118] Step 4: Calculate the maximum mutual information coefficient between electricity price and electricity price related variables. The result is as follows: Figure 5 shown.

[0119] According to the calculation results of the maximum mutual information coefficient between electricity price and electricity price-related variables, 10 variables including DK2 electricity price, DK1 wind power generation, DK2 wind power generation, DK1 wind power generation forecast value, DK2 wind power generation forecast value, DK power trading volume, DK and DE power trading volume, DK and NO power trading volume, DK1 and DE trading volume and DK1 and NL power trading volume were finally selected as electricity price-related variables to assist in predicting DK1 electricity price data.

[0120] Step 5: Use the swarm decomposition (SWD) algorithm to decompose the electricity price signal and obtain the electricity price subcomponents IMF1 to IMF10, as shown in Figure 6 shown.

[0121] Step 6: Divide the model input and output for the electricity price sub-components and electricity price-related variables, and use the sliding window mechanism to construct the model input and output. The specific construction process is as follows: Figure 7 shown.

[0122] In the figure, n is the length of a certain electricity price data sequence, T∈n, and the input data length of the model is set to t. For example, input the data of the time period T+1 to T+t, predict the future electricity price data of T+t+1 to T+t+24, and then slide this window back 24 steps to get the next set of input and output data. In this example, t is set to 48.

[0123] Step 7: Use the last 30 days of data from December 22, 2021 to December 21, 2022 as the test set to test the trained deep learning model, and the remaining data as the training set to train this deep learning model.

[0124] Step 8: Input the training set data into the MBI-IPMDA-PBISA deep learning model to train the model.

[0125] Step 9: Input the test set data into the trained MBI-IPMDA-PBISA deep learning model to obtain the future electricity price forecast.

[0126] In order to verify the superiority of PMDA proposed in the present invention in the MBI-IPMDA-PBISA prediction model, under the same data preprocessing method, SE-Net, CBAM and CA are used to replace PMDA to obtain three prediction models with different attention mechanisms, namely MBI-ISE-PBISA, MBI-ICBAM-PBISA and MBI-ICA-PBISA, for day-ahead electricity price prediction, and their experimental error results are compared.

[0127] In order to uniformly evaluate the experimental results and analyze the prediction performance of different models, the mean absolute error (e MAE ), mean absolute percentage error (e MAPE ), symmetric mean absolute percentage error (e sMAPE ), root mean square error (e RMSE) and the coefficient of determination (R 2) As a performance evaluation function, its calculation formula is:

[0128]

[0129]

[0130] Where: n represents the length of the electricity price data sequence to be predicted. i Indicates the real value of electricity price; represents the predicted value of electricity price; Represents the average value of the predicted electricity price true value.

[0131] The prediction results are shown in Table 1 and Figure 8 shown.

[0132] Table 1. Day-ahead electricity price forecast error in DK1 region with different attention mechanism models

[0133]

[0134] It can be seen from Table 1 that the MBI-IPMDA-PBISA model with parallel multi-dimensional attention mechanism proposed in this invention has the best prediction effect in the day-ahead electricity price prediction experiment. MAE 、e MAPE 、e sMAPE 、e RMSE and R 2 They are 11.6172 (EUR / MWh), 0.0438, 0.0435, 16.1382 (EUR / MWh) and 0.9818 respectively. Compared with the MBI-ISE-PBISA, MBI-ICBAM-PBISA and MBI-ICA-PBISA prediction models, the e MAE They decreased by 21.34%, 12.30% and 5.93% respectively. MAPE They decreased by 18.44%, 6.81% and 4.37% respectively. sMAPE They decreased by 22.04%, 10.12% and 2.25% respectively. RMSE They decreased by 18.99%, 11.44% and 4.19% respectively. 2 The increases were 0.0100, 0.0054 and 0.0020 respectively.

[0135] In order to more intuitively display the day-ahead electricity price forecast error of the forecasting model with various attention mechanisms, the error results are displayed in the radar chart. The specific effect is as follows: Fig. 9 shown.

[0136] Fig. 9The brown line represents the prediction error of the MBI-ISE-PBISA model, the blue line represents the prediction error of the MBI-ICBAM-PBISA model, the purple line represents the prediction error of the MBI-ICA-PBISA model, and the red line represents the prediction error of the MBI-IPMDA-PBISA model proposed in the present invention. MAE The range is 0~15(EUR / MWh), e MAPE The range is 0~0.1, e sMAPE The range is 0~0.1, e RMSE The range is 0~20(EUR / MWh), R 2 The range is 0.9~1. Fig. 9 It can be seen that the prediction errors of the MBI-ISE-PBISA prediction model are the largest, and the determination coefficient (R 2) The smallest, indicating that the model with channel attention mechanism has the worst prediction performance. The prediction errors of MBI-ICBAM-PBISA and MBI-ICA-PBISA prediction models are slightly smaller, indicating that the prediction performance of the models with CBAM and CA is slightly better than that of MBI-ISE-PBISA model. The MBI-IPMDA-PBISA model with PMDA proposed in the present invention has the smallest prediction errors, and R 2 The highest value indicates that its fitting effect is the best and its model prediction performance is the best.

[0137] The above specific embodiments are preferred embodiments of the present invention, and are not intended to limit the specific implementation scope of the present invention. The scope of the present invention includes but is not limited to the present specific embodiments. All equivalent changes made in accordance with the shape, structure, and method of the present invention are within the protection scope of the present invention.

Claims

1. A day-ahead electricity price prediction method based on a parallel multi-dimensional attention mechanism, characterized in that: The steps include: S1. Collect electricity price data and electricity price related variables, perform data preprocessing on the collected data, and construct an electricity price prediction data set; S2. Based on the electricity price prediction data set, calculate the maximum mutual information coefficient between the electricity price data and electricity price-related variables, and screen out electricity price-related variables that have a certain correlation with the electricity price through the maximum mutual information coefficient; S3, using a decomposition algorithm to decompose the electricity price data to obtain electricity price sub-components; S4, build the MBI-IPMDA-PBISA deep learning model; S5, dividing the electricity price subcomponents and the electricity price related variables screened in S2 into input and output to obtain a data set sample, and merging the electricity price subcomponents and electricity price related variables in each sample to obtain an electricity price subcomponent input matrix and an electricity price related variable input matrix; S6. Divide the data set samples into a training set and a test set. First, input the training set into the MBI-IPMDA-PBISA deep learning model for training, and then input the test set into the trained MBI-IPMDA-PBISA deep learning model for testing to predict future electricity prices.

2. The method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism according to claim 1, characterized in that: The data preprocessing includes processing abnormal values ​​in the electricity price prediction data set and standardizing the electricity price prediction data set.

3. The method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism according to claim 2, characterized in that: The processing of abnormal values ​​includes: for data missing values, a sliding window filling method is used to supplement; when the time nodes are the same and the data values ​​are repeated, one of the data is directly used, and when the time nodes are the same but the data values ​​are different, the average value of the two data can be taken for subsequent calculation; for data abnormal values, the 3δ criterion or the Grubbs test method is used to test the electricity price data and the electricity price related variable data. If a certain point is determined to be an abnormal value data, it is deleted and then treated as a missing data value.

4. The method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism according to claim 1, characterized in that: The construction of the MBI-IPMDA-PBISA deep learning model includes: Sa. For the input branch of the electricity price sub-component, use two layers of Inception modules in series to extract multi-scale spatial features of the electricity price sub-component to obtain several multi-scale spatial feature matrices, and stack the extracted multi-scale spatial feature matrices in the third dimension to obtain the multi-scale spatial features of the electricity price sub-component; Sb, adjusting the attention weights of each dimension in the multi-scale spatial features of the valence sub-component by serially parallelizing the multi-dimensional attention mechanism; Sc. For the input branch of electricity price related variables, the structure of Inception module and BiLSTM module in parallel is used to simultaneously extract the spatial features and temporal features of electricity price related variables, and the spatial and temporal features are fused to obtain the spatiotemporal features of electricity price related variables; Sd, adjust the internal attention weights in the spatiotemporal features of electricity price-related variables through the cascaded self-attention mechanism; Se, the multi-scale spatial features of the electricity price sub-components after adjusting the attention weights on each dimension and the spatiotemporal features of the electricity price-related variables after adjusting the internal attention weights are fused and further integrated through the fully connected layer and the regression layer to obtain the output of the MBI-IPMDA-PBISA deep learning model.

5. The method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism according to claim 4, characterized in that: The Sa includes: the first Inception module of the electricity price sub-component input branch is calculated, and the calculation process of the second Inception module under this branch and the Inception module in the electricity price related variable input branch is the same, and the multi-scale spatial feature matrix extracted by the second Inception module of the electricity price sub-component input branch is input into the serial parallel multi-dimensional attention mechanism; wherein, the calculation process of the first Inception module of the electricity price sub-component input branch is as shown in the following formula: Where: F p is the Danish electricity price decomposition matrix, is the weight matrix of the i-th convolution kernel of the 1×1 convolution branch; They are the weight matrices of the i-th convolution kernel of the first layer 3×3, 5×5, and 7×7 convolutions respectively; They are the weight matrices of the i-th convolution kernel of the second layer 3×3, 5×5, and 7×7 convolutions respectively; is the bias of the i-th convolution kernel of the 1×1 convolution; They are the biases of the i-th convolution kernel of the first layer 3×3, 5×5, and 7×7 convolutions respectively; They are the biases of the i-th convolution kernel of the second layer 3×3, 5×5, and 7×7 convolutions respectively; Output feature of 1×1 convolution branch; They are the output features of the first layer 3×3, 5×5, and 7×7 convolutions respectively; are the output features of the second layer 3×3, 5×5, and 7×7 convolutions respectively; δ r is the Relu activation function; the feature matrices extracted from the four convolution branches are concat-connected to obtain the output multi-scale spatial feature matrix Similarly, the multi-scale spatial feature matrix output by the second Inception module in the electricity price subcomponent input branch and the Inception module in the electricity price related variable input branch is and 6. The method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism according to claim 4, characterized in that: The Sb includes: Sb1, multi-scale spatial feature matrix through global maximum pooling and global average pooling The spatial information of the image is compressed and aggregated to obtain a feature map with an output size of C×1×1. Two layers of 1×1 convolution are set for scaling learning and activated by the Sigmoid function to obtain the attention weight on the first dimension. The calculation process is shown in the following formula: Where: GMP and GAP are global maximum pooling and global average pooling operations; is the vector dot product; and is the weight of the i-th 1×1 convolution kernel in the first and second layers in the C dimension; and is the bias of the i-th 1×1 convolution kernel in the first and second layers; It is the feature output after global maximum pooling and global average pooling dot product processing; and f CA is the output after feature learning through the first and second layers of 1×1 convolution, σ is the Sigmoid activation function; Sb2, use two one-dimensional maximum pooling and one-dimensional average pooling to compress and aggregate the vertical spatial information and horizontal spatial information of the multi-scale spatial feature matrix respectively, and obtain the feature maps with output of C×H×1 and C×1×W. The calculation process is shown in the following formula: Where: and for The output of the jth channel in the H dimension after average pooling and maximum pooling; and for The output of the jth channel in dimension W after average pooling and maximum pooling; τ represents the maximum pooling and average pooling operations of different spatial ranges for each channel; Two layers of 1×1 convolution are set for scaling learning and activated by the Sigmoid function to obtain the vertical and horizontal attention weights. The calculation process is shown in the following formula: Where: and is the weight of the i-th 1×1 convolution kernel in the first and second layers in the H dimension; and is the bias of the i-th 1×1 convolution kernel in the first and second layers in the H dimension; and is the weight of the i-th 1×1 convolution kernel in the first and second layers in dimension W; and is the bias of the i-th 1×1 convolution kernel in the first and second layers in the W dimension; and f HA The output after learning the H-dimensional features through the first and second layers of 1×1 convolution; and f WA The output after learning the W-dimensional features through the first and second layers of 1×1 convolution; Sb3, multiply the attention weights obtained in different dimensions by the original multi-scale spatial feature matrix to obtain the multi-dimensional attention feature matrix f DA , as shown below:

7. The method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism according to claim 4, characterized in that: The Sc comprises: For the input branch of electricity price related variables, the Inception and BiLSTM parallel structures are used to simultaneously extract the spatial features and time series features of electricity price related variables; the input matrix of electricity price related variables is transformed into a 1-dimensional long vector through the flat layer and input into the BiLSTM module. The connection between historical data and future data is strengthened through the forward and reverse propagation laws, and the time series features of electricity price related variables are extracted. The calculation process is shown in the following formula; Where: and are the outputs of the forward hidden layer and the backward hidden layer at time t respectively; and are the weight matrices that map the input layer to the forward hidden layer and the backward hidden layer respectively; and are the weight matrices that map the output of the forward hidden layer and the backward hidden layer from the previous moment to the current moment; b (1) and b (2) are the biases of the forward hidden layer and the backward hidden layer respectively; W1 and W2 are the weight matrices mapping the output of the forward hidden layer and the backward hidden layer to the output layer respectively; δt is the Tanh activation function; It is the output of the electricity price related variable matrix after the flattening layer; Represents the output of the output layer; similarly, this time series feature f v,1 After inputting into the second BiLSTM network of this branch, the time series features are obtained as follows: Subsequent use represent The spatial features extracted by the Inception module The time series features extracted by the BiLSTM module after being flattened by the flat layer Perform feature fusion to obtain the spatiotemporal characteristics of electricity price related variable data f v , the calculation process is shown as follows: in, Represents concatenation from the first dimension.

8. The method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism according to claim 4, characterized in that: The Sd comprises: Through the self-attention mechanism, the different position information in the spatiotemporal characteristics of the electricity price-related variable data are associated and interacted with each other, and the dependencies therein are learned more comprehensively, focusing on the important information therein. The calculation process is shown in the following formula: Where: W i Q , W i K , W i V represents the initialized weight matrix, Q i , K i 、V i denote the query vector, key vector and value vector respectively, d k represents the dimension of the key vector, Z represents the weight coefficient, W 0 represents the weight matrix.

9. The method for predicting day-ahead electricity prices based on a parallel multi-dimensional attention mechanism according to claim 1, characterized in that: Each sample in the data set samples includes an electricity price sub-component, electricity price-related variables, and the true value of future electricity prices. After the electricity price sub-component input matrix and the electricity price-related variable input matrix are obtained, each sample finally includes the electricity price sub-component input matrix, the electricity price-related variable input matrix, and the true value of future electricity prices.

Images