Transform-based multivariable fusion power price medium and short term prediction method

Abstract

The invention belongs to the technical field of electric power price prediction, and particularly relates to a multivariable fusion electric power price medium and short term prediction method based on Transform, which comprises the following steps of: collecting a data set; obtaining electric power price data and related external source variables of the current day, wherein the external source variables comprise an electric power load prediction value and a wind power generation prediction value; and data preprocessing. And preprocessing the collected data, wherein the preprocessing comprises missing value filling and data cleaning. And sequence data embedding is carried out, and each power price sequence is sequentially embedded into a time token and a variable token. And performing multi-step prediction, and outputting a prediction result. According to the method, the time sequence characteristics of the power price data and the exogenous variable characteristics influencing the price are integrated into the model, so that the model can extract the time sequence characteristics and the variable characteristics of the power price sequence at the same time, and the prediction precision of the power price is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electricity price prediction, and in particular relates to a Transformer-based multivariable fusion electricity price medium- and short-term prediction method. Background Art

[0002] As the complexity of global energy markets increases, the importance of electricity price forecasting has also increased significantly. Electricity prices have time dependence, such as the impact of electricity consumption on prices during holidays, and variable correlation, such as the influence of multiple factors such as power generation methods, power load capacity, climate conditions, policy changes, and market behavior, presenting complex and nonlinear electricity price dynamics. Identifying and refining these patterns from electricity price data can lead to better forecasting results. Accurate electricity price forecasting is not only crucial for energy suppliers and power companies, but also has a profound impact on the stability and economic benefits of the entire electricity market. Therefore, it is crucial to develop high-performance electricity forecasting models to simultaneously identify time dependence and variable correlation in electricity price data.

[0003] Deep learning (DL) and machine learning (ML) have achieved great success in natural language processing and time series forecasting. They also play a huge role in the field of electricity price forecasting. In terms of using traditional machine learning to achieve electricity price forecasting, methods such as ARIMA (arithmetic integrated moving average) and support vector regression (SVR) are widely used, but these methods cannot accurately capture the nonlinear relationship in the electricity price series and the complex interaction relationship between multiple variables. Therefore, more and more research and practical applications have begun to use deep learning methods to overcome these limitations.

[0004] Electricity price prediction methods based on deep learning can generally be divided into two categories: recurrent neural network (RNN)-based and Transformer-based methods. The former integrates multiple gating mechanisms and designs two successful variants, including long short-term memory network (LSTM) and gated recurrent unit (GRU). However, since RNN and its variants are difficult to learn long-distance dependencies, they are prone to gradient vanishing or gradient explosion problems during training. The latter, by using the self-attention mechanism, can flexibly learn the long- and short-term dependencies between various positions in the sequence, showing more efficient parallel processing and superior feature extraction capabilities. However, many current studies only consider the single time series factor of electricity price, without considering the influence of other exogenous variables related to electricity price, and without considering time dependency and multivariate correlation at the same time. Therefore, it is necessary to provide a Transformer-based method to simultaneously extract time dependency and multivariate correlation in electricity price series. Summary of the invention

[0005] In view of the technical problem that many existing studies only consider the single time series factor of electricity price, do not consider the impact of other exogenous variables related to electricity price, and do not consider time dependence and multivariate correlation at the same time, the present invention provides a multivariate fusion medium- and short-term electricity price prediction method based on Transformer to achieve accurate prediction of electricity price.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A Transformer-based multivariable fusion short-term and medium-term electricity price forecasting method includes the following steps:

[0008] S1. Data set collection: obtain the electricity price data of the day and related exogenous variables, including power load forecast value and wind power generation forecast value;

[0009] S2, data preprocessing: preprocess the collected data, including missing value filling and data cleaning; then, divide the data set into training set, validation set and test set, perform standardization, and finally perform de-standardization to restore the original data format;

[0010] S3, embedding sequence data; embedding the electricity price sequence in time dimension and variable dimension in turn, embedding them into time tokens and variable tokens respectively, and merging the two;

[0011] S4. Feature extraction: extracting time dependency and variable correlation feature information from electricity price series;

[0012] S5. Output the prediction results; finally, output the future multi-step sequence through linear projection mapping to achieve medium- and short-term prediction of electricity prices.

[0013] The method for collecting the data set in S1 is: obtaining electricity price data from the open power system OPSD, and each electricity price series includes two additional exogenous variables: day-ahead load forecast values ​​and day-ahead wind power generation forecast values.

[0014] The data preprocessing method in S2 is: for missing values, linear interpolation is used to fill them, and the data is standardized and de-standardized:

[0015]

[0016]

[0017]

[0018]

[0019] The original data is converted into standardized values ​​according to the mean μi and standard deviation σi. The standardized data will have zero mean and unit variance. After the prediction is completed, the same mean and standard deviation are used for inverse transformation to convert the prediction results back to the scale of the original data for evaluation. Finally, the training set, validation set, and test set are divided in a ratio of 7:1:2.

[0020] The method for embedding sequence data in S3 is:

[0021] S3.1, Time Embedding Module: In the time embedding module, the data of the same time step is embedded into a token, so that each token can represent the complete information of a time point, and each token has position information to indicate the time sequence; this module contains convolutional layers and position encoding layers, each convolutional layer is composed of a Causal Convolutional Conv1d causal convolutional layer, a LayerNorm1d normalization layer and a GELU activation function in sequence; the position encoding layer uses sine function and cosine function for calculation, and the calculation formula is as follows:

[0022]

[0023] S3.2. Variable embedding module: The variable embedding module generates variable tokens by transposing the time tokens, and embeds the entire time series of each variable into a token independently to focus on the global characteristics of the variables in the sequence; this module contains linear layers, each of which consists of a Linear linear layer and a LayerNorm1d normalization layer. Since the order of the sequence is implicitly stored in each variable token, position embedding is no longer required for variable embedding.

[0024] The method for extracting the time dependency and variable correlation feature information in the electricity price series in S4 is: the embedded vector obtained in S3 is sequentially passed through the time feature extraction module to extract the time dependency and the variable feature extraction module to extract the variable correlation, and finally the prediction result is obtained through full connection layer mapping.

[0025] The method of extracting time dependency through the time feature extraction module and extracting variable correlation through the variable feature extraction module is as follows:

[0026] S4.1, temporal feature extraction module: extract temporal features from the temporal embedding vector obtained in S3.1. This module consists of two layers: a multi-head attention mechanism layer and a fully connected feedforward neural network. The first layer is a multi-head attention mechanism combined with a causal mask, as shown in the following formula. The second layer is a fully connected feedforward network. The two layers use residual connections, and then perform layer normalization.

[0027]

[0028] S4.2, variable feature extraction module: The variable embedding vector obtained in S3.2 is used to extract variable features. This module consists of two layers, including a multi-head attention mechanism layer and a fully connected feedforward neural network. The first layer is a multi-head attention mechanism, and the second layer is a fully connected feedforward network. The two layers use residual connection and then perform layer normalization.

[0029] The prediction method is evaluated using mean absolute error (MAE), root mean square error (RMSE), symmetric mean percentage error (SMAPE) and correlation coefficient.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] Without modifying any native components of Transformer, the present invention first captures the complex dynamic relationship of the electricity price series by embedding data of the same time step into time tokens with causal convolution embedding and position embedding, and independently embedding the entire time series of each variable into different token variables. Secondly, in the feature extraction module, the attention mechanism and feedforward neural network are applied in the modules for extracting time features and extracting variable features respectively to extract time dependency and variable correlation. Finally, multi-step outputs are predicted by linear projection. The present invention re-examines the role of each core component of Transformer and successfully captures the time dependency and variable correlation in the original electricity price series. In the actual application of electricity price prediction, the use of the present invention can better extract the complex features in the electricity price series and improve the accuracy of electricity price prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0033] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0034] Figure 1 It is a schematic diagram of the process of the present invention;

[0035] Figure 2 It is a module diagram of the present invention;

[0036] Figure 3 The Transformer structure diagram for extracting time features and variable features of the present invention;

[0037] Figure 4 This is the attention mechanism diagram of the Transformer of the present invention;

[0038] Figure 5 It is the scaled dot product attention map in the Transformer attention mechanism of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. These descriptions are only to further illustrate the features and advantages of the present invention, rather than to limit the claims of the present invention. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0040] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0041] In this embodiment, if Figure 1 , Figure 2 As shown, the following steps are included:

[0042] Step 1, data set collection. Obtain the electricity price data of the day and related exogenous variables, including power load forecast and wind power generation forecast. Obtain electricity price data from the Open Power System (OPSD). Each electricity price series includes two additional exogenous variables: load day-ahead forecast and wind power generation day-ahead forecast.

[0043] Step 2: Data preprocessing. The collected data is preprocessed, including missing value filling and data cleaning. Next, the data set is divided into training set, validation set and test set, standardized, and finally de-standardized to restore the original data format.

[0044] For missing values, linear interpolation is used to fill them. In order to better train and converge the model, the data is standardized and de-standardized, and the formula is as follows.

[0045]

[0046]

[0047]

[0048]

[0049] That is, the original data is converted to standardized values ​​according to the mean μi and standard deviation σi. The standardized data will have zero mean and unit variance. After the prediction is completed, the same mean and standard deviation are used for inverse transformation to convert the prediction results back to the scale of the original data for evaluation. Finally, the training set, validation set, and test set are divided in a ratio of 7:1:2.

[0050] Step 3: embed the sequence data. The electricity price sequence is embedded in the time dimension and variable dimension in turn, embedded into time tokens and variable tokens respectively, and the two are fused.

[0051] Step 3-1: Time embedding module: In the time embedding module, the data of the same time step is embedded into a token, so that each token can represent the complete information of a time point, and each token has position information to indicate the time sequence. This module contains convolutional layers and position encoding layers. Each convolutional layer consists of a Causal Convolutional Conv1d causal convolutional layer, a LayerNorm1d normalization layer, and a GELU activation function in sequence. The position encoding layer uses sine and cosine functions for calculation, and the calculation formula is as follows.

[0052]

[0053] Step 3-2: Variable embedding module: The variable embedding module embeds the entire time series of each variable into a token independently by transposing the time token to generate the variable token, so as to focus on the global characteristics of the variable in the sequence. This module contains linear layers, each of which consists of a Linear linear layer and a LayerNorm1d normalization layer. Since the order of the sequence is implicitly stored in each variable token, position embedding is no longer required when embedding variables.

[0054] Step 4: Feature extraction. Figure 3 As shown, the embedding vector obtained in step 3 is sequentially passed through the time feature extraction module to extract time dependency and the variable feature extraction module to extract variable correlation, and finally the prediction result is obtained through full connection layer mapping.

[0055] Step 4-1: Extracting time feature module: In step 4-1, Figure 4As shown in the figure, the time embedding vector obtained in step 3-1 is used to extract the time feature. This module consists of two layers: a multi-head attention mechanism layer and a fully connected feedforward neural network. The first layer is a multi-head attention mechanism combined with a causal mask, and the formula is as follows. The second layer is a fully connected feedforward network. The two layers use residual connections, and then layer normalization is performed.

[0056]

[0057] Step 4-2: Extract variable feature module: In step 4-2, Figure 5 As shown in the figure, the variable embedding vector obtained in step 3-2 is used to extract variable features. This module consists of two layers, a multi-head attention mechanism layer and a fully connected feedforward neural network. The first layer is a multi-head attention mechanism, and the second layer is a fully connected feedforward network. The two layers are connected with residuals, and then layer normalization is performed.

[0058] Step 5: Output the prediction results. Finally, the future multi-step sequence is output through linear projection mapping to achieve medium- and short-term prediction of electricity prices.

[0059] The short- and medium-term price forecasting method of the electricity market based on Transformer extracting time features and variable features uses the cross entropy loss, and the formula is:

[0060]

[0061] The method is implemented using Pytorch and trained using the Adam optimizer. After multiple experiments to adjust the parameters, the present invention sets the learning rate to 0.0001, the batch size to 32, and each experiment runs 100 cycles, allowing early stopping when the effect is good.

[0062] The Transformer-based multivariable fusion medium- and short-term electricity price forecasting method proposed in the present invention has achieved good forecasting results in the recognized Nordic (Nord Pool) public electricity price data set. The forecasting results are shown in Table 1. The evaluation indicators include mean absolute error MAE, root mean square error RMSE, symmetric mean percentage error SMAPE and correlation coefficient, which proves that the Transformer-based multivariable fusion medium- and short-term electricity price forecasting method proposed in the present invention has good forecasting ability.

[0063] Table 1 Prediction results

[0064]

[0065] The present invention provides a new medium- and short-term prediction method for multivariable fusion electricity prices based on Transformer: without modifying any native components of Transformer, first, by embedding data of the same time step into time tokens with causal convolution embedding and position embedding, and independently embedding the entire time series of each variable into different token variables, the complex dynamic relationship of the electricity price series is captured. Secondly, in the feature extraction module, the attention mechanism and the feedforward neural network are applied in the modules for extracting time features and extracting variable features, respectively, to extract time dependency and variable correlation. Finally, multi-step outputs are predicted by linear projection. The present invention re-examines the role of each core component of Transformer and successfully captures the time dependency and variable correlation in the original electricity price series. In the actual application of electricity price prediction, the use of the present invention can better extract the complex features in the electricity price series and improve the accuracy of electricity price prediction.

[0066] Only the preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention, and various changes should be included in the protection scope of the present invention.

Claims

1. A medium- and short-term prediction method for electricity prices based on multivariable fusion of Transformer, characterized in that: The following steps are involved: S1. Data set collection: obtain the electricity price data of the day and related exogenous variables, including power load forecast value and wind power generation forecast value; S2, data preprocessing: preprocess the collected data, including missing value filling and data cleaning; then, divide the data set into training set, validation set and test set, perform standardization, and finally perform de-standardization to restore the original data format; S3, embedding sequence data; embedding the electricity price sequence in time dimension and variable dimension in turn, embedding them into time tokens and variable tokens respectively, and merging the two; S4, feature extraction: extracting time dependency and variable correlation feature information from electricity price series; S5. Output the prediction results; finally, output the future multi-step sequence through linear projection mapping to achieve medium- and short-term prediction of electricity prices.

2. According to claim 1, a method for medium- and short-term prediction of electricity prices based on multivariable fusion of Transformer is characterized in that: The method for collecting the data set in S1 is: obtaining electricity price data from the open power system OPSD, and each electricity price series includes two additional exogenous variables: day-ahead load forecast values ​​and day-ahead wind power generation forecast values.

3. According to claim 1, a method for medium- and short-term prediction of electricity prices based on multivariable fusion of Transformer is characterized in that: The data preprocessing method in S2 is: for missing values, linear interpolation is used to fill them, and the data is standardized and de-standardized: The original data is converted into standardized values ​​according to the mean μi and standard deviation σi. The standardized data will have zero mean and unit variance. After the prediction is completed, the same mean and standard deviation are used for inverse transformation to convert the prediction results back to the scale of the original data for evaluation. Finally, the training set, validation set, and test set are divided in a ratio of 7:1:

2.

4. According to claim 1, a method for medium- and short-term prediction of electricity prices based on multivariable fusion of Transformer is characterized in that: The method for embedding sequence data in S3 is: S3.1, Time Embedding Module: In the time embedding module, the data of the same time step is embedded into a token, so that each token can represent the complete information of a time point, and each token has position information to indicate the time sequence; this module contains convolutional layers and position encoding layers, each convolutional layer is composed of a Causal Convolutional Conv1d causal convolutional layer, a LayerNorm1d normalization layer and a GELU activation function in sequence; the position encoding layer uses sine function and cosine function for calculation, and the calculation formula is as follows: S3.2, variable embedding module: The variable embedding module generates variable tokens by transposing the time tokens, and embeds the entire time series of each variable into a token independently to focus on the global characteristics of the variables in the sequence; This module contains linear layers, each of which consists of a Linear linear layer and a LayerNorm1d normalization layer. Since the order of the sequence is implicitly stored in each variable token, position embedding is no longer required when embedding variables.

5. According to claim 1, a method for medium- and short-term prediction of electricity prices based on multivariable fusion of Transformer is characterized in that: The method for extracting the time dependency and variable correlation feature information in the electricity price series in S4 is: the embedded vector obtained in S3 is sequentially passed through the time feature extraction module to extract the time dependency and the variable feature extraction module to extract the variable correlation, and finally the prediction result is obtained through full connection layer mapping.

6. According to claim 5, a method for medium- and short-term prediction of electricity prices based on multivariable fusion of Transformer is characterized in that: The method of extracting time dependency through the time feature extraction module and extracting variable correlation through the variable feature extraction module is as follows: S4.1, temporal feature extraction module: extract temporal features from the temporal embedding vector obtained in S3.

1. This module consists of two layers: a multi-head attention mechanism layer and a fully connected feedforward neural network. The first layer is a multi-head attention mechanism combined with a causal mask, as shown in the following formula. The second layer is a fully connected feedforward network. The two layers use residual connections, and then perform layer normalization. S4.2, variable feature extraction module: The variable embedding vector obtained in S3.2 is used to extract variable features. This module consists of two layers, including a multi-head attention mechanism layer and a fully connected feedforward neural network. The first layer is a multi-head attention mechanism, and the second layer is a fully connected feedforward network. The two layers use residual connection and then perform layer normalization.

7. The method for medium- and short-term prediction of electricity prices based on Transformer multivariable fusion according to claim 1 is characterized in that: The prediction method is evaluated using mean absolute error (MAE), root mean square error (RMSE), symmetric mean percentage error (SMAPE) and correlation coefficient.

Images