Medium and long term power load prediction method based on integrated Transform

Abstract

The invention discloses a medium and long term power load prediction method based on an integrated Transform, and belongs to the technical field of power markets. Aiming at the problems that the existing method is difficult to capture complex space-time dependency in time sequence data, is difficult to adapt to a nonlinear mode caused by factors such as market fluctuation and cannot effectively integrate external factors such as economy, population and climate, the space-time dependency is effectively mined through a graph convolutional neural network embedding layer, and the adaptability is high; historical information can be selectively stored through the long-short-term memory network embedding layer, the long-term dependence problem is solved, and the method adapts to dynamic changes of time sequences; external information is effectively integrated through a time sequence multi-head attention layer, and the comprehensive processing capacity of the model on the external information is improved. According to the method, the space-time embedding layer, the long and short term embedding layer and the multi-head attention layer are designed, model construction is carried out from three dimensions of space-time dependence, a nonlinear mode and external information integration of a time sequence, and the accuracy and robustness of prediction are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power market, and specifically relates to a medium- and long-term power load forecasting method based on an integrated Transformer. Background Art

[0002] Power load forecasting is crucial for the efficient operation and accurate planning of power systems. It can provide decision support and reduce production costs. Although traditional time series forecasting methods have achieved results in many fields, they still face challenges in power load forecasting. The main problems include: the sharp fluctuations in the market and economy have increased data volatility, and existing methods are difficult to capture these complex changes; consumer behavior and the popularity of new energy have changed the seasonality and long-term trends of electricity demand, making traditional models such as the difference autoregressive moving average model (ARIMA) inflexible in dealing with the nonlinearity and mutations of long-term series; long-term forecasting needs to consider the complex interactions of factors such as economic growth, population and climate, and traditional methods fail to effectively integrate these external information. Therefore, it is necessary to use advanced technologies such as deep learning to improve the nonlinearity and complexity of time series forecasting.

[0003] Traditional medium- and long-term power load forecasting methods are mainly based on two categories: statistical learning and deep learning. Statistical methods, such as ARIMA and exponential smoothing (ETS), are simple and easy to understand, but they assume linear correlation and are difficult to adapt to multivariate sequences and nonlinear patterns, and lack adaptability. Deep learning methods, including convolutional neural networks (CNN), graph convolutional neural networks (GCN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and gated recurrent units (GRU), can capture nonlinearities and temporal patterns, but often ignore dependencies between sequences. The Transformer model captures long-term dependencies through a self-attention mechanism, but the model's interpretability is limited and it fails to fully utilize the relationships between sequences. More advanced medium- and long-term power load forecasting models are urgently needed to address the shortcomings of the above methods. Summary of the invention

[0004] In view of the limitations of the existing single data source method and the problem that the existing method is difficult to capture the complex spatiotemporal dependencies in time series data and difficult to adapt to the nonlinear changes in electricity price data, the present invention provides a medium- and long-term power load forecasting method based on an integrated Transformer.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A medium- and long-term power load forecasting method based on an integrated Transformer comprises the following steps:

[0007] Step 1: Collect and store historical power load data;

[0008] Step 2: Preprocess historical data and construct training sets and test sets;

[0009] Further, the step 2 specifically includes: based on the historical power load data, establishing an original data feature table including timestamps and related influencing factors, identifying and annotating auxiliary features, aggregating to form a summary data table, creating a processed summary data table through normalization processing, dividing data from the table to construct a training set and a test set, and providing structured input for model training and evaluation;

[0010] Step 3: Design a medium- and long-term power load forecasting model that includes a GCN spatiotemporal embedding layer, an LSTM long- and short-term embedding layer, and a time series multi-head attention layer; where:

[0011] GCN spatiotemporal embedding layer: uses graph convolution operations to capture the spatiotemporal features of time series;

[0012] LSTM long short-term embedding layer: uses long short-term memory units to capture the common evolution patterns and temporal dependencies of time series;

[0013] Time series multi-head attention: synchronously update and fuse the inputs of each embedding layer;

[0014] Furthermore, the GCN spatiotemporal embedding layer uses graph convolution operations to capture the spatiotemporal features of the time series. The specific operations are:

[0015] Step 3.1: Input X∈R L×M , taking the sequence at each time step as a node, learning the interdependence between time series and time series, and the hidden layer output H X ∈R L×F ;

[0016] Use formula (1) to obtain the calculation result X of the GCN attention module attention , use formula (2) to obtain the output node feature matrix X of the attention module A , use formula (3) to obtain the adjacency matrix A that needs to be input into the GCN architecture X , use formula (4) to obtain the input H of the l+1th layer of GCN l+1 ;

[0017] Wherein, formula (1) is expressed as:

[0018]

[0019] Where σ1 represents the sigmoid activation function, W x represents the corresponding weight matrix, b x represents the bias term; T represents the transposition operation;

[0020] Formula (2) is expressed as:

[0021] X A =X⊙X attention (2)

[0022] Where ⊙ represents element-by-element multiplication;

[0023] Formula (3) is expressed as:

[0024]

[0025] Where σ1 represents the sigmoid activation function, W A Represents the learnable weight parameters used to learn the adjacency matrix, which are randomly initialized using uniform distribution U(0,1);

[0026] Formula (4) is expressed as:

[0027]

[0028] Where σ2 represents the ReLU activation function, W l represents the learning weight, I is the identity matrix used to contain the node's own information. express The degree matrix of

[0029] From formula (1-3), we can see that the initial hidden state H 0 =X A , when the number of layers is determined, the final output of the hidden layer is determined according to formula (4), denoted as H X ;

[0030] Step 3.2: Enter X T ∈R M×L , taking each attribute as a node, learning the interdependence between attributes and attribute sequences, and transforming the input dimension according to formula (1-4) to obtain the output of this hidden layer

[0031] Step 3.3: Connect the hidden layer outputs of the two GCN parts to obtain the final output H of the GCN spatiotemporal embedding layer out ;

[0032] Use formula (5) to get the output H of the GCN part out ;

[0033] Among them, formula (5) is expressed as:

[0034]

[0035] In the formula, H outIndicates the state of fusion of space and time;

[0036] The LSTM long short-term embedding layer uses long short-term memory units to capture the common evolution pattern and time dependency of time series. The specific operations are as follows:

[0037] Step 3.4: Input X∈R L×M , taking the entire time series as input, learning the interaction between attributes, the co-evolution pattern between attributes over time, and the long-term dependency, according to formula (6-10), the comprehensive hidden layer representation H of this layer is obtained m ∈R L×M×d ;

[0038] For each time step t, we input an M-dimensional vector x (i) ∈R 1×L , use formula (6) to obtain the forget gate f of LSTM t , use formula (7) to obtain the input gate i of LSTM t , use formula (8) to obtain the output gate o of LSTM t , use formula (9) to obtain the memory unit c of LSTM t , use formula (10) to obtain the hidden state h of LSTM at time step t t ;

[0039] Among them, formula (6) is expressed as:

[0040] f t =σ1(W f x t +U f h t-1 +b f ) (6)

[0041] In the formula, σ1 represents the sigmoid activation function, x t represents the input at time step t, h t-1 represents the hidden state at time step t-1, W f and U f Respectively represent the weight matrix of the forget gate, b f Represents the bias of the forget gate;

[0042] Formula (7) is expressed as:

[0043] i t =σ1(W i x t +U i h t-1 +b i ) (7)

[0044] Where σ1 represents the sigmoid activation function, Wi and U i Represent the weight matrix of the input gate, b i represents the bias of the input gate;

[0045] Formula (8) is expressed as:

[0046] o t =σ1(W o x t +U o h t-1 +b o ) (8)

[0047] Where σ1 represents the sigmoid activation function, W o and U o Represent the weight matrix of the output gate, b o represents the bias of the output gate;

[0048] Formula (9) is expressed as:

[0049] c t =f t ⊙c t-1 +(i t ⊙φ(W c x t +U c h t-1 +b c )) (9)

[0050] Where φ represents the tanh function, ⊙ represents element-by-element multiplication, and W c and U c Represent the weight matrix of the memory unit, b c represents the bias of the memory unit;

[0051] Formula (10) is expressed as:

[0052] h t =o t ⊙φ(c t ) (10)

[0053] In the formula, o t represents the output gate at time step t, c t Represents the memory cell at time step t;

[0054] According to formula (6-9), the state of each key gate of LSTM is calculated. The last hidden layer representation H of this layer is obtained according to formula (10) m ∈R L×M×d ;

[0055] Step 3.5: Enter x (i) ∈R1×L , use a LSTM for each attribute to learn the temporal dynamic pattern of a single attribute and mine local short-term features. According to formula (6-10), after M attributes are processed by M LSTMs, the output H is obtained. s ∈R L ×M×d ;

[0056] Step 3.6: The outputs of the two LSTM parts are processed by the global average pooling layer and fused using the weighted summation method to obtain the final output H' of the LSTM long-term and short-term embedding layer. out ;

[0057] Use formula (11) to get the first output H of the LSTM part m The result H after the global average pooling layer m_pooled ; Use formula (12) to obtain the second output H of the LSTM part s The result H after the global average pooling layer s_pooled ; Use formula (13) to obtain the final output H' of the LSTM long-term and short-term embedding layer out ;

[0058] Among them, formula (11) is expressed as:

[0059]

[0060] Where d represents the hidden layer dimension; (:,:,i) represents extracting the i-th two-dimensional slice;

[0061] Formula (12) is expressed as:

[0062]

[0063] Formula (13) is expressed as:

[0064] H' out =W s H s_pooled ⊕W m H m_pooled (13)

[0065] Where W s +W m =1, indicating the weight of each of the two parts;

[0066] The time series multi-head attention synchronously updates and fuses the inputs of each embedding layer. The specific operations are as follows:

[0067] Step 3.7: Assume that there are h heads in the Transformer model and the input is H∈R L×M, for each head j (j=1,…,h), the input is projected into the query, key and value matrix through the conversion layer, and the attention layer calculates the attention between the query and the key and assigns it;

[0068] Use formula (14) to obtain the query, key and value matrix Q after projecting the input i , K i 、V i , use formula (15) to obtain the attention A between the query and the key i ;

[0069] Among them, formula (14) is expressed as:

[0070]

[0071] In the formula, and represents the corresponding weight matrix;

[0072] Formula (15) is expressed as:

[0073]

[0074] In the formula, σ3 represents the softmax function, and the factor Used to avoid gradient disappearance;

[0075] Step 3.8: Input the information of the GCN layer, LSTM layer and the original embedding representation into the time series multi-head attention, and calculate the attention of the three respectively according to formulas (14, 15) A H,i ;

[0076] Use formula (16) to obtain the attention A of the information represented by the fusion GCN layer and LSTM layer and the original embedding i ;

[0077] Wherein, formula (16) is expressed as:

[0078]

[0079] In the formula, α0, α out , are trainable parameters, which are used to capture various types of information of the three embedding layers in one layer at the same time;

[0080] Step 3.9: Connect the outputs of the h heads into a matrix By outputting the weight matrix Perform projection to complete the output of this layer;

[0081] Step 3.10: After completing the calculation work according to the above work, since the model is based on the classic architecture of Transformer, its attention calculation mechanism has been modified, two layers of GCN and LSTM have been added, and its input channel has been improved to capture more information. In order to facilitate the expansion of its performance, the subsequent encoding, decoding and other related training work uses the same components as the original Transformer.

[0082] Step 4: Use the training set to train the model, use the test set to evaluate the model performance, and obtain the optimized load forecasting model;

[0083] Step 5: Input the power load data to be predicted, and the model outputs the medium- and long-term power load forecast results.

[0084] Compared with the prior art, the present invention has the following advantages:

[0085] Different from the prior art, the medium- and long-term power load forecasting method based on integrated Transformer of the present invention constructs a graph structure of the time series through a spatiotemporal embedding layer based on a graph convolutional neural network, which can effectively mine the spatiotemporal dependencies; through an embedding layer based on a long short-term memory network, it can effectively mine the common evolution pattern and time dependency of a single time step and the overall sequence to adapt to nonlinear characteristics; through multi-head attention of the time series, it can synchronously update and fuse the information of the three embedding layers, effectively integrating external factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 This is an ET model framework diagram of a medium- and long-term power load forecasting method based on an integrated Transformer provided by the present invention.

[0087] Figure 2 It is a GCN spatiotemporal embedding layer graph of a medium- and long-term power load forecasting method based on an integrated Transformer provided by the present invention.

[0088] Figure 3 It is an LSTM long-term and short-term embedding layer graph of a medium- and long-term power load forecasting method based on an integrated Transformer provided by the present invention. DETAILED DESCRIPTION

[0089] In order to gain a deeper understanding of the present invention, we will provide a comprehensive and detailed description of the present invention. However, the present invention has multiple implementations and is not limited to the specific examples listed herein. The presentation of these examples is intended to deepen the comprehensive understanding of the disclosure of the present invention.

[0090] A medium- and long-term power load forecasting method based on an integrated Transformer comprises the following steps:

[0091] Step 1: Collect and store historical power load data;

[0092] Step 2: Preprocess historical data and construct training sets and test sets;

[0093] Further, the step 2 specifically includes: based on the historical power load data, establishing an original data feature table including timestamps and related influencing factors, identifying and annotating auxiliary features, aggregating to form a summary data table, creating a processed summary data table through normalization processing, dividing data from the table to construct a training set and a test set, and providing structured input for model training and evaluation;

[0094] Step 3: Design a medium- and long-term power load forecasting model that includes a GCN spatiotemporal embedding layer, an LSTM long- and short-term embedding layer, and a time series multi-head attention;

[0095] Furthermore, the medium- and long-term power load forecasting model described in step 3 includes: a GCN spatiotemporal embedding layer, an LSTM long- and short-term embedding layer, and a time series multi-head attention; wherein:

[0096] GCN spatiotemporal embedding layer: uses graph convolution operations to capture the spatiotemporal features of time series;

[0097] LSTM long short-term embedding layer: uses long short-term memory units to capture the common evolution patterns and temporal dependencies of time series;

[0098] Time series multi-head attention: synchronously update and fuse the inputs of each embedding layer;

[0099] Furthermore, the GCN spatiotemporal embedding layer uses graph convolution operations to capture the spatiotemporal features of the time series. The specific operations are:

[0100] Step 3.1: Input X∈R L×M , taking the sequence at each time step as a node, learning the interdependence between time series and time series, and the hidden layer output H X ∈R L×F ;

[0101] Use formula (1) to obtain the calculation result X of the GCN attention module attention , use formula (2) to obtain the output node feature matrix X of the attention module A , use formula (3) to obtain the adjacency matrix A that needs to be input into the GCN architecture X , use formula (4) to obtain the input H of the l+1th layer of GCN l+1 ;

[0102] Wherein, formula (1) is expressed as:

[0103]

[0104] Where σ1 represents the sigmoid activation function, W x represents the corresponding weight matrix, b x represents the bias term; T represents the transposition operation;

[0105] Formula (2) is expressed as:

[0106] X A =X⊙X attention (2)

[0107] Where ⊙ represents element-by-element multiplication;

[0108] Formula (3) is expressed as:

[0109]

[0110] Where σ1 represents the sigmoid activation function, W A Represents the learnable weight parameters used to learn the adjacency matrix, which are randomly initialized using uniform distribution U(0,1);

[0111] Formula (4) is expressed as:

[0112]

[0113] Where σ2 represents the ReLU activation function, W l represents the learning weight, I is the identity matrix used to contain the node's own information. express The degree matrix of

[0114] From formula (1-3), we can see that the initial hidden state H 0 =X A Once the number of layers is determined, the final output of the hidden layer can be determined according to formula (4), denoted as H X ;

[0115] Step 3.2: Enter X T ∈R M×L , taking each attribute as a node, learning the interdependence between attributes and attribute sequences, and transforming the input dimension according to formula (1-4) to obtain the output of this hidden layer

[0116] Step 3.3: Connect the hidden layer outputs of the two GCN parts to obtain the final output H of the GCN spatiotemporal embedding layer out ;

[0117] Use formula (5) to get the output H of the GCN part out ;

[0118] Among them, formula (5) is expressed as:

[0119]

[0120] In the formula, H out Indicates the state of fusion of space and time;

[0121] The LSTM long short-term embedding layer uses long short-term memory units to capture the common evolution pattern and time dependency of time series. The specific operations are as follows:

[0122] Step 3.4: Input X∈R L×M , taking the entire time series as input, learning the interaction between attributes, the co-evolution pattern between attributes over time, and the long-term dependency, according to formula (6-10), the comprehensive hidden layer representation H of this layer is obtained m ∈R L×M×d ;

[0123] For each time step t, we input an M-dimensional vector x (i) ∈R 1×L , use formula (6) to obtain the forget gate f of LSTM t , use formula (7) to obtain the input gate i of LSTM t , use formula (8) to obtain the output gate o of LSTM t , use formula (9) to obtain the memory unit c of LSTM t , use formula (10) to obtain the hidden state h of LSTM at time step t t ;

[0124] Among them, formula (6) is expressed as:

[0125] f t =σ1(W f x t +U f h t-1 +b f ) (6)

[0126] In the formula, σ1 represents the sigmoid activation function, x t represents the input at time step t, h t-1 represents the hidden state at time step t-1, W f and U f Respectively represent the weight matrix of the forget gate, b f represents the bias of the forget gate;

[0127] Formula (7) is expressed as:

[0128] i t =σ1(W i xt +U i h t-1 +b i ) (7)

[0129] Where σ1 represents the sigmoid activation function, W i and U i Represent the weight matrix of the input gate, b i represents the bias of the input gate;

[0130] Formula (8) is expressed as:

[0131] o t =σ1(W o x t +U o h t-1 +b o ) (8)

[0132] Where σ1 represents the sigmoid activation function, W o and U o Represent the weight matrix of the output gate, b o represents the bias of the output gate;

[0133] Formula (9) is expressed as:

[0134] c t =f t ⊙c t-1 +(i t ⊙φ(W c x t +U c h t-1 +b c )) (9)

[0135] Where φ represents the tanh function, ⊙ represents element-by-element multiplication, and W c and U c Represent the weight matrix of the memory unit, b c represents the bias of the memory unit;

[0136] Formula (10) is expressed as:

[0137] h t =o t ⊙φ(c t ) (10)

[0138] In the formula, o t represents the output gate at time step t, c t Represents the memory cell at time step t;

[0139] According to formula (6-9), the state of each key gate of LSTM can be calculated. According to formula (10), the last hidden layer representation H of this layer can be obtained. m ∈R L×M×d ;

[0140] Step 3.5: Enter x (i) ∈R 1×L , use a LSTM for each attribute to learn the temporal dynamic pattern of a single attribute and mine local short-term features. According to formula (6-10), after M attributes are processed by M LSTMs, the output H is obtained. s ∈R L ×M×d ;

[0141] Step 3.6: The outputs of the two LSTM parts are processed by the global average pooling layer and fused using the weighted summation method to obtain the final output H' of the LSTM long-term and short-term embedding layer. out ;

[0142] Use formula (11) to get the first output H of the LSTM part m The result H after the global average pooling layer m_pooled ; Use formula (12) to obtain the second output H of the LSTM part s The result H after the global average pooling layer s_pooled ; Use formula (13) to obtain the final output H' of the LSTM long-term and short-term embedding layer out ;

[0143] Among them, formula (11) is expressed as:

[0144]

[0145] Where d represents the hidden layer dimension; (:,:,i) represents extracting the i-th two-dimensional slice;

[0146] Formula (12) is expressed as:

[0147]

[0148] Formula (13) is expressed as:

[0149] H' out =W s H s_pooled ⊕W m H m_pooled (13)

[0150] Where W s +W m =1, indicating the weight of each of the two parts;

[0151] The time series multi-head attention synchronously updates and fuses the inputs of each embedding layer. The specific operations are as follows:

[0152] Step 3.7: Assume that there are h heads in the Transformer model and the input is H∈R L×M ,For each head j (j=1,…,h),, the input is projected into the query, key and value matrix through the conversion layer, and the attention layer calculates the attention between the query and the key and assigns it;

[0153] Use formula (14) to obtain the query, key and value matrix Q after projecting the input j , K j 、V j , use formula (15) to obtain the attention A between the query and the key j ;

[0154] Among them, formula (14) is expressed as:

[0155]

[0156] In the formula, and represents the corresponding weight matrix;

[0157] Formula (15) is expressed as:

[0158]

[0159] In the formula, σ3 represents the softmax function, and the factor Used to avoid gradient disappearance;

[0160] Step 3.8: Input the information of the GCN layer, LSTM layer and the original embedding representation into the time series multi-head attention, and calculate the attention of the three respectively according to formulas (14, 15) A H,j ;

[0161] Use formula (16) to obtain the attention A of the information represented by the fusion GCN layer and LSTM layer and the original embedding j ;

[0162] Wherein, formula (16) is expressed as:

[0163]

[0164] In the formula, α0, α out , are trainable parameters, which are used to capture various types of information of the three embedding layers in one layer at the same time;

[0165] Step 3.9: Connect the outputs of the h heads into a matrix By outputting the weight matrix Perform projection to complete the output of this layer;

[0166] Step 3.10: After completing the calculation work according to the above work, since the model is based on the classic architecture of Transformer, its attention calculation mechanism has been modified, two layers of GCN and LSTM have been added, and its input channel has been improved to capture more information. In order to facilitate the expansion of its performance, the subsequent encoding, decoding and other related training work uses the same components as the original Transformer.

[0167] Step 4: Use the training set to train the model, use the test set to evaluate the model performance, and obtain the optimized load forecasting model;

[0168] Step 5: Input the power load data to be predicted, and the model outputs the medium- and long-term prediction results.

[0169] Example 1

[0170] (1) Experimental data

[0171] Taking the ETTh1 dataset as an example, the medium- and long-term power load forecasting method based on the integrated Transformer is studied; the research data includes 17420 time steps and 7 attributes. The prediction method adopts the ET method (the present invention), Autoformer, Informer, Pyraformer and PatchTST methods, and the evaluation criteria are MSE and MAE;

[0172] (2) Experimental process

[0173] The following steps are used to forecast medium- and long-term power load:

[0174] Step S1: Collect and store historical data, download the public dataset ETTh1, and store it in the cloud server. The dataset contains 17420 time steps of power transformer data with a time granularity of 1 hour.

[0175] Step S2: Preprocess the historical data and construct the training set and test set. The training set / validation set / test set are divided into December / April / April respectively.

[0176] Step S3: using the medium- and long-term power load forecasting model of the ET method, Autoformer, Informer, Pyraformer and PatchTST method, training the model using the training set, evaluating the model performance using the test set, and obtaining the optimized load forecasting model;

[0177] Step S4: Input the power load data to be predicted, and the model outputs the evaluation results MSE and MAE.

[0178] Use formula (17) to calculate MSE:

[0179]

[0180] In formula (17), y represents the true value, represents the predicted value;

[0181] Use formula (18) to calculate MAE:

[0182]

[0183] The experimental results of this embodiment are shown in Table 1, which respectively test the prediction results using the ET method, Autoformer, Informer, Pyraformer and PatchTST methods.

[0184] Table 1 Comparison of evaluation results of different prediction methods on ETTh1 dataset

[0185]

[0186] Experiments show that the ET method has significantly improved accuracy compared with other benchmark methods on the ETTh1 dataset. In order to verify the overall effectiveness, five experiments used different medium- and long-term power load forecasting methods. From Table 1, it can be found that the ET method can more accurately capture the complexity of load changes, effectively improve the forecasting performance, and verify its superiority in power load forecasting.

[0187] Example 2

[0188] (1) Experimental data

[0189] Taking the ETTm1 dataset as an example, the medium- and long-term power load forecasting method based on the integrated Transformer is studied; the research data includes 69680 time steps and 7 attributes. The prediction methods adopt the ET method, Autoformer, Informer, Pyraformer and PatchTST method, and the evaluation criteria are MSE and MAE;

[0190] (2) Experimental process

[0191] The following steps are used to forecast medium- and long-term power load:

[0192] Step S1: Collect and store historical data, download the public dataset ETTm1, and store it in the cloud server. The dataset contains 69680 time steps of power transformer data with a time granularity of 15 minutes.

[0193] Step S2: Preprocess the historical data to construct the training set and test set. The training set / validation set / test set are divided into December / April / April respectively.

[0194] Step S3: using the medium- and long-term power load forecasting model of the ET method, Autoformer, Informer, Pyraformer and PatchTST method, training the model using the training set, evaluating the model performance using the test set, and obtaining the optimized load forecasting model;

[0195] Step S4: Input the power load data to be predicted, and the model outputs the evaluation results MSE and MAE.

[0196] Use formula (17) to calculate MSE;

[0197] Calculate MAE using formula (18);

[0198] The experimental results of this embodiment are shown in Table 2, which tests the prediction results using the ET method, Autoformer, Informer, Pyraformer and PatchTST methods respectively.

[0199] Table 2 Comparison of evaluation results of different prediction methods on the ETTm1 dataset

[0200]

[0201] Experiments show that the ET method has achieved significant improvement in accuracy compared with other benchmark methods on the ETTm1 dataset. In order to verify the overall effectiveness, five experiments used different medium- and long-term power load forecasting methods. From Table 2, it can be found that the ET method can more accurately simulate the dynamic changes of the load, thus achieving a breakthrough in forecasting performance, proving its high efficiency and reliability in medium- and long-term power load forecasting.

[0202] Example 3

[0203] (1) Experimental data

[0204] Taking the ECL dataset as an example, the medium- and long-term power load forecasting method based on the integrated Transformer is studied; the research data includes 26304 time steps and 321 attributes. The prediction methods adopt the ET method, Autoformer, Informer, Pyraformer and PatchTST method, and the evaluation criteria are MSE and MAE;

[0205] (2) Experimental process

[0206] The following steps are used to forecast medium- and long-term power load:

[0207] Step S1: Collect and store historical data, download the public dataset ECL, and store it in the cloud server. The dataset contains the power consumption load data of 321 customers in 26,304 time steps, in units of 15 minutes kW, and each column represents a customer.

[0208] Step S2: Preprocess the historical data and construct the training set and test set. The training set / validation set / test set are divided into 15 / 3 / 4 months.

[0209] Step S3: using the medium- and long-term power load forecasting model of the ET method, Autoformer, Informer, Pyraformer and PatchTST method, training the model using the training set, evaluating the model performance using the test set, and obtaining the optimized load forecasting model;

[0210] Step S4: Input the power load data to be predicted, and the model outputs the evaluation results MSE and MAE.

[0211] Use formula (17) to calculate MSE;

[0212] Calculate MAE using formula (18);

[0213] The experimental results of this embodiment are shown in Table 3, which respectively test the prediction results using the ET method, Autoformer, Informer, Pyraformer and PatchTST methods.

[0214] Table 3 Comparison of evaluation results of different prediction methods on the ECL dataset

[0215]

[0216]

[0217] Experiments show that the ET method has achieved significant improvement in accuracy compared with other benchmark methods on the ECL dataset. In order to verify the overall effectiveness, five experiments used different medium- and long-term power load forecasting methods. From Table 3, it can be found that the ET method has demonstrated its high accuracy in simulating dynamic changes in load and significantly improved the forecasting effect.

[0218] It can be found that the method of the present invention can improve the effect of medium- and long-term power load forecasting to a certain extent. At the same time, the experiment of this embodiment shows that the ET algorithm as a whole is 2.1% lower in MSE and 1.5% lower in MAE relative to the suboptimal results. Compared with the other three classic methods based on the Transformer architecture, ET performs better in all experimental results. Compared with the suboptimal method PatchTST, the ET algorithm shows a better performance in longer-term predictions, which further verifies that the ET algorithm can effectively combine the long-term characteristics of the LSTM embedding layer and has a significant performance improvement in long-term predictions.

[0219] Based on the above experimental results, the following conclusions can be drawn:

[0220] (1) Based on the spatiotemporal embedding layer of the graph convolutional neural network, the graph structure of the time series is constructed, which can effectively mine spatiotemporal dependencies;

[0221] (2) Based on the embedding layer of the long short-term memory network, the co-evolution patterns and temporal dependencies of a single time step and the entire sequence are effectively mined to adapt to nonlinear characteristics;

[0222] (3) Time series multi-head attention enables it to synchronously update and fuse the information of the three embedding layers and effectively integrate external factors.

[0223] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in the field. Although the illustrative specific embodiments of the present invention are described above to facilitate the understanding of the present invention by the technical personnel in the field, it should be clear that the present invention is not limited to the scope of the specific embodiments. For the ordinary technical personnel in the field, as long as various changes are within the spirit and scope of the present invention defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A medium- and long-term power load forecasting method based on an integrated Transformer, characterized in that: The following steps are involved: Step 1: Collect and store historical power load data; Step 2: Preprocess historical data and construct training sets and test sets; Step 3: Design a medium- and long-term power load forecasting model that includes a GCN spatiotemporal embedding layer, an LSTM long- and short-term embedding layer, and a time series multi-head attention; Step 4: Use the training set to train the model, use the test set to evaluate the model performance, and obtain the optimized load forecasting model; Step 5: Input the power load data to be predicted, and the model outputs the medium- and long-term power load forecast results.

2. According to claim 1, a medium- and long-term power load forecasting method based on an integrated Transformer is characterized in that: The step 2 preprocesses the historical data, and constructs the training set and the test set, specifically including: based on the historical power load data, establishing an original data feature table containing timestamps and related influencing factors, identifying and annotating auxiliary features, summarizing to form a summary data table, creating a processed summary data table through normalization processing, dividing the data from the table to construct the training set and the test set, and providing structured input for model training and evaluation.

3. According to claim 2, a medium- and long-term power load forecasting method based on an integrated Transformer is characterized in that: The step 3 designs a medium- and long-term power load forecasting model including a GCN spatiotemporal embedding layer, an LSTM long- and short-term embedding layer, and a time series multi-head attention; wherein: GCN spatiotemporal embedding layer: uses graph convolution operations to capture the spatiotemporal features of time series; LSTM long short-term embedding layer: uses long short-term memory units to capture the common evolution patterns and temporal dependencies of time series; Multi-head attention in time series: Synchronously updating and fusing inputs of various embedding layers.

4. According to claim 3, a medium- and long-term power load forecasting method based on an integrated Transformer is characterized in that: In step 3, the GCN spatiotemporal embedding layer uses graph convolution operations to capture the spatiotemporal features of the time series. The specific operations are: Step 3.1: Input X∈R L×M , taking the sequence at each time step as a node, learning the interdependence between time series, and the hidden layer output H X ∈R L×F ; Use formula (1) to obtain the calculation result X of the GCN attention module attention , use formula (2) to obtain the output node feature matrix X of the attention module A , use formula (3) to obtain the adjacency matrix A that needs to be input into the GCN architecture X , use formula (4) to obtain the input H of the l+1th layer of GCN l+1 ; Wherein, formula (1) is expressed as: Where σ1 represents the sigmoid activation function, W x represents the corresponding weight matrix, b x represents the bias term; T represents the transposition operation; Formula (2) is expressed as: X A =X⊙X attention (2) Where ⊙ represents element-by-element multiplication; Formula (3) is expressed as: Where σ1 represents the sigmoid activation function, W A Represents the learnable weight parameters used to learn the adjacency matrix, which are randomly initialized using uniform distribution U(0,1); Formula (4) is expressed as: Where σ2 represents the ReLU activation function, W l represents the learning weight, I is the identity matrix used to contain the node's own information. express The degree matrix of From formula (1-3), we know that the initial hidden state H 0 =X A , when the number of layers is determined, the final output of the hidden layer is determined according to formula (4), denoted as H X ; Step 3.2: Enter X T ∈R M×L , taking each attribute as a node, learning the interdependence between attributes and attribute sequences, and transforming the input dimension according to formula (1-4) to obtain the output of this hidden layer Step 3.3: Connect the hidden layer outputs of the two GCN parts to obtain the final output H of the GCN spatiotemporal embedding layer out ∈R L×M ; Use formula (5) to get the output H of the GCN part out ; Among them, formula (5) is expressed as: In the formula, H out Indicates the state of fusion of space and time.

5. According to claim 4, a medium- and long-term power load forecasting method based on integrated Transformer is characterized in that: In step 3, the LSTM long short-term embedding layer uses long short-term memory units to capture the common evolution pattern and time dependency of the time series. The specific operations are as follows: Step 3.4: Enter X T ∈R M×L , taking the entire time series as input, learning the interaction between attributes, the co-evolution pattern between attributes over time, and the long-term dependency, according to formula (6-10), the comprehensive hidden layer representation H of this layer is obtained m ∈R L ×M×d ; For each time step t, we input an M-dimensional vector x (i) ∈R 1×L , use formula (6) to obtain the forget gate f of LSTM t , use formula (7) to obtain the input gate i of LSTM t , use formula (8) to obtain the output gate o of LSTM t , use formula (9) to obtain the memory unit c of LSTM t , use formula (10) to obtain the hidden state h of LSTM at time step t t ; Among them, formula (6) is expressed as: f t =σ1(W f x t +U f h t-1 +b f ) (6) In the formula, σ1 represents the sigmoid activation function, x t represents the input at time step t, h t-1 represents the hidden state at time step t-1, W f and U f Respectively represent the weight matrix of the forget gate, b f Represents the bias of the forget gate; Formula (7) is expressed as: i t =σ1(W i x t +U i h t-1 +b i ) (7) Where σ1 represents the sigmoid activation function, W i and U i Represent the weight matrix of the input gate, b i represents the bias of the input gate; Formula (8) is expressed as: o t =σ1(W o x t +U o h t-1 +b o ) (8) Where σ1 represents the sigmoid activation function, W o and U o Represent the weight matrix of the output gate, b o represents the bias of the output gate; Formula (9) is expressed as: c t =f t ⊙c t-1 +(i t ⊙φ(W c x t +U c h t-1 +b c )) (9) Where φ represents the tanh function, ⊙ represents element-by-element multiplication, and W c and U c Represent the weight matrix of the memory unit, b c represents the bias of the memory unit; Formula (10) is expressed as: h t =o t ⊙φ(c t ) (10) In the formula, o t represents the output gate at time step t, c t Represents the memory cell at time step t; According to formula (6-9), the state of each key gate of LSTM is calculated. The last hidden layer representation H of this layer is obtained according to formula (10) m ∈R L×M×d ; Step 3.5: Enter x (i) ∈R 1×L , use a LSTM for each attribute to learn the temporal dynamic pattern of a single attribute and mine local short-term features. According to formula (6-10), after M attributes are processed by M LSTMs, the output H is obtained. s ∈R L ×M×d ; Step 3.6: The outputs of the two LSTM parts are processed by the global average pooling layer and fused using the weighted summation method to obtain the final output H' of the LSTM long-term and short-term embedding layer. out ; Use formula (11) to get the first output H of the LSTM part m The result H after the global average pooling layer m_pooled ; Use formula (12) to obtain the second output H of the LSTM part s The result H after the global average pooling layer s_pooled ; Use formula (13) to obtain the final output H' of the LSTM long-term and short-term embedding layer out ; Wherein, formula (11) is expressed as: Where d represents the hidden layer dimension; (:,:,i) represents extracting the i-th two-dimensional slice; Formula (12) is expressed as: Formula (13) is expressed as: Where W s +W m =1, indicating the weight of each of the two parts.

6. According to claim 5, a medium- and long-term power load forecasting method based on integrated Transformer is characterized in that: In step 3, the time series multi-head attention synchronously updates and fuses the inputs of each embedding layer. The specific operations are: Step 3.7: Assume that there are h heads in the Transformer model and the input is H∈R L×M , for each head j (j=1,...,h), the input is projected into the query, key and value matrix through the conversion layer, and the attention layer calculates the attention between the query and the key and assigns it; Use formula (14) to obtain the query, key and value matrix Q after projecting the input j , K j 、V j , use formula (15) to obtain the attention A between the query and the key j ; Among them, formula (14) is expressed as: In the formula, and represents the corresponding weight matrix; Formula (15) is expressed as: In the formula, σ3 represents the softmax function, and the factor Used to avoid gradient disappearance; Step 3.8: Input the information of the GCN layer, LSTM layer and the original embedding representation into the time series multi-head attention, and calculate the attention of the three respectively according to formulas (14, 15) A H,j ; Use formula (16) to obtain the attention A of the information represented by the fusion GCN layer and LSTM layer and the original embedding j ; Wherein, formula (16) is expressed as: In the formula, α0, α out , are trainable parameters, which are used to capture various types of information of the three embedding layers in one layer at the same time; Step 3.9: Connect the outputs of the h heads into a matrix By outputting the weight matrix Perform projection to complete the output of this layer; Step 3.10: After completing the calculations based on the above work, the components used in subsequent related training work are the same as the original Transformer.

Images