Multi-task short-term power load forecasting method based on attention time series decomposition

Abstract

The application discloses a kind of based on attention time series decomposition multi-task short-term power load prediction method, first, the historical power load from different regions is converted into corresponding space-time diagram representation, then the space-time diagram representation is decomposed into trend component, seasonal component and residual component, and each component obtained by decomposition is weighted, each component after weighting is spliced into weighted power load data, and time feature is captured to the weighted power load data by gate time convolution, finally, by multi-task graph convolution, non-euclidean space feature is extracted Linear transformation, obtain prediction result.The application uses attention mechanism to weight the components decomposed from power load data by STL, to help the model focus on more important components, so as to more effectively capture time features.Power load data is divided into different tasks for multi-task learning, to enhance the generalization ability of the model.

Description

Technical Field

[0001] This application belongs to the field of power load forecasting technology, and in particular relates to a multi-task short-term power load forecasting method based on attention time series decomposition. Background Technology

[0002] Accurate short-term load forecasting (STLF) is crucial for the safe and efficient operation of modern power systems. On one hand, accurate STLF helps grid operators prevent equipment damage or grid failures caused by sudden increases in power load. On the other hand, it also helps power plants scientifically plan their power generation to improve efficiency and reduce operating costs.

[0003] Short-term power load forecasting methods are mainly divided into three categories: statistical methods, machine learning-based methods, and deep learning-based methods. Statistical methods, such as Autoregressive Models (AR), Autoregressive Moving Average Models (ARMA), and Autoregressive Integrated Moving Average Models (ARIMA), have advantages such as simplicity, strong interpretability, and ease of implementation, but they struggle to capture nonlinear features. Machine learning methods, such as Support Vector Regression Machines (SVR), Multilayer Perceptrons (MLP), and Feedforward Neural Networks (FNN), excel in handling nonlinear relationships, but struggle with complex functions and feature engineering. Deep learning methods, such as Gated Recurrent Units (GRU), Convolutional Neural Networks (CNN), Long Short-Term Memory Networks (LSTM), and Temporal Convolutional Networks (TCN), possess powerful feature extraction capabilities and have been widely applied in power load forecasting over the past decade. However, while deep learning-based methods have achieved some success in short-term load forecasting, they still face significant challenges in capturing the deep temporal and spatial characteristics of power load data.

[0004] STL (Seasonal and Trend decomposition using Loess) is an algorithm for time series decomposition that breaks down time series data into trend, seasonal, and residual components. Each component provides corresponding features for deep learning methods, helping to extract complex temporal information from electricity load data. However, current research neglects the weight differences between the different components after STL decomposition when extracting temporal features, thus failing to fully extract deep-level temporal features.

[0005] Graph Convolutional Networks (GCNs) are neural networks that process non-Euclidean spatial relationships and are commonly used to extract spatial features from electricity load data. Assuming each network node represents a set of electricity load data, GCNs can aggregate information from neighboring nodes and update the state of the current node after a nonlinear transformation. However, as the number of nodes increases, the number of edges and the size of the adjacency matrix also increase, leading to overly complex spatial patterns in the electricity load data. This increases the risk of overfitting and reduces the generalization ability of GCNs. Summary of the Invention

[0006] The purpose of this application is to provide a multi-task short-term power load forecasting method based on attention time series decomposition to overcome the technical problems pointed out in the background art.

[0007] To achieve the above objectives, the technical solution of this application is as follows: A multi-task short-term power load forecasting method based on attention time series decomposition includes: Construct and train a prediction network model, which includes a spatiotemporal embedding module, an attention time-series decomposition module, a gated temporal convolution module, and a multi-task graph convolution module; In the spatiotemporal embedding module, historical power loads from different regions are transformed into corresponding spatiotemporal map representations through spatiotemporal embedding. In the attention time series decomposition module, the spatiotemporal graph representation is decomposed into trend components, seasonal components, and residual components. The decomposed components are weighted and then concatenated to form weighted power load data. In the gated temporal convolution module, time features are captured on the weighted power load data to obtain a feature representation that includes time features; The feature representation containing temporal features is input into the multi-task graph convolution module. The graph convolution network based on the learnable adjacency matrix extracts non-Euclidean spatial features to obtain a feature representation containing spatial features. Then, a linear transformation is performed on the feature representation containing spatial features to obtain the prediction result.

[0008] Furthermore, the process of decomposing the spatiotemporal graph representation into trend components, seasonal components, and residual components, and then weighting each component to obtain weighted power load data includes: The spatiotemporal graph representation is decomposed into trend components, seasonal components, and residual components using the STL time series decomposition method. Global average pooling is used to compress the feature map of each component into a scalar; The gating mechanism is used to process the scalar corresponding to each component to obtain the weight of each component; Each component is multiplied by a weight and then concatenated to obtain the weighted power load data.

[0009] Furthermore, the step of using a gating mechanism to process the scalar corresponding to each component and obtain the weight of each component includes: in, It is a scalar generated from the trend component. It is a scalar generated from seasonal components. It is a scalar generated from the residual components. It is the weight of the trend component. It is the weight of the seasonal component. It is the weight of the residual components. and It is a learnable weight matrix. It is the ReLU function. It is the Sigmoid function.

[0010] Furthermore, the linear transformation of the feature representation containing spatial features is performed to obtain the prediction result, and the following operations are executed: in, It is a prediction result. Represents the learnable weight matrix. It's a bias. This represents a feature representation that includes spatial characteristics.

[0011] Furthermore, the multi-task short-term power load forecasting method based on attention time series decomposition also includes: When training the prediction network model, the spatiotemporal graph representation after spatiotemporal embedding is divided into different clusters. Then, the prediction results are divided into cluster prediction results corresponding to each cluster according to the number of clusters. The average loss corresponding to each cluster is calculated, and the parameters of the prediction network model are optimized to obtain the trained prediction network model.

[0012] Furthermore, the formula for calculating the average loss is as follows: in, It is the j-th cluster that has been divided. It refers to the number of clusters. This is the cluster prediction result corresponding to the j-th cluster. It is the average loss.

[0013] This application proposes a multi-task short-term power load forecasting method based on attention time series decomposition, which has the following technical advantages: One approach proposes an ASTL method that combines an attention mechanism with STL, which uses an attention mechanism to weight the components decomposed from the power load data by STL to help the model focus on the more important components, thereby capturing temporal features more effectively.

[0014] On another front, a multi-task graph convolutional network (MG) that combines multi-task learning with graph convolutional networks is proposed. This involves dividing the power load data into different clusters for multi-task learning, which helps to learn simple spatial patterns with fewer nodes, thereby enhancing the model's generalization ability. Attached Figure Description

[0015] Figure 1 This is a flowchart of the multi-task short-term power load forecasting method based on attention time series decomposition in this application.

[0016] Figure 2 This is a schematic diagram of the prediction network model structure in this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] One embodiment of this application, such as Figure 1 As shown, a multi-task short-term power load forecasting method based on attention time series decomposition is provided, including: Step S1: Construct and train a prediction network model, which includes a spatiotemporal embedding module, an attention time series decomposition module, a gated temporal convolution module, and a multi-task graph convolution module.

[0019] This embodiment constructs and trains a prediction network model (abbreviated as: MG-ASTL), the framework of which is as follows: Figure 2As shown, the MG-ASTL model comprises a spatiotemporal embedding module, an attention-based time series decomposition module (ASTL), a gated temporal convolution module (gated TCN), and a multi-task graph convolution module (MG). First, the spatiotemporal embedding module converts historical electricity load data from different regions into corresponding spatiotemporal graphs. Next, the ASTL module uses STL to decompose the electricity load data into three components: a trend component, a seasonal component, and a residual component, and weights them using an attention mechanism. Then, the gated TCN module further extracts temporal features. Finally, the MG module divides the electricity load data into different clusters for multi-task learning and extracts spatial features using graph convolutional convolution (GCN).

[0020] Step S2: In the spatiotemporal embedding module, historical power loads from different regions are converted into corresponding spatiotemporal map representations through spatiotemporal embedding.

[0021] Spatiotemporal embedding is the foundation of spatiotemporal prediction models based on graph neural networks (GNNs) and is a relatively mature technology in this field. In this embodiment, to embed power load data from different regions into a graph, it is assumed that each node represents a region, and historical power loads from different regions can be stitched together to form a corresponding spatiotemporal graph representation. ,in Indicates time Historical power load characteristic matrix of different regions For characteristic number, For the current time, For the number of regions, For historical time steps.

[0022] Step S3: In the attention time series decomposition module, the spatiotemporal graph representation is decomposed into trend components, seasonal components, and residual components. The decomposed components are then weighted and concatenated to form weighted power load data.

[0023] STL (Seasonal-Trend decomposition procedure based on Loess) is a data preprocessing method that decomposes electricity load data into three components: trend component, seasonal component, and residual component. Generally, the different components have varying degrees of importance for extracting temporal features. Therefore, in this embodiment, the ATL module uses an attention mechanism to weight the components decomposed by STL, thereby improving STL's ability to capture temporal features.

[0024] First, the spatiotemporal graph identifier is decomposed into three components using the STL time series decomposition method. This process is shown in equation (1):

[0025] (1) in, A spatiotemporal representation of historical electricity loads from different regions. It is a trend component. It is a seasonal portion. It is the residual component. It is an STL operation.

[0026] Next, global average pooling is used to compress the feature map of each component into a scalar, which represents the global distribution of features among the components. This process is shown in Equation (2-4):

[0027] (2) (3) (4) in, It is a scalar generated from the trend component. It is a scalar generated from seasonal components. It is a scalar generated from the residual components. and It is an index. It is the number of regions. This is the input window.

[0028] Then, a gating mechanism is used to process the scalar corresponding to each component to obtain the weight of each component. This process is shown in equation (5-7).

[0029] (5) (6) (7) in, It is the weight of the trend component. It is the weight of the seasonal component. It is the weight of the residual components. and It is a learnable weight matrix. It is the ReLU function. It is the Sigmoid function.

[0030] Finally, each component is multiplied by a weight, allowing the MG-ASTL model to focus on the more important components for more efficient temporal feature extraction. This process is illustrated in Equation (8-11).

[0031] (8) (9) (10) (11) in, It is a weighted trend component. It is a weighted seasonal component. It is a weighted residual component. This represents weighted electricity load data. It's a splicing operation.

[0032] This embodiment achieves attention-based time series decomposition by weighting each component after decomposition. Different components have different degrees of importance for time feature extraction. In this embodiment, the ASTL module uses an attention mechanism to weight the components after STL decomposition to improve the ability of STL to capture time features.

[0033] Step S4: In the gated temporal convolution module, time feature capture is performed on the weighted power load data to obtain a feature representation containing time features.

[0034] The gated TCN module further captures temporal features using TCN and employs a gating mechanism to control the information flow to avoid gradient vanishing or gradient explosion. This process is illustrated in equations (12-17). Equation (12) calculates the input gate; equation (13) calculates the forget gate; equation (14) calculates the output gate; equation (15) calculates the candidate cell state; and equation (16) uses the forget gate and input gate to control the flow. Cell state at any given time and The information flow of candidate cell states at time t, obtained Cell state at any given time; Equation (17) uses an output gate for control. The flow of information about the cell state at any given moment is obtained. The hidden state at each moment. Through iterative calculations along the time dimension, the gated TCN module ultimately obtains a feature representation that includes temporal features. .

[0035] (12) (13) (14) (15) (16) (17) in, express Time input gate, express The Gate of Forgetting Time express The output gate at any time, express The state of candidate cells at any given time. express Cellular state at any given moment express The representation of time-related features. express Weighted power load data at any given time; , , and It's a deviation. It is a time step. It is the Sigmoid function. It's the Tanh function. It's a TCN operation. It is Hadamaji.

[0036] Step S5: Input the feature representation containing temporal features into the multi-task graph convolution module. Extract non-Euclidean spatial features based on the graph convolution network with learnable adjacency matrix to obtain the feature representation containing spatial features. Then, perform a linear transformation on the feature representation containing spatial features to obtain the prediction result.

[0037] In this embodiment, the feature representation containing temporal features obtained by the gated temporal convolution module is... The input is fed into the MG module. In the MG module, a graph convolutional network based on a learnable adjacency matrix is ​​used to extract non-Euclidean spatial features. The process is shown in equations (18-19).

[0038] (18) (19) in, This represents a feature representation that includes time characteristics. This represents a feature representation that includes spatial characteristics. Represents a learnable adjacency matrix. Represents the degree matrix of the diagonal nodes. Represents the identity matrix. Represents the learnable weight matrix. Represents the node embedding matrix. It is the ReLU function. It is the SoftMax function.

[0039] Then, the corresponding prediction results are generated through linear transformation. This process is shown in equation (20).

[0040] (20) in, It is a prediction result. Represents the learnable weight matrix. It is a bias.

[0041] In another embodiment of this application, graph convolutional networks are able to extract non-Euclidean spatial features from power load data. However, as the number of nodes increases, the generalization ability of graph convolutional networks decreases. Therefore, this embodiment of a multi-task short-term power load forecasting method based on attention time series decomposition further includes:

[0042] When training the prediction network model, the spatiotemporal graph representation after spatiotemporal embedding is divided into different clusters. Then, the prediction results are divided into cluster prediction results corresponding to each cluster according to the number of clusters. The average loss corresponding to each cluster is calculated, and the parameters of the prediction network model are optimized to obtain the trained prediction network model.

[0043] Specifically, this embodiment uses a multi-task learning approach to train the network model, thereby learning simple spatial patterns with fewer nodes to enhance the generalization ability of the graph convolutional network. For example, OPTICS (Ordering points to identify the clustering structure) is used to divide the spatiotemporal map identifiers corresponding to power load data in different regions into different clusters, and the prediction results are divided according to the number of clusters. The power load data and cluster prediction results of each cluster generate a task for multi-task learning. Then, the loss value of each task is calculated using the mean squared error loss function, and the average loss on all tasks is obtained by summing and averaging. Finally, the network parameters are optimized using the Adam algorithm to obtain a model that performs well on all tasks. This process is shown in equations (21-23).

[0044] (twenty one) (twenty two) (twenty three) in, It is the j-th cluster that has been divided. A spatiotemporal representation of historical electricity loads from different regions. It refers to the number of clusters. It's an OPTICS operation. This is the cluster prediction result corresponding to the j-th cluster. It is the average loss. It is an index.

[0045] This embodiment divides the power load data into different clusters for multi-task learning to improve the generalization ability of the graph convolutional network.

[0046] To verify the above technical solution, this application also provides experimental results. Two real-world power datasets, GEFCom2012 and GEFCom 2017, were used in the experiments to verify the effectiveness of the MG-ASTL model. The GEFCom2012 dataset contains 39,576 energy data records from 20 regions in the United States from January 1, 2004 to June 30, 2008. The GEFCom2017 dataset contains 61,344 energy data records from 500 regions in the United States from January 1, 2005 to December 31, 2011. This application will select a portion of the power load data from these two datasets for the experiment.

[0047] In the experiment, the MG-ASTL model was compared with the following baseline models to verify the effectiveness of the MG-ASTL model in short-term power load forecasting: GRU: A recurrent neural network based on a dual-gated mechanism; LSTM: Recurrent Neural Network Based on Triple Gating Mechanism; CNN-LSTM: A hybrid model that combines CNN and LSTM; Graph WaveNet: A hybrid model combining diffusing convolution and adaptive graphs; CNN-BiLSTM: A hybrid model that combines CNN and bidirectional LSTM; GCN-LSTM: A hybrid model that combines GCN and LSTM.

[0048] To evaluate the MG-ASTL model from multiple perspectives, the experiment used three evaluation metrics: mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE).

[0049] In the experiments, the input window size was set to 24, and the output window sizes were set to 3, 7, and 12. Regions selected on the GEFCom2012 dataset were set to 10, 15, and 20, and regions selected on the GEFCom2017 dataset were set to 20, 60, and 80. Weight decay was set to 0.0001, the learning rate to 0.001, the training epochs to 100, the batch size to 64, the loss function to MSE, the node embedding matrix depth to 10, the optimizer to Adam, the number of GCN layers to 2, and the hidden size to 64. For fairness, all baseline model hyperparameter values ​​were derived from their original references.

[0050] The experiments compared the performance of the MG-ASTL model with the baseline model under different conditions on two datasets. Tables 1 and 2 show the average results obtained by repeating the experiment five times for each model.

[0051] Table 1 Table 2 Table 1 compares the performance of the MG-ASTL model and the baseline model on the GEFCom2012 dataset, and Table 2 compares their performance on the GEFCom2017 dataset. From Tables 1 and 2, the following conclusions can be drawn: (1) MG-ASTL outperforms most baseline models on both datasets, validating the effectiveness of MG-ASTL.

[0052] (2) As the number of nodes increases, the performance of GRU and LSTM on both datasets deteriorates, indicating that GRU and LSTM are inadequate in extracting spatial features.

[0053] (3) Compared with CNN-LSTM and CNN-BiLSTM, which use CNN to capture Euclidean spatial features, GraphWaveNet and GCN-LSTM perform better on both datasets, indicating that non-Euclidean spatial features are more effective than Euclidean spatial features in short-term power load forecasting.

[0054] (4) As the number of nodes increases, MG-ASTL performs better and better on both datasets, indicating that MG-ASTL can still maintain good spatial feature extraction ability even when the number of nodes increases.

[0055] (5) When the output window is increased from 3 hours to 12 hours, MG-ASTL still performs well on both datasets, indicating that MG-ASTL has good robustness.

[0056] Simultaneously, ablation experiments were conducted to compare the performance of MG-ASTL and its different variant models to verify the necessity of each module. The variant models are as follows:

[0057] Variant 1: Removes MG-ASTL from the ASTL module, i.e., does not use weighted components as additional features; Variant 2: Removed DBSCAN's MG-ASTL, meaning that OPTICS is not used to divide the power load data into different clusters; Variant 3: Removes the MG-ASTL of the gated TCN module, i.e., does not use gated TCN to capture temporal features; Variant 4: Removes MG-ASTL from the MG module, meaning the MG module is not used to capture spatial features.

[0058] The performance of MG-ASTL and its variant models was compared under different conditions on two datasets. Tables 3 and 4 show the average results after five repetitions of each model.

[0059] Table 3 Table 4 Table 3 compares the performance of the MG-ASTL model and its variant on the GEFCom2012 dataset, and Table 4 compares their performance on the GEFCom2017 dataset. From Tables 3 and 4, the following conclusions can be drawn: (1) Variant 1 performed worse than MG-ASTL on both datasets, indicating that the ASTL module allows the model to focus on more important components to effectively extract temporal features.

[0060] (2) Variant 2 performed worse than MG-ASTL on both datasets, indicating that the MG module can use OPTICS to capture spatial features and enhance generalization ability.

[0061] (3) Variant 3 performed worse than MG-ASTL on both datasets, indicating that time characteristics are very important for MG-ASTL to predict short-term power load.

[0062] (4) Variant 4 performed worse than MG-ASTL on both datasets, indicating that spatial features are very important for MG-ASTL to predict short-term power load.

[0063] This application proposes an MG-ASTL model for short-term power load forecasting, which employs both ASL and MG to capture temporal and spatial features. Specifically, ASL uses an attention mechanism to weight different components, allowing MG-ASTL to focus on more important components and thus more effectively capture temporal features. MG uses DBSCAN to divide the power load data into different clusters, which helps MG-ASTL learn simple spatial patterns, thereby improving its generalization ability. Extensive comparative experiments demonstrate that MG-ASTL outperforms the baseline model.

[0064] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A multi-task short-term power load forecasting method based on attention time series decomposition, characterized in that, The multi-task short-term power load forecasting method based on attention time series decomposition includes: Construct and train a prediction network model, which includes a spatiotemporal embedding module, an attention time-series decomposition module, a gated temporal convolution module, and a multi-task graph convolution module; In the spatiotemporal embedding module, historical power loads from different regions are transformed into corresponding spatiotemporal map representations through spatiotemporal embedding. In the attention time series decomposition module, the spatiotemporal graph representation is decomposed into trend components, seasonal components, and residual components. The decomposed components are weighted and then concatenated to form weighted power load data. In the gated temporal convolution module, time features are captured on the weighted power load data to obtain a feature representation that includes time features; The feature representation containing temporal features is input into the multi-task graph convolution module. The graph convolution network based on the learnable adjacency matrix extracts non-Euclidean spatial features to obtain a feature representation containing spatial features. Then, a linear transformation is performed on the feature representation containing spatial features to obtain the prediction result. The process of decomposing the spatiotemporal graph representation into trend components, seasonal components, and residual components, weighting each component, and concatenating the weighted components into weighted power load data includes: The spatiotemporal graph representation is decomposed into trend components, seasonal components, and residual components using the STL time series decomposition method. Global average pooling is used to compress the feature map of each component into a scalar; The gating mechanism is used to process the scalar corresponding to each component to obtain the weight of each component; Each component is multiplied by a weight and then concatenated to obtain the weighted power load data; The process of using a gating mechanism to process the scalar corresponding to each component and obtain the weight of each component includes: ； ； ； in, It is a scalar generated from the trend component. It is a scalar generated from seasonal components. It is a scalar generated from the residual components. It is the weight of the trend component. It is the weight of the seasonal component. It is the weight of the residual components. and It is a learnable weight matrix. It is the ReLU function. It is the Sigmoid function.

2. The multi-task short-term power load forecasting method based on attention time series decomposition as described in claim 1, characterized in that, The linear transformation of the feature representation containing spatial features is performed to obtain the prediction result, and the following operations are executed: ； in, It is a prediction result. Represents the learnable weight matrix. It's a bias. This represents a feature representation that includes spatial characteristics.

3. The multi-task short-term power load forecasting method based on attention time series decomposition as described in claim 1, characterized in that, The multi-task short-term power load forecasting method based on attention time series decomposition further includes: When training the prediction network model, the spatiotemporal graph representation after spatiotemporal embedding is divided into different clusters. Then, the prediction results are divided into cluster prediction results corresponding to each cluster according to the number of clusters. The average loss corresponding to each cluster is calculated, and the parameters of the prediction network model are optimized to obtain the trained prediction network model.

4. The multi-task short-term power load forecasting method based on attention time series decomposition as described in claim 3, characterized in that, The formula for calculating the average loss is as follows: ； in, It is the j-th cluster that has been divided. It is the number of clusters. It is the cluster prediction result corresponding to the j-th cluster. It is the average loss.

Images