Short-term power load prediction method and device based on space-time neural network model

Abstract

The invention discloses a short-term power load prediction method and device based on a space-time neural network model, and the method employs a novel space-time neural network model, the space-time neural network model comprises a preset number of adaptive space-time feature extraction modules, and in the adaptive space-time feature extraction modules, the space-time feature extraction modules are used for extracting the space-time features. Spatial features and temporal features are extracted through self-adaptive graph convolution branches and gating circulation unit branches respectively, and spatial-temporal features are obtained through fusion. Through a preset number of self-adaptive spatial-temporal feature extraction modules, after the features output by the last self-adaptive spatial-temporal feature extraction module and the input features are subjected to residual connection, linear transformation is carried out, and a prediction result is obtained. According to the model provided by the invention, potential non-Euclidean spatial features can be extracted by using an improved learnable adjacency matrix, and a predetermined adjacency matrix does not need to be used as priori knowledge, so that the precision of short-term power load prediction can be further improved.

Description

Technical Field

[0001] The present application belongs to the technical field of power load forecasting, and in particular, relates to a short-term power load forecasting method and device based on a spatiotemporal neural network model. Background Art

[0002] Electricity load forecasting is crucial to the power system and can provide valuable forecasting information for power management departments, power suppliers and distribution companies. Inaccurate electricity load forecasting may lead to an imbalance in electricity supply and demand, resulting in huge economic losses to all parties involved in the power system. Currently, the power grid faces a high degree of uncertainty caused by weather conditions, user behavior and distributed renewable energy. In addition, as the number and types of equipment connected to the power grid are increasing, the power load consumption pattern is becoming more and more complex. Therefore, electricity load forecasting is a very challenging task. According to the time range of the forecast, load forecasting can be divided into three categories: ultra-short-term load forecasting (one hour in advance), short-term load forecasting (one day to one week in advance) and medium- and long-term load forecasting (one month to several years in advance).

[0003] Most of the existing power load forecasting methods can be divided into three categories: statistical-based methods, machine learning-based methods, and deep learning-based methods. Statistical-based methods such as the autoregressive moving average model (ARMA) and the differenced autoregressive moving average model (ARIMA) have been used for power load forecasting. However, statistical-based methods cannot handle nonlinear relationships, while machine learning-based methods have stronger capabilities in this regard. Therefore, scholars have conducted in-depth research on machine learning-based methods for power load forecasting, such as feedforward neural networks (FNN) and support vector regression models (SVR). However, compared with machine learning-based methods, deep learning-based methods have more powerful feature extraction capabilities. In recent years, deep learning-based methods such as long short-term memory networks (LSTM), recurrent neural networks (RNN), gated recurrent units (GRU), and convolutional neural networks (CNN) have been successfully applied in the field of power load forecasting.

[0004] However, due to similar weather conditions and user behaviors, non-Euclidean spatial relationships are formed between adjacent areas. The above methods mainly use temporal relationships to predict power loads, ignoring the non-Euclidean spatial relationships between adjacent areas.

[0005] In recent years, graph neural networks (GNNs) and their variants, such as graph attention networks (GATs) and graph convolutional networks (GCNs), have performed well in capturing non-Euclidean spatial features. Combining RNN-based models with GNN-based models can capture spatiotemporal features more comprehensively, thereby achieving higher prediction accuracy. However, most GNN-based models require a pre-determined adjacency matrix, making it difficult to represent the real-time spatial relationship between power loads. Summary of the invention

[0006] The purpose of this application is to provide a short-term power load forecasting method and device based on a spatiotemporal neural network model, which uses a learnable adjacency matrix to perform spatiotemporal short-term power load forecasting based on GNN to further improve the accuracy of short-term power load forecasting.

[0007] In order to achieve the above purpose, the technical solution of this application is as follows: A short-term power load forecasting method based on a spatiotemporal neural network model, wherein the spatiotemporal neural network model includes a preset number of adaptive spatiotemporal feature extraction modules, wherein the adaptive spatiotemporal feature extraction module includes an adaptive graph convolution branch and a gated recurrent unit branch, and the short-term power load forecasting method based on the spatiotemporal neural network model includes: Through spatiotemporal embedding, the historical power loads from different regions are transformed into corresponding spatiotemporal graph representations; Taking the spatiotemporal graph representation as input, in the first adaptive spatiotemporal feature extraction module of the spatiotemporal neural network model, the spatial features and temporal features are extracted through the adaptive graph convolution branch and the gated recurrent unit branch respectively, and the spatiotemporal features are fused. After performing residual connection on the input features and output features of each adaptive spatiotemporal feature extraction module, they are input into the next adaptive spatiotemporal feature extraction module to extract spatiotemporal features; The features output by the last adaptive spatiotemporal feature extraction module are connected to the input features through residual connection and then linearly transformed to obtain the prediction results.

[0008] Furthermore, the adaptive graph convolution branch adopts an improved learnable adjacency matrix, and the improved learnable adjacency matrix is ​​obtained by the following formula: in, represents the learnable node embedding matrix, represents the query matrix, represents the key matrix, represents the value matrix, is the dimension of the node embedding matrix, , , represents the learnable weight matrix, represents the weighted node embedding matrix, Represents the improved learnable adjacency matrix.

[0009] Furthermore, the fusion obtains spatiotemporal features, including: Among them, the represents the spatial features extracted by the adaptive graph convolution branch, represents the temporal features extracted from the gated recurrent unit branch, Represents the fused spatiotemporal features.

[0010] Furthermore, the performing residual connection on the input features and output features of each adaptive spatiotemporal feature extraction module includes: in, Indicates The spatiotemporal features extracted by the adaptive spatiotemporal feature extraction module, Indicates The input features of the adaptive spatiotemporal feature extraction module, Represents the features after residual connection.

[0011] The present application also proposes a short-term power load forecasting device based on a spatiotemporal neural network model, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions implement the steps of the above method when executed by the processor.

[0012] This application proposes a short-term power load forecasting method and device based on a spatiotemporal neural network model, which uses a new spatiotemporal neural network model and combines adaptive GCN (AGCN for short) and GRU to extract potential spatiotemporal features to further improve the accuracy of short-term power load forecasting. Compared with previous research work, the model proposed in this application can use an improved learnable adjacency matrix to extract potential non-Euclidean space features, and does not require the use of a predetermined adjacency matrix as prior knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of the short-term power load forecasting method based on the spatiotemporal neural network model of this application.

[0014] Figure 2Schematic diagram of the spatiotemporal neural network model of this application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0016] An embodiment of the present application provides a short-term power load forecasting method based on a spatiotemporal neural network model, such as Figure 1 As shown, including: Step S1: transform the historical power loads from different regions into corresponding spatiotemporal graph representations through spatiotemporal embedding.

[0017] Spatiotemporal embedding is the basis of the spatiotemporal prediction model based on GNN and is a relatively mature technology in this field. In order to embed the power load data of different regions into the graph, this embodiment assumes that each node represents a region, and the historical power loads from different regions can be converted into corresponding spatiotemporal graph representations by splicing. ,in Indicates time The historical power load characteristic matrix of different regions at that time is is the feature number, is the current time, is the number of regions, is the historical time step.

[0018] Step S2: Taking the spatiotemporal graph representation as input, in the first adaptive spatiotemporal feature extraction module of the spatiotemporal neural network model, extracting spatial features and temporal features through an adaptive graph convolution branch and a gated recurrent unit branch respectively, and fusing them to obtain spatiotemporal features.

[0019] This application constructs a new spatiotemporal neural network model, such as Figure 2 The constructed spatiotemporal neural network model includes a preset number of adaptive spatiotemporal feature extraction modules, and the input features and output features of each adaptive spatiotemporal feature extraction module are connected by residual connection and then input into the next adaptive spatiotemporal feature extraction module.

[0020] Each adaptive spatiotemporal feature extraction module consists of an adaptive graph convolution branch (AGCN) and a gated recurrent unit branch (GRU), and then the two branches are fused to extract potential spatiotemporal features.

[0021] The adaptive graph convolution branch (AGCN) proposed in this application can capture the non-Euclidean spatial characteristics of power load data in neighboring areas without using a predetermined adjacency matrix.

[0022] Constructing an adjacency matrix is ​​very important for GNN-based models. Therefore, in order to construct a suitable adjacency matrix to improve the performance of GNN-based models, scholars have studied many methods in depth. In order to train a model for spatiotemporal prediction without using prior knowledge, researchers in the prior art have proposed a learnable adjacency matrix, which can be expressed as formula (1):

[0023] (1) in, represents the learnable adjacency matrix, Represents the learnable node embedding matrix. Formula (1) uses the SoftMax function to normalize the learnable adjacency matrix and uses the ReLU function to eliminate weak connections in the matrix. will be randomly initialized and updated via the stochastic gradient descent algorithm. During this training process, the learnable adjacency matrix adaptively learns non-Euclidean spatial relationships by automatically adjusting the edge weights between nodes.

[0024] In order to enhance the ability of the learnable adjacency matrix to extract non-Euclidean space features, this application improves the learnable adjacency matrix using the attention mechanism. The improved learnable adjacency matrix can be expressed as formula (2-6):

[0025] (2) (3) (4) (5) (6) in, represents the learnable node embedding matrix, represents the query matrix, represents the key matrix, Represents a matrix of values. is the dimension of the node embedding matrix, which is used to limit the size of the attention scores. , , represents the learnable weight matrix, represents the weighted node embedding matrix, represents the improved learnable adjacency matrix. The attention score can be obtained by Multiply This application uses the SoftMax function to normalize the attention scores so that their sum is 1. Then, the attention scores are multiplied by , and obtain the weighted node embedding matrix. The weighted node embedding matrix enables the adjacency matrix to selectively focus on different edge weights, thereby more effectively learning non-Euclidean spatial relationships.

[0026] GCN has attracted much attention and performed well in extracting non-Euclidean spatial features. A single-layer GCN can obtain the spatial features of first-order neighboring nodes, and by stacking multiple GCN layers, information transfer between multi-order neighboring nodes can be achieved. By combining the improved learnable adjacency matrix, GCN can be expressed as formula (7-8):

[0027] (7) (8) in, Indicates based on The resulting diagonal node degree matrix, Represents the feature matrix containing spatial information (spatial features), is a space-time diagram, represents the learnable weight matrix, represents the improved learnable adjacency matrix, express The identity matrix of represents an adjacency matrix with self-loops, Represents the activation function.

[0028] This application proposes an AGCN that combines an improved learnable adjacency matrix with GCN, which can capture the potential non-Euclidean space characteristics of power load without using prior knowledge.

[0029] RNN can process inputs of any length, but its performance will decrease due to gradient vanishing or gradient exploding. GRU is a variant of RNN, which overcomes the above shortcomings of RNN through the gating mechanism. Therefore, this application uses GRU to capture the temporal characteristics of historical power load. GRU can be expressed as formula (9-12):

[0030] (9) (10) (11) (12) in, Represents the reset gate, represents the update gate, represents the candidate feature matrix, Indicates time The feature matrix containing time information when Indicates time The space-time diagram shows that represents the learnable weight matrix of the reset gate, represents the learnable weight matrix of the update gate, represents the learnable weight matrix of the candidate feature matrix, represents the Hadamard product, Represents the feature matrix (time feature) containing time information at the current moment.

[0031] In this embodiment, the temporal feature finally obtained by the gated recurrent unit branch (GRU) is expressed as .

[0032] Finally, the spatial features obtained from AGCN and GRU are and time characteristics The fusion process can be expressed as formula (13):

[0033] (13) in, represents the spatiotemporal feature matrix containing spatiotemporal information, and The dimensions are the same, represents the spatial features extracted by the adaptive graph convolution branch, , Represents the fused spatiotemporal features.

[0034] Step S3: After performing residual connection on the input features and output features of each adaptive spatiotemporal feature extraction module, the features are input to the next adaptive spatiotemporal feature extraction module for spatiotemporal feature extraction.

[0035] The process can be expressed as formula (14): (14) in, Indicates The spatiotemporal features extracted by the adaptive spatiotemporal feature extraction module, Indicates The input features of the adaptive spatiotemporal feature extraction module, Represents the feature after residual connection, that is +1 input feature of the adaptive spatiotemporal feature extraction module. ,but =X.

[0036] Step S4: After performing a residual connection between the features output by the last adaptive spatiotemporal feature extraction module and the input features, a linear transformation is performed to obtain a prediction result.

[0037] The linear transformation can be expressed as formula (15): (15) in, represents the predicted power load in different regions, A learnable weight matrix representing a linear transformation, represents the bias of the linear transformation, It is the result of residual connection between the output features of the last adaptive spatiotemporal feature extraction module and the input features.

[0038] The adaptive spatiotemporal feature extraction module (AGGRU block for short) of this application uses the adaptive graph convolution branch AGCN and the gated recurrent unit branch GRU to extract the spatial and temporal features of the power load. AGCN can capture the non-Euclidean spatial features of power load data in adjacent areas without using a predetermined adjacency matrix. In order to prevent information from being lost when passing between network layers, each AGGRU block is connected to the next block through a residual connection. Finally, by stacking multiple AGGRU blocks, a deeper neural network can be obtained.

[0039] In order to verify the prediction performance of the spatiotemporal neural network model (AGGRU model for short), this application provides experimental data for illustration. The experiment will evaluate the performance on two power load data sets. Both data sets are from an unknown area in the United States and are sampled at intervals of 60 minutes. Among them, GEFCom2012 contains energy data of 20 users from January 1, 2004 to June 30, 2008. GEFCom2017 contains energy data of 500 users from January 1, 2005 to December 31, 2011. The experiment uses the power load data in the above data sets to conduct experiments, and uses the power load data of the previous 24 hours to predict the power load data of the next 24 hours.

[0040] The experiment compares the AGGRU model with the following comparison models to verify its performance: ARIMA: ARIMA is an improved ARMA that can handle non-stationary data.

[0041] SVR: SVR is a type of SVM for regression analysis that can handle nonlinear data.

[0042] FNN: FNN is an artificial neural network in which information passes through hidden layers to the output layer.

[0043] LSTM: LSTM is a variant of RNN that uses a gating mechanism to handle long-range dependencies.

[0044] GRU: GRU is a variant of RNN that uses a gating mechanism to handle long-range dependencies.

[0045] CNN-LSTM: CNN-LSTM uses CNN and LSTM to capture spatial features and temporal features respectively.

[0046] CNN-GRU: CNN-GRU uses CNN and GRU to capture spatial features and temporal features respectively.

[0047] This experiment uses four commonly used evaluation indicators to compare the performance of the AGGRU model with the above comparison models, namely, mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE), and symmetric mean absolute percentage error (SMAPE).

[0048] The hyperparameters of the AGGRU model mainly include learning rate, batch size, weight decay rate, training rounds, loss function, optimizer, hidden layer size and number of blocks. In this experiment, the learning rate is set to 0.001, the batch size is set to 64, the weight decay rate is set to 0.0001, the training rounds are set to 100, the loss function is set to MSE, the optimizer is set to Adam, the hidden layer size is set to 64, and the number of blocks is set to 3.

[0049] Table 1 shows the performance of the AGGRU model and its comparative models on the GEFCom2012 dataset. Table 2 shows the performance of the AGGRU model and its comparative models on the GEFCom2017 dataset.

[0050] As shown in Tables 1 and 2, in most cases, deep learning-based methods outperform statistical-based methods and machine learning-based methods. Among all models, ARIMA performs the worst because it has difficulty capturing the nonlinear characteristics of power load data. In addition, SVR performs second best because it has limitations in processing large-scale data sets.

[0051] Table 1 Table 2 Experiments show that the performance of the hybrid model based on CNN and RNN (GRU and LSTM) is better than that of the single model based on RNN, indicating that the hybrid model has a stronger ability to express spatiotemporal features than the single model. Specifically, on the two datasets, CNN-GRU and CNN-LSTM showed better performance than GRU and LSTM in most evaluation indicators. For example, on the two datasets, the MAE, RMSE, MAPE and SMAPE of CNN-GRU are 1.53%, 0.28%, 0.45% and 0.04% higher than those of GRU, respectively, while the MAE, RMSE and MAPE of CNN-LSTM are 0.74%, 0.19% and 0.08% higher than those of LSTM, respectively.

[0052] On the two data sets, the MAE, RMSE, MAPE and SMAPE of the AGGRU model are 9.51%, 11.10%, 7.28% and 7.20% higher than those of CNN-GRU, respectively, indicating that non-Euclidean spatial features can more accurately reflect the spatial relationship of power load data than Euclidean spatial features.

[0053] This application also designs an ablation experiment to compare different variants of the AGGRU model to verify the performance of different modules. Specifically, the following variants are designed.

[0054] AGGRU (nonAGCN): The model architecture of AGGRU (nonAGCN) is the same as the AGGRU model, but the former only uses GRU to extract the temporal characteristics of power load.

[0055] AGGRU (nonGRU): The model architecture of AGGRU (nonGRU) is the same as that of the AGGRU model, but the former only uses AGCN to extract the spatial features of power load.

[0056] AGGRU (nonEL): The model architecture of AGGRU (nonEL) is the same as the AGGRU model, but the former uses a similarity-based adjacency matrix instead of the improved learnable adjacency matrix.

[0057] AGGRU (nonSA): The model architecture of AGGRU (nonSA) is the same as the AGGRU model, but the former does not use the attention mechanism.

[0058] Tables 3 and 4 show the performance comparison of the AGGRU model and its different variants on the GEFCom2012 and GEFCom2017 datasets, respectively.

[0059] Table 3 Table 4 As can be seen from Tables 3 and 4, the AGGRU model outperforms AGGRU (nonAGCN) and AGGRU (nonGRU) in performance, indicating that compared with single temporal features or spatial features, spatiotemporal features have higher value for power load forecasting. AGGRU (nonEL) performs worse than AGGRU and AGGRU (nonSA) in most indicators, indicating that the improved learnable adjacency matrix is ​​more suitable for GNN-based power load forecasting than the predetermined adjacency matrix. AGGRU (nonSA) performs worse than AGGRU, indicating that the attention mechanism is effective in the improved learnable adjacency matrix.

[0060] This application proposes a new spatiotemporal short-term power load forecasting model AGGRU, which uses AGCN and GRU to capture the spatiotemporal characteristics of historical power load. By introducing an improved learnable adjacency matrix, the AGGRU model can adaptively extract non-Euclidean spatial features of adjacent areas, and can selectively focus on different edge weights through the attention mechanism, thereby more effectively learning non-Euclidean spatial features.

[0061] In one embodiment, the present application also provides a short-term power load forecasting device based on a spatiotemporal neural network model, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions implement the steps of the above method when executed by the processor.

[0062] For the specific limitations of the short-term power load forecasting device based on the spatiotemporal neural network model, please refer to the limitations of the short-term power load forecasting method based on the spatiotemporal neural network model mentioned above, which will not be repeated here. The above-mentioned short-term power load forecasting device based on the spatiotemporal neural network model can be implemented in whole or in part through software, hardware and a combination thereof. It can be embedded in or independent of the processor in the computer device in the form of hardware, or it can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0063] The memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory stores a computer program that can be run on the processor, and the processor implements the short-term power load forecasting method based on the spatiotemporal neural network model in the embodiment of the present invention by running the computer program stored in the memory.

[0064] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory is used to store a program, and the processor executes the program after receiving an execution instruction.

[0065] The processor may be an integrated circuit chip with data processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0066] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A short-term power load forecasting method based on spatiotemporal neural network model, It is characterized in that The spatiotemporal neural network model includes a preset number of adaptive spatiotemporal feature extraction modules, the adaptive spatiotemporal feature extraction module includes an adaptive graph convolution branch and a gated recurrent unit branch, and the short-term power load forecasting method based on the spatiotemporal neural network model includes: Through spatiotemporal embedding, the historical power loads from different regions are transformed into corresponding spatiotemporal graph representations; Taking the spatiotemporal graph representation as input, in the first adaptive spatiotemporal feature extraction module of the spatiotemporal neural network model, the spatial features and temporal features are extracted through the adaptive graph convolution branch and the gated recurrent unit branch respectively, and the spatiotemporal features are fused. After performing residual connection on the input features and output features of each adaptive spatiotemporal feature extraction module, they are input into the next adaptive spatiotemporal feature extraction module to extract spatiotemporal features; The features output by the last adaptive spatiotemporal feature extraction module are connected to the input features through residual connection and then linearly transformed to obtain the prediction results.

2. The short-term power load forecasting method based on the spatiotemporal neural network model according to claim 1, It is characterized in that The adaptive graph convolution branch adopts an improved learnable adjacency matrix, and the improved learnable adjacency matrix is ​​obtained by the following formula: in, represents the learnable node embedding matrix, represents the query matrix, represents the key matrix, represents the value matrix, is the dimension of the node embedding matrix, , , represents the learnable weight matrix, represents the weighted node embedding matrix, Represents the improved learnable adjacency matrix.

3. The short-term power load forecasting method based on the spatiotemporal neural network model according to claim 1, It is characterized in that The fusion obtains spatiotemporal features, including: Among them, the represents the spatial features extracted by the adaptive graph convolution branch, represents the temporal features extracted from the gated recurrent unit branch, Represents the fused spatiotemporal features.

4. The short-term power load forecasting method based on the spatiotemporal neural network model according to claim 1, It is characterized in that The step of performing residual connection on the input features and output features of each adaptive spatiotemporal feature extraction module includes: in, Indicates The spatiotemporal features extracted by the adaptive spatiotemporal feature extraction module, Indicates The input features of the adaptive spatiotemporal feature extraction module, Represents the features after residual connection.

5. A short-term power load forecasting device based on a spatiotemporal neural network model, comprising a processor and a memory storing a plurality of computer instructions, It is characterized in that When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.

Images