A power load prediction method based on a SAINet neural network model

Abstract

The application discloses a power load prediction method based on a SAINet neural network model, which comprises the following steps: step one, acquiring power load historical factors and power load meteorological factors; step two, acquiring a power load data set; step three, acquiring a power load training data set and a power load test data set; step four, training and acquiring a trained SAINet neural network model; and step five, using the trained SAINet neural network model to predict the power load and output a power load prediction value. The application aims at further improving the accuracy of power load prediction and provides the power load prediction method based on the SAINet neural network model.

Description

Technical Field

[0001] This invention belongs to the field of power systems and related technologies, specifically relating to a method for power load forecasting, and more particularly to a method for power load forecasting based on the SAINet neural network model. Background Technology

[0002] Electricity load forecasting is a crucial component of power system planning and a foundation for the economic operation of the power system, playing a vital role in both planning and operation. In the national path of green development, carbon neutrality and carbon peaking have become phased goals, placing electricity load forecasting in an even more important position. However, due to meteorological factors such as daily average temperature, rainfall, and humidity, randomness and uncertainty in electricity load are unavoidable. The nonlinearity of grid load becomes increasingly complex with the increase of input parameters, rendering traditional electricity load forecasting methods inadequate for current development needs.

[0003] Patent document CN103208036B discloses a short-term load detection method based on power users. This method establishes an autoregressive moving average (ARMA) model based on user power load data. After identifying and estimating the parameters of the ARMA model, it calculates the predicted power load data. Patent document CN104318334B discloses a short-term power load prediction method based on a long correlation FARIMA model. It uses rescaled range analysis to filter the power load data, obtaining an effective power load sequence {X}. t After that, from the sequence {X} t A FARIMA model of the power load sequence is established, and the power load value is predicted based on the FARIMA model. However, the above method cannot effectively extract information on meteorological factors affecting power load, resulting in significant errors in the accuracy of power load prediction.

[0004] The key to improving the accuracy of power load forecasting lies in feature extraction from power data. When extracting features, the location information, trend, periodicity and time information of the load sequence should be fully utilized, and a deeper and more effective neural network model is needed for calculation.

[0005] To obtain power load forecast data efficiently and accurately, this patent proposes the SAINet neural network model to predict power load. Summary of the Invention

[0006] The purpose of this invention is to address the technical problems of traditional methods in power load forecasting, which rely excessively on historical power load data and lack effective information extraction on meteorological factors affecting power load. Instead, this invention provides an intelligent power load forecasting method based on a combination of convolutional neural networks and self-attention mechanisms.

[0007] A SAINet network model, the network structure of which is as follows:

[0008] Input features → Several SAI-Block modules forming a binary tree structure → Merging and rearranging → Fusion → Fully connected layer → Output features;

[0009] The first layer of the binary tree has a SAI-Block module. The features output by multiple SAI-Block modules in the last layer of the binary tree are merged, rearranged, and then fused with the input features of the SAI-Block module in the first layer of the binary tree. The fused features are then input into the fully connected layer.

[0010] The network structure of the SAI-Block module is as follows:

[0011] Feature F 偶 →First self-attention layer→First exp function processing; Feature F 奇 →Second self-attention layer→Second exp function processing;

[0012] Feature F 偶 The second exp function processes → the first Hadamard product → F 1 偶 Feature F 奇 The first exp function processes the product → the second Hadamard product → F 1 奇 ;

[0013] F 1 奇 →First convolutional layer, F 1 偶 →First Fusion→F 2 偶 ;F 1 偶 →Second convolutional layer, F 1 奇 →Second Fusion→F 2 奇 ;

[0014] Input → Downsampling → Feature F 奇 →F 1 奇 →F 2 奇 Input → Downsampling → Feature F 偶 →F 1偶 →F 2 偶 .

[0015] The structures of the first and second convolutional layers are as follows:

[0016] Input features → First 1×1 convolutional layer → 3×3 convolution → Second 1×1 convolutional layer, Input feature x → Fusion operation → Output features.

[0017] The stacked SAINet network includes several serially connected SAINet networks, which are connected to each other via residual connection modules.

[0018] The SAI-Block module operates using the following steps:

[0019] First, feature F 偶 and feature F 奇 Perform self-attention mechanism operations separately, process the data after the operations using the exp function, and then combine them with the feature F. 偶 and feature F 奇 The feature F is obtained by performing a product operation. 1 偶 and feature F 1 奇 See formula (1). The above interactive operation can be regarded as an action on feature F. 偶 and feature F 奇 Scaling transformation;

[0020] Secondly, for feature F 1 偶 and feature F 1 奇 Two sequences are convolutionally processed separately, and feature F is obtained. 1 偶 With the features F after convolution operation 1 奇 Perform addition operation, feature F 1 奇 Features F after convolution 1 偶 The addition operation is shown in formula (2). The final output after interactive learning of the sequence is the feature F. 2 偶 and feature F 2 奇 ;

[0021] F 1 奇 = F 奇 ⊙ exp ( Φ(F 偶 ) ) , F 1 偶= F 偶 ⊙ exp ( ψ(F 奇 ) ) (1)

[0022] F 2 奇 = F 1 奇 +( ρ(F 1 偶 ) ) , F 2 偶 = F 1 偶 + ( η(F 1 奇 (2)

[0023] In equations (1) and (2), ⊙ represents the Hadamard product; exp represents an exponential function with the natural constant e as its base.

[0024] Feature F 1 偶和 Feature F 1 奇 These are the sequence data after the self-attention mechanism has been applied; feature F 2 偶 and feature F 2 奇 It is the final output sequence data after all interactive learning, where Φ and ψ represent self-attention mechanism operations, and ρ and η represent convolution operations.

[0025] The method for power load forecasting using the SAINet neural network model includes the following steps:

[0026] Step 1: Obtain historical factors and meteorological factors related to power load;

[0027] Step 2: Obtain the power load dataset;

[0028] Step 3: Obtain the power load training dataset and the power load test dataset;

[0029] Step 4: Train and obtain the trained SAINet neural network model;

[0030] Step 5: Use the trained SAINet neural network model to predict the power load and output the predicted power load value.

[0031] In step four, the SAINet network model as described in any one of claims 1 to 5 is used.

[0032] In step one, the historical factors of power load include historical power load values, and the meteorological factors of power load include the highest temperature, lowest temperature, relative humidity, and rainfall of the predicted area.

[0033] In step two, historical factors and meteorological factors of power load are aggregated to obtain a power load dataset;

[0034] In step three, the power load dataset is divided to obtain a power load training dataset and a power load test dataset.

[0035] When training the SAINet neural network model, the input sample data is downsampled and divided into two sub-sequence features F. 偶 and feature F 奇 .

[0036] Compared with the prior art, the present invention has the following technical effects:

[0037] 1) This invention adds a set of convolutional layers to the convolution operation, forming an inverted bottleneck structure. At the cost of increasing the computation of some parameters, it improves the nonlinear performance of the model. In this invention, the SAINet neural network model, while having the advantages of a tree structure, adds a self-attention mechanism and an inverted bottleneck structure, which more effectively extracts the potential features of power load data.

[0038] 2) This invention introduces a self-attention mechanism, which can effectively extract global features of the data during interactive learning of data features. Attached Figure Description

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0040] Figure 1 This is a schematic diagram of the SAINet network structure in the invention;

[0041] Figure 2 for Figure 1 A schematic diagram of the network structure of the SAI-Block module in the middle;

[0042] Figure 3 for Figure 2 A schematic diagram of the network structure of the middle convolutional layer;

[0043] Figure 4 This is a schematic diagram of the stacked SAINet network structure in this invention;

[0044] Figure 5 for Figure 4 Network structure diagram of the middle connection module;

[0045] Figure 6 This is a flowchart of the present invention;

[0046] Figure 7 This is the power load forecasting diagram in this invention;

[0047] Figure 8 This is a schematic diagram illustrating the error between the actual and predicted power load values ​​in this invention. Detailed Implementation

[0048] like Figure 1 As shown, a SAINet network model has the following network structure:

[0049] Input features → Several SAI-Block modules forming a binary tree structure → Merging and rearranging → Fusion → Fully connected layer → Output features;

[0050] The first layer of the binary tree has a SAI-Block module. The features output by multiple SAI-Block modules in the last layer of the binary tree are merged, rearranged, and then fused with the input features of the SAI-Block module in the first layer of the binary tree. The fused features are then input into the fully connected layer.

[0051] Arranging SAI-Blocks into a binary tree structure has the advantage that each SAI-Block has local and global features of the entire time series, which enhances feature extraction. After completing all downsampling-self-attention-interactive learning operations, the extracted features are rearranged into a new sequence representation and added to the original time series in the form of residual connections. A fully connected network is then used as the decoder for prediction.

[0052] like Figure 2 As shown, the structure of the SAI-Block module is as follows:

[0053] Feature F 偶 →First self-attention layer→First exp function processing; Feature F 奇 →Second self-attention layer→Second exp function processing;

[0054] Feature F 偶 The second exp function processes → the first Hadamard product → F 1 偶 Feature F 奇 The first exp function processes the product → the second Hadamard product → F 1 奇 ;

[0055] F 1 奇 →First convolutional layer, F 1 偶 →First Fusion→F 2 偶 ;F 1偶 →Second convolutional layer, F 1 奇 →Second Fusion→F 2 奇 ;

[0056] Input → Downsampling → Feature F 奇 →F 1 奇 →F 2 奇 Input → Downsampling → Feature F 偶 →F 1 偶 →F 2 偶 .

[0057] like Figure 3 As shown, the structures of the first and second convolutional layers are as follows:

[0058] Input features → First 1×1 convolutional layer → 3×3 convolution → Second 1×1 convolutional layer, Input feature x → Fusion operation → Output features.

[0059] like Figure 4 As shown, a stacked SAINet network consists of several serially connected SAINet networks, which are linked together via residual connection modules. When there are enough training samples, the SAINet networks can be stacked to obtain better prediction accuracy.

[0060] like Figure 5 As shown, the residual connection mechanism is as follows:

[0061] Redidual block = Activation( x+F(x) ) (7)

[0062] After introducing the residual network structure, when performing feature extraction on complex data, the stacked SAINet structure avoids the problems of gradient explosion and gradient vanishing when making data prediction. Activation represents the activation function.

[0063] The SAI-Block module operates using the following steps:

[0064] First, feature F 偶 and feature F 奇 Perform self-attention mechanism operations separately, process the data after the operations using the exp function, and then combine them with the feature F. 偶 and feature F 奇 The feature F is obtained by performing a product operation. 1 偶 and feature F 1 奇See formula (1). The above interactive operation can be regarded as an action on feature F. 偶 and feature F 奇 Scaling transformation;

[0065] Secondly, for feature F 1 偶 and feature F 1 奇 Two sequences are convolutionally processed separately, and feature F is obtained. 1 偶 With the features F after convolution operation 1 奇 Perform addition operation, feature F 1 奇 Features F after convolution 1 偶 The addition operation is shown in formula (2). The final output after interactive learning of the sequence is the feature F. 2 偶 and feature F 2 奇 ;

[0066] F 1 奇 = F 奇 ⊙ exp ( Φ(F 偶 ) ) , F 1 偶 = F 偶 ⊙ exp ( ψ(F 奇 ) ) (1)

[0067] F 2 奇 = F 1 奇 +( ρ(F 1 偶 ) ) , F 2 偶 = F 1 偶 + ( η(F 1 奇 (2)

[0068] In equations (1) and (2), ⊙ represents the Hadamard product; exp represents an exponential function with the natural constant e as its base.

[0069] Feature F 1 偶和 Feature F 1 奇 These are the sequence data after the self-attention mechanism has been applied; feature F 2 偶 and feature F2 奇 It is the final output sequence data after all interactive learning, where Φ and ψ represent self-attention mechanism operations, and ρ and η represent convolution operations.

[0070] The working principle of the self-attention mechanism is:

[0071]

[0072] Q = W Q Z (4) K=W K Z (5) V=W V Z (6)

[0073] Equation (3) describes the self-attention mechanism. Using self-attention on data assigns different weights to the internal information of the data features, enabling subsequent neural network feature extraction and global learning, thus improving the efficiency of feature extraction. Q, K, and V are tensors identical to Z, as shown in equations (4), (5), and (6), respectively. Q W k W v Let represent the parameter matrix, d represent the feature dimension of Q, and softmax represent the activation function.

[0074] This invention also provides a method for power load forecasting using the SAINet neural network model, which includes the following steps:

[0075] Step 1: Obtain historical factors and meteorological factors related to power load;

[0076] Step 2: Obtain the power load dataset;

[0077] Step 3: Obtain the power load training dataset and the power load test dataset;

[0078] Step 4: Train and obtain the trained SAINet neural network model;

[0079] Step 5: Use the trained SAINet neural network model to predict the power load and output the predicted power load value.

[0080] In step four, the SAINet network model as described in any one of claims 1 to 5 is used.

[0081] In step one, the historical factors of power load include historical power load values, and the meteorological factors of power load include the highest temperature, lowest temperature, relative humidity, and rainfall of the predicted area.

[0082] In step two, historical factors and meteorological factors of power load are aggregated to obtain a power load dataset;

[0083] In step three, the power load dataset is divided to obtain a power load training dataset and a power load test dataset.

[0084] When training the SAINet neural network model, the input sample data is downsampled and divided into two sub-sequence features F. 偶 and feature F 奇 .

[0085] Depend on Figure 7 It can be seen that the smooth part represents the predicted power load line, while the rough-edged part represents the actual power load line; from Figure 7 and Figure 8 It can be seen that the present invention can effectively complete the task of power load forecasting and obtain more accurate forecast data.

[0086] This invention addresses the difficulty in extracting latent features from power load data by establishing a SAINet neural network to effectively extract these features and accurately predict power load. The SAINet neural network model constructed in this invention utilizes an attention mechanism to enhance the extraction of global features from power load data. Based on the convolutional structure, this invention adds additional convolutional layers, forming an inverted bottleneck structure. This increases the number of parameters and computational cost, but improves the model's nonlinear performance. The method based on the SAINet neural network model proposed in this invention achieves intelligent short- and medium-term power load prediction under the influence of multiple factors.

Claims

1. A method for power load forecasting using the SAINet neural network model, characterized in that, Includes the following steps: Step 1: Obtain historical factors and meteorological factors related to power load; Step 2: Obtain the power load dataset; Step 3: Obtain the power load training dataset and the power load test dataset; Step 4: Train and obtain the trained SAINet neural network model; Step 5: Use the trained SAINet neural network model to predict the power load and output the predicted power load value; In step one, the historical factors of power load include historical power load values, and the meteorological factors of power load include the highest temperature, lowest temperature, relative humidity, and rainfall of the predicted area. In step two, historical factors and meteorological factors of power load are aggregated to obtain a power load dataset; In step three, the power load dataset is divided to obtain a power load training dataset and a power load test dataset; When training the SAINet neural network model, the input sample data is downsampled and divided into two sub-sequence features F. 偶 and feature F 奇 ; The SAINet neural network model is specifically as follows: Input features → Several SAI-Block modules forming a binary tree structure → Merging and rearranging → Fusion → Fully connected layer → Output features; The first layer of the binary tree has a SAI-Block module. The features output by multiple SAI-Block modules in the last layer of the binary tree are merged, rearranged, and then fused with the input features of the SAI-Block module in the first layer of the binary tree. The fused features are then input into the fully connected layer. In a stacked SAINet network, there are several serially connected SAINet neural network models, which are connected to each other through residual connection modules. The SAI-Block module operates using the following steps: First, feature F 偶 and feature F 奇 Perform self-attention mechanism operations separately, process the data after the operations using the exp function, and then combine them with the feature F. 偶 and feature F 奇 The feature F is obtained by performing a product operation. 1 偶 and feature F 1 奇 The process is shown in formula (1). The above interactive operation can be regarded as the interaction of feature F. 偶 and feature F 奇 Scaling transformation; Secondly, for feature F 1 偶 and feature F 1 奇 Two sequences are convolutionally processed separately, and feature F is obtained. 1 偶 With the features F after convolution operation 1 奇 Perform addition operation, feature F 1 奇 Features F after convolution 1 偶 The addition operation is performed, as shown in formula (2). The final output of the sequence after interactive learning is the feature F. 2 偶 and feature F 2 奇 ; ； ； In equations (1) and (2), ⊙ represents the Hadamard product; exp represents an exponential function with the natural constant e as its base. Feature F 1 偶 and feature F 1 奇 These are the sequence data after the self-attention mechanism has been applied; feature F 2 偶 and feature F 2 奇 It is the final output sequence data after all interactive learning. , This represents the operation of the self-attention mechanism. , This represents the convolution operation.

Images