Power load prediction method based on improved Transform algorithm

Abstract

The invention discloses a power load prediction method based on an improved Transform algorithm. The method comprises the following steps: collecting power load related data; the power load related data comprises historical power load data and environment information data; preprocessing the power load related data to obtain processed data; performing feature extraction and feature fusion on the processed data to obtain a fused feature vector; extracting time sequence feature information from the fused feature vectors; and predicting the power load according to the time sequence characteristic information to obtain a prediction result. The method can improve the feature expression capability and calculation efficiency of prediction, and remarkably improves the precision of power load prediction.

Description

Technical Field

[0001] The present invention relates to the electric power field, and in particular to an electric power load forecasting method based on an improved Transformer algorithm. Background Art

[0002] In the power system, load forecasting is a crucial task, which is not only related to the stable operation of the power system, but also directly affects the balance of power supply and demand, power dispatching and the reasonable arrangement of power generation plan. However, power load forecasting faces many challenges, mainly including the nonlinearity, time-varying and uncertainty of load, which makes it quite difficult to accurately predict load.

[0003] Traditional load forecasting methods often rely on statistical models or time series analysis methods, such as the autoregressive integrated moving average model (ARIMA) and support vector machine (SVM). These methods can capture the changing trend of load to a certain extent, but when dealing with complex and changeable load data, the prediction accuracy is often difficult to guarantee. Especially in the context of the increase in the proportion of renewable energy power generation and the increase in extreme climate events, the power load is affected by many factors, and its prediction difficulty is further increased.

[0004] In recent years, with the rapid development of artificial intelligence technology, deep learning models have shown great potential in the field of power load forecasting. Among them, the Transformer model, as a natural language processing model based on the attention mechanism, has attracted much attention because of its ability to capture long-distance dependencies and process nonlinear features in sequence data. In power load forecasting, the Transformer model can effectively mine the potential features in the load sequence and improve the prediction accuracy. However, power load data often contains rich location information, trend, periodicity, time and other large-scale data information. When processing these large-scale data, the Transformer model has high computational complexity and low computational efficiency.

[0005] Therefore, in order to solve the above problems, a power load forecasting method based on the improved Transformer algorithm is needed, which can improve the feature expression ability and computational efficiency of the prediction and significantly improve the accuracy of power load forecasting. Summary of the invention

[0006] In view of this, the purpose of the present invention is to overcome the defects in the prior art and provide a power load forecasting method based on an improved Transformer algorithm, which can improve the feature expression ability and computational efficiency of the prediction and significantly improve the accuracy of power load forecasting.

[0007] The power load forecasting method based on the improved Transformer algorithm of the present invention comprises:

[0008] Collecting power load related data; the power load related data includes historical power load data and environmental information data;

[0009] Preprocessing the power load related data to obtain processed data;

[0010] Extract features from the processed data and perform feature fusion to obtain a fused feature vector;

[0011] Extracting time series feature information from the fused feature vector;

[0012] The power load is predicted according to the time series characteristic information to obtain the prediction result.

[0013] Furthermore, the environmental information data includes weather data, temperature and humidity data, and agricultural activity data.

[0014] Furthermore, the power load related data is preprocessed, specifically including:

[0015] Identify outliers and missing values ​​in power load related data;

[0016] Use the average value of the time near the abnormal point to complete the data repair;

[0017] The min-max standardization method is used to normalize the power load related data;

[0018] Perform dimensionality-upgrading processing on the historical power load data, and upgrade the historical power load data in the original two-dimensional feature data format into a three-dimensional data format;

[0019] Among them, the three-dimensional data format is [time_step, input_size, batch_size], time_step represents the time step, input_size represents the input feature dimension, and batch_size represents the number of samples.

[0020] Further, feature extraction is performed on the processed data, specifically including:

[0021] Using sliding window technology, a fixed-size window is slid on the historical power load data, and the statistics in each window are calculated. The maximum and minimum values ​​in the statistics are used as sequence features, and the long-term trend features are extracted from the historical power load data using the time series analysis method.

[0022] Extract the weather status and take the features that have a significant impact on the load as weather features;

[0023] Extract temperature and humidity values ​​to obtain temperature and humidity characteristics, and divide the temperature and humidity values ​​into different intervals to reflect the impact of different temperatures on the load;

[0024] Agricultural activities are divided into different types according to their seasonal characteristics, and each type is assigned a unique code.

[0025] Furthermore, feature fusion is performed on the processed data, specifically including:

[0026] The time alignment method is used to ensure that the feature vector at each time point contains all kinds of features corresponding to that time point;

[0027] The extracted sequence features, trend features, weather features, temperature and humidity features, and agricultural activity features are concatenated into a long feature vector in a set order; all concatenated feature vectors are normalized.

[0028] Furthermore, the temporal feature information is extracted from the fused feature vector, including:

[0029] Construct a prediction model based on the improved Transformer; the prediction model includes an encoder and a decoder; the encoder includes input embedding and position encoding, additive attention mechanism, residual connection and root mean square layer normalization and position forward propagation network; wherein, input embedding maps each input data to a word embedding vector; position encoding represents the position information of each word in the input sequence; additive attention mechanism is used to capture the relationship between each other; residual connection is used to solve the problem of gradient disappearance in model training; layer normalization is used to ensure that the model training is more stable;

[0030] The decoder includes a masked self-attention layer, a residual connection and a root mean square layer normalization, and a position-by-position feedforward neural network; wherein the masked self-attention layer allows the decoder to only pay attention to all positions before the current position when processing the current position, so as to maintain the autoregressive characteristics; the residual connection and layer normalization are used to better train the model and solve the problem of vanishing and exploding gradients in the network; the position-by-position feedforward neural network is used to independently process the representation of each position;

[0031] The prediction model is trained to obtain a trained prediction model; the fused feature vector is input into the trained prediction model and the following processing is performed:

[0032] Suppose a query vector q and a set of key-value pairs (k1, v1), ..., (k i , v i ),…,(k n , v n ), and enhances the expressive power through a nonlinear activation function of the additive attention mechanism:

[0033] f att (q,k i )=tanh(w a [q; k i ]),

[0034] Among them, f att is the attention function; k i represents the key of the i-th key-value pair; w a is the weight matrix, [;] represents vector concatenation; tanh() represents the activation function;

[0035] The fused feature vector is converted into attention weight a through a linear layer and a softmax function i , used to represent each value v i The calculation formula is as follows:

[0036] Represents the weight matrix W corresponding to the linear layer v The transpose of

[0037] According to the calculated attention weight a i Perform weighted summation on all vectors to obtain the final context vector c:

[0038] The output vector c is processed in two ways. One side is input into the feedforward neural network layer and then normalized in the root mean square layer. The other side is not processed and then residually connected with the processed data. The formula is as follows:

[0039]

[0040]

[0041] Among them, c i represents the i-th element of vector c; RMS(c) represents the normalization of vector c; g i represents the normalized scaling factor; It is the value after RMS normalization calculation.

[0042] Further, the prediction model is trained, specifically including:

[0043] Determine the features and feature dimensions input into the prediction model based on the extracted features;

[0044] Randomly extract 70%, 20%, and 10% of the power load related data as training set, test set, and validation set respectively;

[0045] Determine the input step size, number of encoders and number of decoders of the prediction model;

[0046] Determine the prediction step size of the prediction model;

[0047] Determine the size of data that the prediction model processes at one time, the discard rate, and the training cycle;

[0048] Determine the Adam optimizer and ensure that the initial learning rate is adjusted between [0, 1];

[0049] Gelu is used as the activation function; the root mean square error RMSE is used as the model evaluation standard. In each iteration, the parameters of the prediction model are adjusted according to the root mean square error between the prediction result and the true value, so that after multiple trainings, the prediction result of the prediction model can continue to approach the true value.

[0050] Furthermore, the power load is predicted based on the time series characteristic information, including:

[0051] Input the time series feature information into the fully connected layer of the prediction model to obtain the output features of the fully connected layer;

[0052] The output features of the fully connected layer are input into the activation layer of the prediction model, and the power load prediction results are output.

[0053] The beneficial effects of the present invention are as follows: the present invention discloses a method for predicting power load based on an improved Transformer algorithm, which embeds features of historical power load data by introducing a feature embedding layer, and fuses features of environmental related factors to obtain a fused feature vector. The Transformer model is improved by introducing additive attention mechanism, residual connection, root mean square layer normalization and other technical means, and the nonlinear time series dependency is extracted from the fused feature vector by using the improved Transformer model, and load prediction is realized through a fully connected network, thereby improving computational efficiency and prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0055] Figure 1 It is a schematic diagram of the flow chart of the power load forecasting method of the present invention;

[0056] Figure 2 Schematic diagram of the prediction model architecture based on the improved Transformer of the present invention. DETAILED DESCRIPTION

[0057] The present invention is further described below in conjunction with the accompanying drawings, as shown in the drawings:

[0058] This embodiment discloses a power load forecasting method based on an improved Transformer algorithm, comprising the following steps:

[0059] Collecting power load related data; the power load related data includes historical power load data and environmental information data;

[0060] Preprocessing the power load related data to obtain processed data;

[0061] Extract features from the processed data and perform feature fusion to obtain a fused feature vector;

[0062] Extracting time series feature information from the fused feature vector;

[0063] The power load is predicted according to the time series characteristic information to obtain the prediction result.

[0064] In this embodiment, the environmental information data includes weather data, temperature and humidity data, and agricultural activity data. The historical power load data is mainly obtained from the database of the power company, and these data usually include hourly or daily power load values.

[0065] Weather data is mainly obtained from meteorological data service providers, which are related to power load, including temperature, precipitation, humidity, wind speed, wind direction, sunshine duration, etc.

[0066] Temperature and humidity data are mainly obtained from meteorological stations. Agricultural activity data mainly collects irrigation, the use of agricultural machinery and equipment, and the time data of animal husbandry activities, and the collection time is 15 minutes.

[0067] By using the above data, we can deeply explore the characteristics of these data and provide rich inputs for the load forecasting model, thereby improving the accuracy of the forecast.

[0068] In this embodiment, the power load related data is preprocessed, specifically including:

[0069] The box plot method is used to analyze the data and identify outliers and missing values ​​in the power load related data; when it is lower than the lower limit set by the box plot or higher than the upper limit set, it can be regarded as an outlier. There are discontinuous points in the upper and lower parts of the figure, which are the detected abnormal data.

[0070] Through interpolation, the data is repaired using the average value of the time near the abnormal point;

[0071] The min-max normalization method is used to normalize the power load related data; for each attribute, minA and maxA are respectively the minimum and maximum values ​​of attribute A, and an original value x of A is mapped to a value x' in the interval [0,1] through min-max normalization, and the formula is: new data = (original data - minimum value) / (maximum value - minimum value);

[0072] Perform dimensionality-upgrading on the historical power load data, and upgrade the original two-dimensional feature data format of the historical power load data into a three-dimensional data format, which is used as the input of the prediction model;

[0073] Among them, the three-dimensional data format is [time_step, input_size, batch_size], time_step represents the time step, input_size represents the input feature dimension, and batch_size represents the number of samples, that is, the number of samples put into the prediction model.

[0074] In this embodiment, feature extraction is performed on the processed data, specifically including:

[0075] Using sliding window technology, a fixed-size window is slid on the historical power load data, and the statistics in each window are calculated. The maximum and minimum values ​​in the statistics are used as sequence features. Time series analysis methods, such as seasonal decomposition, are used to extract long-term trend features from historical power load data.

[0076] Extract the weather status and take the features that have a significant impact on the load as weather features;

[0077] Extract temperature and humidity values ​​to obtain temperature and humidity characteristics, and divide the temperature and humidity values ​​into different intervals to reflect the impact of different temperatures on the load;

[0078] According to the seasonal characteristics of agricultural activities, agricultural activities are divided into different types, such as irrigation period, harvest period, fallow period, etc., and a unique code is assigned to each type.

[0079] In this embodiment, feature fusion is performed on the processed data, specifically including:

[0080] The time alignment method is used to ensure that the feature vector at each time point contains all kinds of features corresponding to that time point;

[0081] The extracted sequence features, trend features, weather features, temperature and humidity features, and agricultural activity features are concatenated into a long feature vector in a set order to ensure that the prediction model can capture the correlation information between various features;

[0082] All concatenated feature vectors are normalized to ensure their consistency in numerical range and dimension, thereby avoiding numerical problems during model training.

[0083] In this embodiment, extracting time series feature information from the fused feature vector specifically includes:

[0084] Build a prediction model based on improved Transformer; Figure 2 As shown, the prediction model includes an encoder and a decoder; the encoder includes input embedding and position encoding, additive attention mechanism, residual connection and root mean square layer normalization and position forward propagation network; wherein, input embedding maps each input data to a word embedding vector; position encoding represents the position information of each word in the input sequence; additive attention mechanism is used to capture the relationship between each other; residual connection is used to solve the problem of gradient disappearance in model training; layer normalization is used to ensure that the training of the model is more stable;

[0085] The decoder includes a masked self-attention layer, a residual connection and a root mean square layer normalization, and a position-by-position feedforward neural network; wherein the masked self-attention layer allows the decoder to only pay attention to all positions before the current position when processing the current position, so as to maintain the autoregressive characteristics; the residual connection and layer normalization are used to better train the model and solve the problem of vanishing and exploding gradients in the network; the position-by-position feedforward neural network is used to independently process the representation of each position;

[0086] The additive attention mechanism is more stable than the multi-head attention mechanism in terms of computational consistency and numerical stability. In addition, additive attention can capture more complex similarity models through nonlinear transformation and learning of weight parameters, improving the flexibility and expressiveness of the model between different features. Finally, additive attention alleviates the problem of vanishing gradients in traditional Transformers. In addition, the layer normalization in the Transformer encoder and decoder architecture is replaced with root mean square layer normalization, which solves the problem of Transformer computational complexity, reduces computational complexity, and achieves lower computational overhead. It also provides an effect similar to normalization, which is suitable for large-scale distributed computing.

[0087] The prediction model is trained to obtain a trained prediction model; the fused feature vector is input into the trained prediction model and the following processing is performed:

[0088] Suppose a query vector q and a set of key-value pairs (k1, v1), ..., (k i , v i ),…,(k n , v n ), and enhances the expressive power through a nonlinear activation function of the additive attention mechanism:

[0089] f att (q,k i )=tanh(w a [q; k i ]),

[0090] Among them, f att is the attention function; k i represents the key of the i-th key-value pair; w a is the weight matrix, [;] represents vector concatenation; tanh() represents the activation function;

[0091] The fused feature vector is converted into attention weight a through a linear layer and a softmax function i , used to represent each value v i The calculation formula is as follows:

[0092] Represents the weight matrix W corresponding to the linear layer v The transpose of

[0093] According to the calculated attention weight a i Perform weighted summation on all vectors to obtain the final context vector c:

[0094] The output vector c is processed in two ways. One side is input into the feedforward neural network layer and then normalized in the root mean square layer. The other side is not processed and then residually connected with the processed data. The formula is as follows:

[0095]

[0096]

[0097] Among them, c i represents the i-th element of vector c; RMS(c) represents the normalization of vector c; g i represents the normalized scaling factor; It is the value after RMS normalization calculation.

[0098] In this embodiment, training the prediction model specifically includes:

[0099] Determine the features and feature dimensions input into the prediction model based on the extracted features;

[0100] Randomly extract 70%, 20%, and 10% of the power load related data as training set, test set, and validation set respectively;

[0101] Determine the input step size, number of encoders and number of decoders of the prediction model;

[0102] Determine the prediction step size of the prediction model;

[0103] Determine the size of data that the prediction model processes at one time, the discard rate, and the training cycle;

[0104] Determine the Adam optimizer and ensure that the initial learning rate is adjusted between [0, 1];

[0105] Gelu is used as the activation function; the root mean square error RMSE is used as the model evaluation standard. In each iteration, the parameters of the prediction model are adjusted according to the root mean square error between the prediction result and the true value, so that after multiple trainings, the prediction result of the prediction model can continue to approach the true value.

[0106] In this embodiment, the power load is predicted according to the time series characteristic information, specifically including:

[0107] Input the time series feature information into the fully connected layer of the prediction model to obtain the output features of the fully connected layer;

[0108] The output features of the fully connected layer are input into the activation layer of the prediction model, and the power load prediction results are output.

[0109] The present invention provides a method for power load forecasting based on an improved Transformer, which improves the attention mechanism of the traditional Transformer model to an additive attention mechanism, and uses root mean square layer normalization to replace traditional layer normalization, thereby achieving more efficient feature extraction and data processing capabilities. The present invention makes full use of multi-source data, including historical power load data, weather data, temperature and humidity data, and agricultural activity data, and fuses these data through a feature embedding layer, thereby improving the model's ability to express complex time series features and significantly improving the accuracy and stability of power load forecasting. The present invention not only has high prediction accuracy, but also has good generalization ability, and can be widely used in load forecasting and management of power systems, providing a scientific basis for power dispatching and planning.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A power load forecasting method based on an improved Transformer algorithm, characterized in that: include: Collect data related to power load; The power load related data includes historical power load data and environmental information data; Preprocessing the power load related data to obtain processed data; Extract features from the processed data and perform feature fusion to obtain a fused feature vector; Extracting time series feature information from the fused feature vector; The power load is predicted according to the time series characteristic information to obtain the prediction result.

2. The power load forecasting method based on the improved Transformer algorithm according to claim 1 is characterized in that: The environmental information data includes weather data, temperature and humidity data, and agricultural activity data.

3. The power load forecasting method based on the improved Transformer algorithm according to claim 1 is characterized in that: Preprocessing of power load related data, including: Identify outliers and missing values ​​in power load related data; Use the average value of the time near the abnormal point to complete the data repair; The min-max standardization method is used to normalize the power load related data; Perform dimensionality-upgrading processing on the historical power load data, and upgrade the historical power load data in the original two-dimensional feature data format into a three-dimensional data format; Among them, the three-dimensional data format is [time_step, input_size, batch_size], time_step represents the time step, input_size represents the input feature dimension, and batch_size represents the number of samples.

4. The power load forecasting method based on the improved Transformer algorithm according to claim 2 is characterized in that: Perform feature extraction on the processed data, including: Using sliding window technology, a fixed-size window is slid on the historical power load data, and the statistics in each window are calculated. The maximum and minimum values ​​in the statistics are used as sequence features, and the long-term trend features are extracted from the historical power load data using the time series analysis method. Extract the weather status and take the features that have a significant impact on the load as weather features; Extract temperature and humidity values ​​to obtain temperature and humidity characteristics, and divide the temperature and humidity values ​​into different intervals to reflect the impact of different temperatures on the load; Agricultural activities are divided into different types according to their seasonal characteristics, and each type is assigned a unique code.

5. The power load forecasting method based on the improved Transformer algorithm according to claim 4 is characterized in that: The processed data is subjected to feature fusion, including: The time alignment method is used to ensure that the feature vector at each time point contains all kinds of features corresponding to that time point; The extracted sequence features, trend features, weather features, temperature and humidity features, and agricultural activity features are concatenated into a long feature vector in a set order; all concatenated feature vectors are normalized.

6. The power load forecasting method based on the improved Transformer algorithm according to claim 1 is characterized in that: Extracting time series feature information from the fused feature vector includes: Construct a prediction model based on the improved Transformer; the prediction model includes an encoder and a decoder; the encoder includes input embedding and position encoding, additive attention mechanism, residual connection and root mean square layer normalization and position forward propagation network; wherein, input embedding maps each input data to a word embedding vector; position encoding represents the position information of each word in the input sequence; additive attention mechanism is used to capture the relationship between each other; residual connection is used to solve the problem of gradient disappearance in model training; layer normalization is used to ensure that the model training is more stable; The decoder includes a masked self-attention layer, a residual connection and a root mean square layer normalization, and a position-by-position feedforward neural network; wherein the masked self-attention layer allows the decoder to only pay attention to all positions before the current position when processing the current position, so as to maintain the autoregressive characteristics; the residual connection and layer normalization are used to better train the model and solve the problem of vanishing and exploding gradients in the network; the position-by-position feedforward neural network is used to independently process the representation of each position; The prediction model is trained to obtain a trained prediction model; the fused feature vector is input into the trained prediction model and the following processing is performed: Suppose a query vector q and a set of key-value pairs (k1, v1), ..., (k i , v i ),…,(k n , v n ), and enhances the expressive power through a nonlinear activation function of the additive attention mechanism: f att (q,k i )=tanh(w a [q;k i ]), Among them, f att is the attention function; k i represents the key of the i-th key-value pair; w a is the weight matrix, [;] represents vector concatenation; tanh() represents the activation function; The fused feature vector is converted into attention weight a through a linear layer and a softmax function i , used to represent each value v i The calculation formula is as follows: Represents the weight matrix W corresponding to the linear layer v The transpose of According to the calculated attention weight a i Perform weighted summation on all vectors to obtain the final context vector c: The output vector c is processed in two ways. One side is input into the feedforward neural network layer and then normalized in the root mean square layer. The other side is not processed and then residually connected with the processed data. The formula is as follows: Among them, c i represents the i-th element of vector c; RMS(c) represents the normalization of vector c; g i represents the normalized scaling factor; It is the value after RMS normalization calculation.

7. The power load forecasting method based on the improved Transformer algorithm according to claim 6 is characterized in that: Training the prediction model specifically includes: Determine the features and feature dimensions input into the prediction model based on the extracted features; Randomly extract 70%, 20%, and 10% of the power load related data as training set, test set, and validation set respectively; Determine the input step size, number of encoders and number of decoders of the prediction model; Determine the prediction step size of the prediction model; Determine the size of data that the prediction model processes at one time, the discard rate, and the training cycle; Determine the Adam optimizer and ensure that the initial learning rate is adjusted between [0, 1]; Gelu is used as the activation function; the root mean square error RMSE is used as the model evaluation standard. In each iteration, the parameters of the prediction model are adjusted according to the root mean square error between the prediction result and the true value, so that after multiple trainings, the prediction result of the prediction model can continue to approach the true value.

8. The power load forecasting method based on the improved Transformer algorithm according to claim 6 is characterized in that: The power load is predicted based on the time series characteristic information, including: Input the time series feature information into the fully connected layer of the prediction model to obtain the output features of the fully connected layer; The output features of the fully connected layer are input into the activation layer of the prediction model, and the power load prediction results are output.

Images