Electrical load prediction method and system based on Transfomer decoder

Abstract

The invention discloses an electrical load prediction method and system based on a Transfomer decoder. The method comprises the following steps: collecting historical electrical load data from a database of a power system; the collected data are preprocessed, and abnormal values and missing values are removed; the method comprises the following steps: constructing a network architecture of an improved Transform decoder model; according to the improved Transform decoder network architecture, a full connection layer is adopted to replace an original embedded layer, a multi-head self-attention mechanism is added to a decoder layer, and sine and cosine position coding is replaced with rotation position coding; setting hyper-parameters, selecting a proper loss function, training the constructed model, and optimizing model parameters by using a back propagation algorithm until the model converges; containerizing the trained model, and deploying the model as a Web service; predicting a future power load by using the deployed model; compared with the traditional method, the method has the advantages that the capturing capability of the model on the long-term dependency relationship and the time sequence information is obviously enhanced, so that the accurate prediction on the future power load is realized.

Description

Technical Field

[0001] The invention relates to a method and system for predicting power load, and in particular to a method and system for predicting power load based on a Transfomer decoder, and belongs to the technical field of power grid power load prediction. Background Art

[0002] Short-term power load forecasting is crucial to ensure the stable operation of the power grid, especially in the smart grid environment. With the widespread access to new energy and the complex changes in power load, accurate prediction of short-term power load becomes particularly critical. Although traditional forecasting methods such as regression analysis and time series methods can process historical data to a certain extent, they are difficult to cope with the highly uncertain and strongly nonlinear modern power grid status data. In addition, these methods usually require detailed analysis of historical data characteristics and construction of mathematical models. This process seems to be inadequate when faced with the complexity and variability of modern power grid data.

[0003] In recent years, the rapid development of deep learning technology has provided new ideas and technical means for power load forecasting. In particular, LSTM (Long Short-Term Memory Network) and its variants have performed well in processing time series data. LSTM is a special recurrent neural network (RNN) that can remember long-term dependency information and is suitable for processing data with time sequence, such as power load data. However, although LSTM and its variants have achieved certain results in processing time series prediction problems, there are still some limitations: the parameter adjustment of the LSTM model is relatively complex and requires careful design to avoid overfitting or underfitting problems. Although the original intention of LSTM was to overcome the gradient vanishing problem in traditional RNN, long-distance dependency problems may still occur when processing very long time series. It is difficult to achieve efficient parallel computing in the LSTM model during training because the output at each moment depends on the state of the previous moment.

[0004] With the improvement of computing power and the increased availability of large-scale datasets, large-scale Transformer-based models have become a mainstream trend in the field of deep learning. The core of the Transformer model is its attention mechanism, which can capture the dependencies between different positions in the input sequence and is very suitable for processing time series data. In particular, large-scale Transformer-based models have made breakthrough progress in fields such as natural language processing, which has also brought new insights and opportunities to other fields, including power load forecasting. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a method and system for predicting power load based on a Transfomer decoder which can improve prediction accuracy.

[0006] Technical solution: The power load forecasting method based on Transfomer decoder described in the present invention comprises:

[0007] (1) Collect historical power load data from the power system database;

[0008] (2) Preprocess the collected data to remove outliers and missing values;

[0009] (3) constructing a network architecture of an improved Transformer decoder model; the improved Transformer decoder network architecture uses a fully connected layer to replace the original embedding layer, adds a multi-head self-attention mechanism to the decoder layer, and uses rotation position encoding to replace sine and cosine position encoding;

[0010] (4) Set hyperparameters, select appropriate loss functions, train the constructed model, and use the back propagation algorithm to optimize the model parameters until the model converges;

[0011] (5) Containerize the trained model and deploy it as a web service;

[0012] (6) Use the deployed model to predict future power load.

[0013] Furthermore, the step (1) includes setting a collection time range, the collection time range is greater than or equal to 3 years, setting the collection time interval of each day to 15 minutes, generating 96 sampling points every day, collecting historical power load data, and recording the power load of the corresponding timestamp.

[0014] Furthermore, the removal of missing values ​​in step (2) is specifically as follows: when the power load data in the collected data is empty, the corresponding data is removed;

[0015] In the process of removing outliers, the outlier determination method is as follows:

[0016]

[0017] Among them, x n. i represents the power load at the i-th time point on the n-th day, It represents the average power load at the i-th time point on the n-th day, where N is the total number of days. represents the variance at the i-th time point, σ i represents the standard deviation at the i-th time point; when Then we think x n.i is an outlier.

[0018] Furthermore, the step (3) includes constructing an improved Transformer decoder network using the PyTorch framework, adding a multi-head self-attention mechanism to the decoder layer to capture long-term dependencies in the input sequence; using rotational position encoding instead of sine-cosine position encoding; and using a fully connected layer in the input layer instead of the original embedding layer to process the preprocessed data.

[0019] Furthermore, the number of heads of the multi-head self-attention mechanism is set to 4; the number of hidden units of the fully connected layer is set to 512, and the number of decoder layers is set to 4.

[0020] Furthermore, step (4) includes dividing the data set into a training set, a validation set and a test set; using the training set to train the constructed model, using the validation set to adjust parameters, and using the test set to test the model performance.

[0021] Furthermore, the model constructed by training the training set is trained, and the error in the training process is measured by the loss function mean square error MSE, and the mean square error calculation method is:

[0022]

[0023] where y i represents the actual power load at the i-th moment of the day, Indicates the predicted power load at the i-th moment of the day; the Adam optimizer is used to update the parameters, and the training data sliding window is set to 96. 96 consecutive data points are input into the model each time to predict the power load at the next moment.

[0024] Furthermore, step (5) includes using the Flask framework to build a Web service, wherein the Web service includes functions of loading a trained model, receiving HTTP requests, and performing predictions and returning results; building a Docker image including a Flask application and the required operating environment, using Docker commands to build the image and run the container, starting the Web service, deploying the Docker container to a server or cloud, and providing an HTTP interface calling function.

[0025] Furthermore, the step (6) includes using the deployed model to make predictions; when the predicted demand is the electricity load at a certain moment, the model input is the data of 96 sampling points before this moment; when the predicted demand is the electricity consumption for a certain period of time, multiple iterative reasonings are performed, and the predicted data is used as the last sampling point, and the first sampling point is eliminated until the iterative prediction is stopped within the preset prediction range.

[0026] Based on the same inventive concept, the present invention also provides a power load forecasting system based on a Transfomer decoder, comprising:

[0027] A collection module, used to collect historical power load data from the database of the power system;

[0028] The preprocessing module is used to preprocess the collected data and remove abnormal values ​​and missing values;

[0029] A model building module, used to build a network architecture of an improved Transformer decoder model; the improved Transformer decoder network architecture uses a fully connected layer to replace the original embedding layer, adds a multi-head self-attention mechanism to the decoder layer, and uses rotation position encoding to replace sine and cosine position encoding;

[0030] The model training module is used to set hyperparameters, select appropriate loss functions, train the constructed model, and optimize the model parameters using the back propagation algorithm until the model converges;

[0031] The model deployment module is used to containerize the trained model and deploy it as a web service;

[0032] A prediction module, used to predict future power load using the deployed model;

[0033] The display module is used to provide a user interface and display the prediction results on the user interface.

[0034] Beneficial effects: Compared with the existing technology, the present invention significantly enhances the model's ability to capture long-term dependencies and timing information by adopting a simple and efficient numerical data processing method, a multi-head self-attention mechanism, and a rotational position encoding technology, thereby achieving accurate prediction of future power loads; the system of the present invention integrates functions such as data collection and prediction display, supports the intelligent management of the power system, and provides strong technical support for power dispatching and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of the core structure of the Transformer decoder model of an embodiment of the present invention;

[0037] Figure 3 It is a curve diagram of predicted power load and actual power load of an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the technical problems, technical solutions and beneficial effects to be solved by the embodiments of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The specific embodiments described here are only used to explain the present invention and do not limit the present invention.

[0039] As attached Figure 1 As shown, the power load prediction method based on the Transformer decoder of this embodiment includes:

[0040] (1) Collect historical power load data from the power system database;

[0041] (2) Preprocess the collected data to remove outliers and missing values;

[0042] (3) constructing a network architecture of an improved Transformer decoder model; the improved Transformer decoder network architecture uses a fully connected layer to replace the original embedding layer, adds a multi-head self-attention mechanism to the decoder layer, and uses rotation position encoding to replace sine and cosine position encoding;

[0043] (4) Set hyperparameters, select appropriate loss functions, train the constructed model, and use the back propagation algorithm to optimize the model parameters until the model converges;

[0044] (5) Containerize the trained model and deploy it as a web service;

[0045] (6) Use the deployed model to predict future power load.

[0046] Specifically, in step (1), data acquisition: historical power load data is exported from the power system database and stored in CSV format. The data consists of timestamp and power load columns. Sampled data is generated daily, sampled every 15 minutes, and the corresponding timestamp and power load are recorded. A total of 96 sampling points are generated in one day. The power load data of the past three years is extracted for model training to ensure that the model fully captures and learns the data trends.

[0047] Data preprocessing in step (2): Clean and organize the original data to remove missing values ​​and outliers. The rule for removing missing values ​​is to remove the corresponding records when the power load data in the CSV file is empty. The standard deviation is used to measure the degree of dispersion of data distribution. The rule for removing outliers is to use the 3σ principle to calculate the mean (μ) and standard deviation (σ) of all data points at the same time point every day. The standard deviation is a statistical indicator used to measure the degree of dispersion of data distribution. According to the 3σ principle, if it falls outside μ±3σ, it is marked as an outlier. The formula for calculating the mean and variance is:

[0048]

[0049] x n. i represents the amount of electricity at the i-th time point on the n-th day, It represents the average power at the i-th time point on the n-th day, which is equivalent to μ. represents the variance at the i-th time point, σ i Represents the standard deviation at the i-th time point. Satisfying the formula That is, x n. i is an outlier.

[0050] Model construction in step (3): Figure 2 As shown, an improved Transformer decoder network is constructed using the PyTorch framework. The decoder layer of the network contains a multi-head self-attention mechanism (Multi-Head Self-Attention) to capture long-term dependencies in the input sequence. In the present invention, the number of heads of the multi-head attention mechanism is set to 4, and Rotary Positional Encoding (RPE) is used to better characterize the time series characteristics in the power load data. The input layer uses a fully connected layer (Linear Layer) instead of a traditional embedding layer to directly process preprocessed numerical data. The number of hidden units in the fully connected layer is set to 512, and the number of decoder layers is set to 4.

[0051] Model training in step (4): define the loss function and use mean square error (MSE), where the formula for MSE is y i represents the actual power load at the i-th moment of the day, Indicates the predicted power load at the i-th moment of the day. Use the Adam optimizer to update parameters. During training, the data sliding window is set to 96, and 96 consecutive data points are input to the model each time to predict the power load at the next moment. The data set is divided into training set, validation set, and test set. Use the training set to train the model, use the validation set to adjust the parameters, and finally evaluate the model performance on the test set to ensure the generalization ability and prediction accuracy of the model. Monitor the loss changes during the training process to ensure that the model converges effectively.

[0052] Containerized deployment in step (5): Containerize the trained model and deploy it as a Web service. Create a Flask application, define an interface for processing POST requests, load the model and return the prediction results. Write a Dockerfile to build a Docker image, run the Docker container to start the Flask application, and deploy the container to a server or cloud platform. Through the HTTP interface, external systems can easily call the prediction service.

[0053] Prediction and evaluation in step (6): Use the deployed model to predict the power load at a certain time in the future or within a few hours. When the power load at a certain time needs to be predicted, the model input is the data of the 96 sampling points before this time; when predicting the power load of a few hours, the model needs to perform multiple iterative reasoning, taking the predicted data as the last sampling point, excluding the first sampling point, and so on, until the iterative prediction is stopped within the preset prediction range. The comparison results of the power load prediction and actual power load in this embodiment are as follows: Figure 3 shown.

[0054] Based on the same inventive concept, this embodiment also provides a power load forecasting system based on a Transfomer decoder, comprising:

[0055] A collection module, used to collect historical power load data from the database of the power system;

[0056] The preprocessing module is used to preprocess the collected data and remove abnormal values ​​and missing values;

[0057] A model building module, used to build a network architecture of an improved Transformer decoder model; the improved Transformer decoder network architecture uses a fully connected layer to replace the original embedding layer, adds a multi-head self-attention mechanism to the decoder layer, and uses rotation position encoding to replace sine and cosine position encoding;

[0058] The model training module is used to set hyperparameters, select appropriate loss functions, train the constructed model, and optimize the model parameters using the back propagation algorithm until the model converges;

[0059] The model deployment module is used to containerize the trained model and deploy it as a web service;

[0060] A prediction module, used to predict future power load using the deployed model;

[0061] The display module is used to provide a user interface on which the forecast results are displayed, including current power load, comparison with the same period in previous years, voltage level, number of transformers, total transformer capacity, active power, reactive power, ambient temperature and humidity, and water level status indicators.

Claims

1. A method for predicting power load based on Transfomer decoder, characterized in that: include: (1) Collect historical power load data from the power system database; (2) Preprocess the collected data to remove outliers and missing values; (3) constructing a network architecture of an improved Transformer decoder model; the improved Transformer decoder network architecture uses a fully connected layer to replace the original embedding layer, adds a multi-head self-attention mechanism to the decoder layer, and uses rotation position encoding to replace sine and cosine position encoding; (4) Set hyperparameters, select appropriate loss functions, train the constructed model, and use the back propagation algorithm to optimize the model parameters until the model converges; (5) Containerize the trained model and deploy it as a web service; (6) Use the deployed model to predict future power load.

2. The method for predicting power load based on Transfomer decoder according to claim 1, characterized in that: The step (1) includes setting a collection time range, the collection time range is greater than or equal to 3 years, setting a collection time interval of 15 minutes per day, generating 96 sampling points per day, collecting historical power load data, and recording the power load of the corresponding timestamp.

3. The power load forecasting method based on Transfomer decoder according to claim 2 is characterized in that: The specific method of removing missing values ​​in step (2) is that the rule is to remove the corresponding data when the power load data in the collected data is empty; In the process of removing outliers, the outlier determination method is as follows: Among them, x n. i represents the power load at the i-th time point on the n-th day, It represents the average power load at the i-th time point on the n-th day, where N is the total number of days. represents the variance at the i-th time point, σ i represents the standard deviation at the i-th time point; when Then we think x n.i is an outlier.

4. The method for predicting power load based on Transfomer decoder according to claim 1, characterized in that: The step (3) includes building an improved Transformer decoder network using a PyTorch framework, adding a multi-head self-attention mechanism to the decoder layer to capture long-term dependencies in the input sequence; using rotation position encoding instead of sine-cosine position encoding; A fully connected layer is used in the input layer to replace the original embedding layer to process the preprocessed data.

5. The method for predicting power load based on Transfomer decoder according to claim 4, characterized in that: The multi-head self-attention mechanism has 4 heads; the number of hidden units in the fully connected layer is set to 512, and the number of decoder layers is set to 4.

6. The method for predicting power load based on Transformer decoder according to claim 1, characterized in that: The step (4) includes dividing the data set into a training set, a validation set and a test set; using the training set to train the constructed model, using the validation set to adjust parameters, and using the test set to test the model performance.

7. The method for predicting power load based on Transformer decoder according to claim 6, characterized in that: The model constructed by training the training set is trained, and the error in the training process is measured by the loss function mean square error MSE. The mean square error calculation method is: where y i represents the actual power load at the i-th moment of the day, Indicates the predicted power load at the i-th moment of the day; the Adam optimizer is used to update the parameters, and the training data sliding window is set to 96. 96 consecutive data points are input into the model each time to predict the power load at the next moment.

8. The method for predicting power load based on Transfomer decoder according to claim 1, characterized in that: The step (5) includes using the Flask framework to build a Web service, wherein the Web service includes functions of loading a trained model, receiving HTTP requests, and performing predictions and returning results; building a Docker image including a Flask application and the required operating environment, using Docker commands to build the image and run the container, starting the Web service, deploying the Docker container to a server or cloud, and providing an HTTP interface calling function.

9. The method for predicting power load based on Transformer decoder according to claim 1, characterized in that: The step (6) includes using the deployed model to make predictions; when the predicted demand is the power load at a certain moment, the model input is the data of 96 sampling points before this moment; when the predicted demand is the power consumption for a certain period of time, multiple iterative reasonings are performed, and the predicted data is used as the last sampling point, and the first sampling point is eliminated until the iterative prediction is stopped within the preset prediction range.

10. A power load forecasting system based on Transfomer decoder, characterized in that: include: A collection module, used to collect historical power load data from the database of the power system; The preprocessing module is used to preprocess the collected data and remove abnormal values ​​and missing values; A model building module, used to build a network architecture of an improved Transformer decoder model; the improved Transformer decoder network architecture uses a fully connected layer to replace the original embedding layer, adds a multi-head self-attention mechanism to the decoder layer, and uses rotation position encoding to replace sine and cosine position encoding; The model training module is used to set hyperparameters, select appropriate loss functions, train the constructed model, and optimize the model parameters using the back propagation algorithm until the model converges; The model deployment module is used to containerize the trained model and deploy it as a web service; A prediction module, used to predict future power load using the deployed model; The display module is used to provide a user interface and display the prediction results on the user interface.

Images