A method for short-term power load forecasting based on hybrid RetNet and AM-BiLSTM algorithms

By combining RetNet and AM-BiLSTM algorithms, using the bidirectional LSTM model and self-attention mechanism, the problem of insufficient prediction accuracy of traditional methods under limited resources and time is solved, and more efficient and more accurate short-term power load prediction is achieved.

CN117350158BActive Publication Date: 2024-11-26HUBEI CENT CHINA TECH DEV OF ELECTRIC POWER +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311327368.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-13
Publication Date
2024-11-26
Estimated Expiration
2043-10-13

AI Technical Summary

Technical Problem

Traditional short-term power load prediction methods are difficult to effectively extract spatiotemporal and spatial characteristics and timing dependencies when resources and time are limited, resulting in insufficient prediction accuracy.

Method used

Using hybrid RetNet and AM-BiLSTM algorithms, combined with bidirectional LSTM model and RetNet to improve the residual network and self-attention mechanism, extract hidden features of the input data and analyze the correlation between the input information in the network and the current load, and optimize the spatiotemporal and time-series data. Dependencies between.

Benefits of technology

The accuracy of short-term load prediction is significantly improved, and the prediction accuracy and efficiency of the model are improved by reducing the average absolute percentage error and root mean square error.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117350158B_ABST
    Figure CN117350158B_ABST
Patent Text Reader

Abstract

The present invention provides a method for short-term power load forecasting of a hybrid RetNet and AM‑BiLSTM algorithm, comprising: step 1, obtaining the historical data of power load from a data center; step 2, preprocessing the acquired historical data of power load; step 3, analyzing the time series characteristics of the preprocessed historical data of power load, and combining external factors and historical data to form model input data; step 4, using the load forecasting model of the hybrid RetNet and AM‑BiLSTM algorithm to forecast the future short-term daily 24-hour power load of the constructed model input data; step 5, using the mean absolute percentage error MAPE and the root mean square error RMSE as evaluation indicators for evaluating the accuracy of the load forecasting model, and evaluating the load forecasting model according to the prediction result of step 4. The present invention can improve the prediction accuracy of the short-term load forecasting model, so as to predict a more accurate short-term power load.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence algorithms, and specifically to a method for short-term power load forecasting that combines RetNet and AM-BiLSTM deep learning algorithms. Background Art

[0002] Load forecasting plays a key role in power dispatching and marketing, providing basic data for basic power generation planning, computer online power grid control, system safety analysis, day-ahead market quotations, intraday (real-time) market transactions, etc., thereby making power dispatching more in line with reality and reducing the deviation between reported power consumption and actual power consumption. Therefore, power load forecasting, especially short-term load forecasting, is particularly important.

[0003] Traditional load forecasting methods include neural network method, time series method, regression analysis method, fuzzy forecasting method, etc. In the time series method, the spatiotemporal features of time series data and the temporal dependencies between time series data have a great impact on the accuracy of prediction. When RetNet is used to extract spatiotemporal features, the GPU memory usage and inference latency are stable when the input sequence length increases, which means that more prediction tasks can be completed with less resources and time. BiLSTM and LSTM can be used to obtain temporal dependencies, but BiLSTM can process both forward and reverse input sequences at the same time, and can better capture the dependencies between data to make more accurate predictions. When resources and time are limited, improving the efficiency and accuracy of extracting features and obtaining temporal dependencies is worry-free and labor-saving. Summary of the invention

[0004] In view of this, the present invention provides a method for short-term load forecasting of electric power by hybrid RetNet and AM-BiLSTM algorithms, adopting a bidirectional LSTM model to make up for the shortcoming that a single LSTM model only considers the historical information of time series data and ignores future information; introducing a RetNet multi-layer improved residual network to extract hidden features of input data; adding an attention mechanism to analyze the correlation between input information and current load in the network and highlighting the influence of important information; optimizing the efficiency of extracting spatiotemporal features and obtaining the dependency relationship between time series data and combining external factors with load data characteristics to improve the prediction accuracy of the short-term load forecasting model.

[0005] A method for short-term power load forecasting based on a hybrid RetNet and AM-BiLSTM algorithm includes the following steps:

[0006] Step 1: Obtain a user's one-year electricity load history data from the data center, including date and hourly load data;

[0007] Step 2: preprocessing the historical power load data obtained in step 1, wherein the preprocessing includes supplementing blank values, correcting abnormal data, and normalizing the data;

[0008] Step 3: Analyze the time series characteristics of the historical power load data preprocessed in step 2, analyze the external factors that can affect the load data according to the load change curve, and combine the external factors and historical data to form the model input data;

[0009] Step 4: Use the load forecasting model of the hybrid RetNet and AM-BiLSTM algorithm to forecast the future short-term daily 24-hour power load on the model input data constructed in step 3;

[0010] Step 5: Use mean absolute percentage error (MAPE) and root mean square error (RMSE) as evaluation indicators to evaluate the accuracy of the load forecasting model, and evaluate the load forecasting model based on the prediction results of step 4.

[0011] Furthermore, the preprocessing of step 2 specifically includes: if a blank value is encountered, according to the construction mapping method, according to the similarity of the rules of time series data, construct a mapping of the existing data to repair the missing data; correct the abnormal data through horizontal and vertical processing methods; finally, normalize the data and process the data to the same order of magnitude.

[0012] Furthermore, the step three specifically includes:

[0013] According to the continuous load change curve after preprocessing, it is concluded that the weekly periodicity shows that the electricity consumption trends of users on weekdays and weekends are different, the daily periodicity shows that the electricity consumption trends of users in the early morning and during the day are different, and the monthly periodicity shows that the electricity consumption trends of users in different seasons are different; based on this, the input data of the designed model are: the h-th hour load value, temperature value and humidity value that need to be predicted on the same day in the 6 months before the prediction date; the h-th hour load value, temperature value and humidity value that need to be predicted on the same day in the 4 weeks before the prediction date; the h-th hour load value, temperature value and humidity value of each day in the week before the prediction date; the load value 24 hours before the prediction date; the actual temperature value and humidity value at the h-th hour on the prediction day; season coding: spring, summer, autumn and winter are marked with 1, 2, 3, and 4; weekday and weekend coding: 0 is weekday, 1 is weekend; holiday and non-holiday coding: 0 is holiday, 1 is non-holiday.

[0014] Furthermore, the load forecasting model in step 4 is based on the basic bidirectional BiLSTM model, and combines the Retnet improved residual network and the self-attention mechanism AM with the BiLSTM model network structure to form a hybrid RetNet and AM-BiLSTM algorithm power short-term load forecasting model, wherein the Retnet improved residual network is composed of L RetNet blocks stacked, and each RetNet block contains two modules: multi-scale retention and feedforward network; the AM-BiLSTM network model combining the self-attention mechanism AM with the BiLSTM model network structure has a total of seven layers: the first layer is the input layer, which inputs the model variables; the second layer is the embedding layer, which maps the input data to a low-dimensional space; the third layer is the BiLSTM layer, which obtains the high-level features of the input data; the fourth layer is the Attention layer, which calculates the weight value of each feature and integrates it into the final feature vector; the fifth layer is the embedding layer, which converts the dimension of the feature to input it into the sixth LSTM layer; the sixth layer is the LSTM layer, which performs power load forecasting according to the final feature vector, and the last layer is the output layer, which outputs the predicted value.

[0015] Furthermore, step four specifically includes: inputting the model input data constructed in step three into the basic model structure composed of a fully connected layer, using SELU as an activation function, and obtaining a preliminary predicted daily load after the basic model structure is continuously fully connected; then, the preliminary predicted daily load is used as the input of the Retnet improved residual network module, and a multi-layer improved residual network is used to extract the hidden features of the input data and generate a feature vector; finally, the feature vector is input into the AM-BiLSTM network model, and the final predicted value is output through the AM-BiLSTM network model.

[0016] The present invention adopts a hybrid RetNet and AM-BiLSTM load forecasting combined model to truly predict the short-term power load in the short term. It optimizes the efficiency of extracting spatiotemporal features and obtaining the dependency relationship between time series data, as well as the combination of external factors and load data characteristics, to improve the prediction accuracy of the short-term load forecasting model, so as to predict a more accurate short-term power load. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of a method for predicting short-term power load based on a hybrid RetNet and AM-BiLSTM algorithm according to an embodiment of the present invention;

[0018] Figure 2 is a data table after data preprocessing according to an embodiment of the present invention;

[0019] Figure 3 It is a structural diagram of a load forecasting model of a hybrid RetNet and AM-BiLSTM algorithm constructed in an embodiment of the present invention;

[0020] Figure 4 This is a diagram of the AM-BiLSTM network structure of an embodiment of the present invention;

[0021] Figure 5 It is a comparison chart between the load forecasting model constructed by the embodiment of the present invention and the traditional LSTM forecasting results. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, the embodiment of the present invention provides a method for short-term power load forecasting by hybrid RetNet and AM-BiLSTM algorithms, which is characterized by adopting a combined model of RetNet and BiLSTM and adding an attention mechanism. Compared with the model based on LSTM alone, the prediction accuracy is greatly improved. The whole process includes acquiring historical data of power load, data preprocessing, data characteristic analysis, building a load forecasting model, evaluating the model and verification, and result analysis. The specific implementation process steps are as follows:

[0024] Step 1. Get the data

[0025] The data of the model comes from the data center.

[0026] This embodiment mainly shows the historical electricity load data of a company in 2022, and the fields are company name, data date, and hourly electricity meter measurement.

[0027] Step 2. Data preprocessing

[0028] Data preprocessing is divided into the following steps:

[0029] (1) Outlier processing: Correct abnormal data through horizontal and vertical processing methods;

[0030] (2) Filling in missing values: Based on the construction mapping method, according to the similarity of the time series data rules, the mapping of the existing data to the missing data is constructed to repair the missing data;

[0031] (3) Data normalization: To eliminate the influence of dimension, the data is normalized. The formula is as follows:

[0032]

[0033] After data preprocessing, the format is as follows Figure 2 As shown, E1 to E24 represent 24-hour electricity meter measurement data.

[0034] Step 3. Analyze data characteristics and construct model input data

[0035] The time series characteristics of the pre-processed historical data of power load are analyzed, and the external factors that can affect the load data are analyzed according to the load change curve. The external factors and historical data are combined to form the model input. Observing the continuous load change curve after pre-processing, it is found that weekly periodicity, daily periodicity, and monthly periodicity have an impact on the user's power load; at the same time, the temperature and humidity on different dates also have an impact on the user's power load. Therefore, we design the model input data as follows: the load value, temperature value, and humidity value of the hth hour that needs to be predicted on the same day in the 6 months before the forecast date; the load value, temperature value, and humidity value of the hth hour that needs to be predicted on the same day in the 4 weeks before the forecast date; the load value, temperature value, and humidity value of the hth hour of each day in the week before the forecast date; the load value of the 24 hours before the forecast date; the actual temperature value and humidity value of the hth hour on the forecast date; season code (spring, summer, autumn, and winter are marked with 1, 2, 3, and 4); weekday and weekend code (0 is a weekday, 1 is a weekend); holiday and non-holiday code (0 is a holiday, 1 is a non-holiday).

[0036] Finally, the model input data structure is shown in Table 1:

[0037] Table 1 Model input data structure table

[0038]

[0039] Step 4. Build a load forecasting model and perform load forecasting

[0040] After the previous three steps, the preparations are ready. Here we explain the network structure of the load forecasting model of the hybrid RetNet and AM-BiLSTM algorithm. The load forecasting model is based on the basic bidirectional BiLSTM model, combining the Retnet improved residual network and the self-attention mechanism AM with the BiLSTM model network structure to form a hybrid RetNet and AM-BiLSTM algorithm power short-term load forecasting model, which solves the problems of many model iterations, feature disappearance, and network degradation, improves the model's back propagation capability, more accurately mines data information, and improves the accuracy of the prediction model.

[0041] This model uses the pytorch framework. When performing load forecasting, first, the model input data constructed in step 3 is input into the basic model structure composed of fully connected layers, and SELU is used as the activation function. After the basic model structure is continuously fully connected, the preliminary predicted daily load is obtained; then, the preliminary predicted daily load is used as the input of the Retnet improved residual network module, and a multi-layer improved residual network is used to extract the hidden features of the input data and generate a feature vector; finally, the feature vector is input into AM-BiLSTM, and the final predicted value is output through AM-BiLSTM. The overall network structure is as follows: Figure 3 shown.

[0042] The improved residual network of Retnet is composed of L stacked RetNet blocks, each of which contains two modules: Multi-Scale Retention (MSR) and Feed-Forward Network (FFN), which is similar to residual connection and pre-LayerNorm. Retnet uses a multi-head retention mechanism to build a multi-scale retention module, which fully integrates the innovation of the Multi-Scale Retention module and the advantages of the Transformer architecture, which makes RetNet have linear memory complexity for long sequences and realizes efficient reasoning of recursive networks.

[0043] The AM-BiLSTM network model combines the AM attention mechanism with the bidirectional time series prediction BiLSTM model. The bidirectional LSTM model adds a reverse LSTM model on top of the basic LSTM model to solve the shortcoming that the model cannot backpropagate. The forward LSTM obtains the historical information of the input data, and the backward LSTM obtains the future information of the input data. The added attention mechanism pays different attention to the input features at different learning times according to the weight value of the input data, thereby improving the computational efficiency. The AM-BiLSTM network model has a total of seven layers: the first layer is the input layer, which inputs the model variables; the second layer is the embedding layer, which maps the input data to a low-dimensional space; the third layer is the BiLSTM layer, which obtains the high-level features of the input data; the fourth layer is the Attention layer, which calculates the weight value of each feature and integrates it into the final feature vector; the fifth layer is the embedding layer, which converts the dimension of the feature to input it into the sixth LSTM layer; the sixth layer is the LSTM layer, which performs power load forecasting based on the final feature vector, and the last layer is the output layer, which outputs the predicted value. The AM-BiLSTM network structure is as follows: Figure 4 shown.

[0044] Step 5. Evaluate the model and validate

[0045] After the load forecasting model is built, the data processed in step 3 is divided into a training set and a test set in a ratio of 9:1. The division is shown in Table 2:

[0046] Table 2

[0047] Dataset Training set Test Set Historical electricity consumption of the company 20220101~20221011 20221012~20221231

[0048] Step 6. Result analysis

[0049] For the above data sets, the prediction results of the hybrid RetNet and AM-BiLSTM network models are compared with the traditional LSTM prediction results. Figure 5 shown.

[0050] The prediction performance evaluation index results are shown in Table 3:

[0051] Table 3

[0052]

[0053] The experimental results show that the two evaluation indicators of the hybrid RetNet and AM-BiLSTM network model are: the mean relative error (MAPE) is 2.83%, and the root mean square error (RMSE) is 0.6869, which are reduced by 2.62% and 0.8731 respectively compared with the traditional LSTM prediction model. Compared with the traditional LSTM network model, the hybrid RetNet and AM-BiLSTM network model proposed in the present invention has improved in both the mean relative error (MAPE) and the root mean square error (RMSE). It shows that the hybrid RetNet and AM-BiLSTM network model can perform efficient and high-quality data information mining on the user's electricity consumption historical load data, and accurately extract the feature relationship between time series data, thereby significantly solving the problem of low prediction accuracy of a single neural network; the introduction of Retnet to improve the residual network structure improves the performance of the model, so that the back propagation efficiency of the model is greatly improved; it is significantly improved in terms of depth, feature extraction, and time.

[0054] The present invention adopts a bidirectional LSTM model to make up for the shortcoming that a single LSTM model only considers the historical information of time series data and ignores future information, thereby realizing more accurate mining of data information; introduces the RetNet multi-layer improved residual network to extract the hidden features of the input data, effectively overcomes the network degradation problem caused by the deepening of the number of hidden layers of the network, and improves the model's back propagation capability; at the same time, adds an attention mechanism to analyze the correlation between the input information in the network and the current load and highlight the influence of important information, thereby improving the speed and accuracy of the model.

[0055] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for short-term power load forecasting based on hybrid RetNet and AM-BiLSTM algorithms. It is characterized in that The steps include: Step 1: Obtain a user's one-year electricity load history data from the data center, including date and hourly load data; Step 2: preprocessing the historical power load data obtained in step 1, wherein the preprocessing includes supplementing blank values, correcting abnormal data, and normalizing the data; Step 3: Analyze the time series characteristics of the historical power load data preprocessed in step 2, analyze the external factors that can affect the load data according to the load change curve, and combine the external factors and historical data to form the model input data; Step 4: Use the load forecasting model of the hybrid RetNet and AM-BiLSTM algorithm to forecast the future short-term daily 24-hour power load on the model input data constructed in step 3; Step 5: Use mean absolute percentage error (MAPE) and root mean square error (RMSE) as evaluation indicators for evaluating the accuracy of the load forecasting model, and evaluate the load forecasting model based on the prediction results of step 4; The preprocessing of step 2 specifically includes: if a blank value is encountered, according to the construction mapping method, according to the similarity of the time series data rules, constructing a mapping of the existing data to repair the missing data; correcting the abnormal data through horizontal and vertical processing methods; and finally normalizing the data to process the data to the same order of magnitude; The load forecasting model in step 4 is based on the basic bidirectional BiLSTM model. The Retnet improved residual network and the self-attention mechanism AM are combined with the BiLSTM model network structure to form a hybrid RetNet and AM-BiLSTM algorithm power short-term load forecasting model, in which the Retnet improved residual network is composed of L RetNet blocks stacked, and each RetNet block contains two modules: multi-scale retention and feedforward network; the AM-BiLSTM network model combining the self-attention mechanism AM with the BiLSTM model network structure has a total of seven layers: the first layer is the input layer, which inputs the model variables; the second layer is the embedding layer, which maps the input data to a low-dimensional space; the third layer is the BiLSTM layer, which obtains the high-level features of the input data; the fourth layer is the Attention layer, which calculates the weight value of each feature and integrates it into the final feature vector; the fifth layer is the embedding layer, which converts the dimension of the feature to input it into the sixth layer of the LSTM layer; the sixth layer is the LSTM layer, which performs power load forecasting according to the final feature vector, and the last layer is the output layer, which outputs the predicted value; The step three specifically includes: According to the continuous load change curve after preprocessing, it is concluded that: the weekly periodicity shows that the electricity consumption trends of users on weekdays and weekends are different, the daily periodicity shows that the electricity consumption trends of users in the early morning and during the day are different, and the monthly periodicity shows that the electricity consumption trends of users in different seasons are different; based on this, the input data of the design model are: the load value, temperature value and humidity value of the hth hour that need to be predicted on the same day in the 6 months before the forecast date; the load value, temperature value and humidity value of the hth hour that need to be predicted on the same day in the 4 weeks before the forecast date; the load value, temperature value and humidity value of the hth hour of each day in the week before the forecast date; the load value of the 24 hours before the forecast date; the actual temperature value and humidity value of the hth hour on the forecast day; season coding: spring, summer, autumn and winter are marked with 1, 2, 3, and 4; weekday and weekend coding: 0 is weekday, 1 is weekend; holiday and non-holiday coding: 0 is holiday, 1 is non-holiday; Step 4 specifically includes: inputting the model input data constructed in step 3 into the basic model structure composed of fully connected layers, using SELU as the activation function, and obtaining the preliminary predicted daily load after the basic model structure is continuously fully connected; then, the preliminary predicted daily load is used as the input of the Retnet improved residual network module, and a multi-layer improved residual network is used to extract the hidden features of the input data and generate a feature vector; finally, the feature vector is input into the AM-BiLSTM network model, and the final predicted value is output through the AM-BiLSTM network model.