A short-term power load prediction method based on TimesNet-Crossformer-LSTM

Abstract

The application discloses a short-term power load prediction method based on TimesNet-Crossformer-LSTM, which comprises the following steps: 1) performing feature screening on the imported data set, and performing normalization processing on the selected features; 2) dividing the training set, the validation set and the test set, and inputting the training set and the validation set into a model for training; 3) the model extracts information from the input time sequence, and extracts the relationship between the period and the inter-period in the time sequence; 4) the Adam algorithm is used to adjust the parameters of the neural network, the network is trained by using back propagation, and the training effect of the model parameters is evaluated by observing the loss value on the validation set; 5) according to the ablation experiment result, tanshark is selected as the activation function of the model; 6) the test set is input into the trained model, the point prediction result is obtained, and the prediction effect of the model is evaluated through four evaluation indexes. Compared with the existing method, the method has higher precision in the short-term power load prediction task.

Description

Technical Field

[0001] The present invention belongs to the technical field of short-term power load probability forecasting, and specifically relates to a short-term power load forecasting method based on TimesNet-Crossformer-LSTM. Background Art

[0002] Due to the limitations of existing technology, electricity cannot be stored in large quantities. Its generation, transmission, distribution, and consumption occur almost simultaneously. Therefore, the supply and demand sides of the power system must be constantly balanced. Therefore, efficient and accurate load forecasting not only facilitates the smooth operation of the power system, but also improves economic and social benefits, providing better power supply for people's normal lives and production. Generally, power load forecasts can be divided into short-term, medium-term, and long-term forecasts based on time spans and operational decisions, and their corresponding responsibilities also vary. Long-term forecasts are used for power system planning, medium-term forecasts are used to better maintain the power system, and short-term forecasts are responsible for providing more accurate load information for the daily operation of the power system.

[0003] In the past, many single models were applied to short-term power load forecasting. Among them, multivariate linear regression accomplishes forecasting by fitting the load curve. By continuously fitting the input and output variables, future variables are obtained, enabling short-term load forecasting. Time series analysis is a classic approach for load forecasting. This involves analyzing the random processes of historical load series, establishing corresponding parameters, and building a model. The model parameters are then estimated. Advances in machine learning have led to its widespread application in load forecasting, including support vector machines (SVMs) and random forests. The emergence of recurrent neural networks (RNNs) has spawned a range of models suitable for time series forecasting. RNNs can effectively memorize short-term data sequences, but are prone to gradient vanishing and gradient exploding problems when processing long sequences. Unlike RNNs, which repeatedly superimpose all input information, long short-term memory networks (LSTMs) can forget unimportant time series information. This approach not only addresses the gradient exploding problem of RNNs but also demonstrates advantages in memorizing long sequences. Consequently, LSTMs have been widely used in time-series power load forecasting. The Gated Recurrent Neural Network (GRU) merges the forget gate and input gate in the LSTM model, merging the cell state and hidden state, making its structure simpler than the LSTM. The Bidirectional Long Short-Term Memory (BiLSTM) neural network implements two LSTM training steps, forward and backward, for time series, further improving the globality and completeness of feature extraction. The Convolutional Neural Network (CNN) can share convolution kernels, offering advantages when processing multidimensional feature vectors. Through convolution and pooling, it can effectively mine and extract hidden information from historical data. The emergence of the Transformer, which uses a self-attention mechanism to weight different positions in a sequence, has gained significant advantages in time series forecasting and has achieved widespread application. The Crossformer uses a two-stage attention mechanism to extract relationships between features and time, and between features, achieving excellent results in high-dimensional time series forecasting. The Autoformer uses a seasonal-trend approach to decompose time series into seasonal and trend components, then uses the autocorrelation coefficient of the time series to identify the most relevant segments. The Informer and Pyraformer reduce complexity through sparse attention, but do not account for the multi-component nature of time series. The Time Two-Dimensional Variational Model (TimesNet) is not limited to the one-dimensionality of time series. It expands the one-dimensional time series data into two-dimensional space for analysis and extracts features within and between cycles, making it effective in long-term and short-term predictions. However, it is easy to ignore the relationship between features and time, and between features. Summary of the Invention

[0004] The purpose of the present invention is to address the deficiencies of the existing technology and provide a short-term power load forecasting method based on TimesNet-Crossformer-LSTM. The TimesNet module converts one-dimensional time series into two-dimensional space for analysis, extracts information from the two-dimensional tensor through 2D convolution, aggregates it into a one-dimensional tensor, and performs two-stage attention on the sequence to maximize the relationship between sequence and time, and feature and feature. Finally, the LSTM module is used to correct the prediction results, and to a certain extent, it helps solve the problems of gradient vanishing and gradient explosion in the overall model when processing long sequences, thereby achieving better effect and accuracy in the short-term power load forecasting task.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions.

[0006] A short-term power load forecasting method based on TimesNet-Crossformer-LSTM includes the following steps:

[0007] Step S1: feature screening is performed on the imported data set, and feature selection is performed using the Pearson correlation coefficient analysis method; and the selected features are normalized;

[0008] Step S2: Divide the filtered and processed data set into a training set, a validation set, and a test set in a ratio of 8:1:1, and input the training set and the validation set into the model for training;

[0009] Step S3: Use the TimesNet-Crossformer-LSTM model for training. Through the special structure of the model, the input one-dimensional time series is converted into a two-dimensional space for information extraction, and the relationship between the cycle and the cycle in the time series is extracted;

[0010] Step S4: Use the Adam algorithm to adjust the parameters of the neural network, use back propagation to train the network, and evaluate the training effect of the model parameters by observing the loss value on the validation set;

[0011] Step S5: Perform an activation function ablation experiment on the TimesNet-Crossformer-LSTM model and select tanshark as the activation function of the model;

[0012] Step S6: Input the test set into the trained model to obtain the point prediction result, and evaluate the model prediction effect using four evaluation indicators.

[0013] Furthermore, in step S1, the feature selection is performed using the Pearson correlation coefficient analysis method, and the formula of the Pearson correlation coefficient analysis method is as follows:

[0014]

[0015] In the above formula, {x i ,,i=1,2,···,n} and {y i ,,i=1,2,···,n} are two sets of sequences with length n, and is the average of two series of length n, r xy is the variable correlation coefficient.

[0016] The selected features are normalized, and the normalization formula is as follows:

[0017]

[0018] In the above formula, z is the original data in a certain dimension; z min is the minimum value of this dimension in the dataset; z max is the maximum value of this dimension in the data set; z′ is the data after z normalization.

[0019] Furthermore, in step S2, the training set and the validation set are input into the model for training. The input of the model is represented as X=[x1···x u-1 ,x i ···x t ] T , t represents the t-th training sample, t>i; and set batch size = 516.

[0020] Furthermore, the process of extracting the intra-cycle and inter-cycle relationships in the time series in step S3 is as follows:

[0021] First, the preprocessed time series is input into the TimesNet-Crossformer-LSTM model. The TimesNet module decomposes the original sequence into a one-dimensional time series through fast Fourier decomposition, and quickly calculates the periodicity of the sequence. The input data is then converted to the frequency domain to analyze the time series. 2D convolution is then used to extract information from the two-dimensional tensor, and then information is aggregated back to one dimension. The time series is then segmented into patches using the DWG in the Crossformer module. The obtained patch embedding serves as the input for the two-stage attention. After the two-stage attention is performed between the sequence and time, and between features, it is input into the LSTM module for prediction and correction, and the final result is output.

[0022] The TimesNet module is a two-dimensional time series tensor obtained under the condition of selected period and frequency. When the selected period or frequency is different, the obtained two-dimensional time series tensor is also different. The TimesNet module performs information mining on the decomposed time series through two-dimensional convolution;

[0023] After the Crossformer module splits the time series into patches, it divides the time series of each variable into multiple blocks according to a certain window size. Each block is mapped through a full connection, and then a two-stage attention layer is used to focus on two stages of the time series, namely the time dimension attention and the variable dimension attention, to capture and predict the intra-cycle and inter-cycle relationships in the time series;

[0024] The LSTM module corrects the prediction results and outputs the intra-cycle and inter-cycle relationships in the corrected time series.

[0025] Furthermore, in step S5, an ablation experiment of activation functions is performed on the TimesNet-Crossformer-LSTM model. The activation functions include handtanh, silu, tanh, tanhshrink, gelu, and relu. The ablation experiment shows that tanhshrink has the highest accuracy in the model prediction task.

[0026] Furthermore, the model prediction effect is evaluated by four evaluation indicators in step S6, which are: mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), and determination coefficient (R) 2 ,in:

[0027]

[0028] In the above formula, where: y s is the predicted value at time s; y rs is the actual value at time s, y average is the average value of the true value, and N is the number of samples.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The TimesNet module provided by the present invention decomposes the one-dimensional time series through fast Fourier decomposition, quickly calculates the periodicity of the series, converts the input data into the frequency domain to analyze the time series, cleverly converts the one-dimensional time series into two-dimensional space for analysis, and then extracts information from the two-dimensional tensor through 2D convolution, and then aggregates the information back to one dimension. This largely explores the relationship between each time period and has better accuracy in short-term series prediction than existing technologies.

[0031] 2. The Crossformer module provided by the present invention first passes through DWG to divide the time series into patches, and the obtained patch embedding is used as the input of the subsequent steps. It then undergoes two-stage attention to perform attention on the time dimension and attention between the feature dimensions, and outputs the relationship between sequence and time, and feature and feature. After the LSTM module corrects the prediction results, it solves the problems of gradient vanishing and gradient exploding in the overall model when processing long sequences to a certain extent, making the overall model more robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without any creative work.

[0033] Figure 1 This is an overall flow chart of a short-term power load forecasting method based on TimesNet-Crossformer-LSTM of the present invention;

[0034] Figure 2 This is a heat map of feature correlation coefficients in an embodiment of the present invention;

[0035] Figure 3 The influence of various functions on the model accuracy in the embodiment of the present invention

[0036] Figure 4 The time series of the TimesNet two-dimensional structure and the two-dimensional kernel capture time change diagram in the embodiment of the present invention;

[0037] Figure 5 : is a structural diagram of the Crossformer module in an embodiment of the present invention;

[0038] Figure 6 4 is a structural diagram of the LSTM module in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to facilitate those skilled in the art to understand and implement the present invention, each step of the method proposed in the present invention is described in detail below. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the claims appended hereto.

[0040] Example

[0041] This example uses a dataset from a region in Australia and a publicly available dataset from the United States. The Australian dataset includes 30 months of power load data from January 1, 2006, to June 1, 2008, with 48 points collected daily at 30-minute intervals. The US publicly available dataset includes 60 months of power load data from June 1, 2003, to June 1, 2008, with 24 points collected daily at 1-hour intervals.

[0042] like Figure 1 As shown, the present invention provides a short-term power load forecasting method based on TimesNet-Crossformer-LSTM, comprising the following steps:

[0043] Step S1: feature screening is performed on the imported data set, and feature selection is performed using the Pearson correlation coefficient analysis method; and the selected features are normalized;

[0044] The formula of the Pearson correlation coefficient analysis method is as follows:

[0045]

[0046] In the above formula, {x i ,,i=1,2,···,n} and {y i, ,i=1,2,···,n} are two sets of sequences with length n, and is the average of two series of length n, r xy is the variable correlation coefficient.

[0047] like Figure 2 Shown is the correlation between each eigenvector and load of the Australian data set used in this embodiment;

[0048] The selected features are normalized, and the normalization formula is as follows:

[0049]

[0050] In the above formula, z is the original data in a certain dimension; z min is the minimum value of this dimension in the dataset; z max is the maximum value of this dimension in the data set; z′ is the data after z normalization.

[0051] Step S2: Divide the filtered and processed data set into a training set, a validation set, and a test set in a ratio of 8:1:1, and input the training set and the validation set into the model for training. The input of the model is represented as X = [x1···x i-1 ,x i ···x t ] T , t represents the t-th training sample, t>i; and set batchsize=516;

[0052] Step S3: Use the TimesNet-Crossformer-LSTM model for training. Through the special structure of the model, the input one-dimensional time series is converted into a two-dimensional space for information extraction, and the relationship between the cycle and the cycle in the time series is extracted;

[0053] First, the preprocessed time series is input into the TimesNet-Crossformer-LSTM model. The TimesNet module decomposes the original sequence into a one-dimensional time series through fast Fourier decomposition, and quickly calculates the periodicity of the sequence. The input data is then converted to the frequency domain to analyze the time series. 2D convolution is then used to extract information from the two-dimensional tensor, and then information is aggregated back to one dimension. The time series is then segmented into patches using the DWG in the Crossformer module. The obtained patch embedding serves as the input for the two-stage attention. After the two-stage attention is performed between the sequence and time, and between features, it is input into the LSTM module for prediction and correction, and the final result is output.

[0054] The TimesNet module is a two-dimensional time series tensor obtained under the condition of selected period and frequency. When the selected period or frequency is different, the obtained two-dimensional time series tensor is also different. The TimesNet module mines information on the decomposed time series through two-dimensional convolution, such as Figure 4 Shown are the time series of TimesNet's two-dimensional structure and the two-dimensional kernel capture time variation diagram;

[0055] like Figure 5As shown in the figure, after the Crossformer module splits the time series into patches, it divides the time series of each variable into multiple blocks according to a certain window size. Each block is mapped through a full connection, and then a two-stage attention layer is used to focus on two stages of the time series, namely the time dimension attention and the variable dimension attention, to capture and predict the intra-cycle and inter-cycle relationships in the time series.

[0056] like Figure 6 As shown in the figure, the LSTM module corrects the prediction results and outputs the intra-cycle and inter-cycle relationships in the corrected time series. The LSTM module strengthens TimesNet's ability to capture long-term dependencies through its special deconstruction and also reduces the occurrence of gradient explosion in the overall model to a certain extent.

[0057] Step S4: Use the Adam algorithm to adjust the parameters of the neural network, use back propagation to train the network, and evaluate the training effect of the model parameters by observing the loss value on the validation set;

[0058] Step S5: Perform an ablation experiment on the activation function of the TimesNet-Crossformer-LSTM model. The activation functions include handtanh, silu, tanh, tanhshrink, gelu, and relu. The ablation experiment shows that tanhshrink has the highest accuracy in the model prediction task, so tanshark is selected as the activation function of the model.

[0059] Step S6: Input the test set into the trained model to obtain the point prediction result, and evaluate the model prediction effect through four evaluation indicators; the four evaluation indicators are: mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), and determination coefficient (R) 2 ,in:

[0060]

[0061]

[0062] In the above formula, where: y s is the predicted value at time s; y rs is the actual value at time s, y average is the average value of the true value, and N is the number of samples.

[0063] The following is a comparison of the prediction results of different models under different data sets to further illustrate the scientific nature of the method of the present invention and its advancement compared to the existing technology.

[0064] Note: The bold font in the following table indicates the optimal indicator among all models.

[0065] ① The comparison of point prediction results of different models under different data sets is shown in Tables 1 and 2 below.

[0066] Table 1. Point predictions of different models on the Australian dataset

[0067]

[0068]

[0069] Table 2. Point predictions of different models on the US dataset

[0070]

[0071] ② The comparison of multi-step prediction results of different models under different data sets is shown in Tables 3 and 4 below.

[0072] Table 3 Multi-step predictions of different models on the Australian dataset

[0073]

[0074] Table 4. Multi-step predictions of different models on the US dataset

[0075]

[0076] From the comparison of the experimental results of the above models between different data sets, it can be seen that the TimesNet-Crossformer-LSTM model proposed in this invention has the best performance and the highest accuracy. Even when the data is replaced, it can still maintain strong robustness, has higher accuracy, and has greater potential.

[0077] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention shall still fall within the scope of protection of the present invention.

Claims

1. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM, characterized in that: The following steps are involved: Step S1: feature screening is performed on the imported data set, and feature selection is performed using the Pearson correlation coefficient analysis method; and the selected features are normalized; Step S2: Divide the filtered and processed data set into a training set, a validation set, and a test set in a ratio of 8:1:1, and input the training set and the validation set into the model for training; Step S3: Use the TimesNet-Crossformer-LSTM model for training. Through the special structure of the model, the input one-dimensional time series is converted into a two-dimensional space for information extraction, and the relationship between the cycle and the cycle in the time series is extracted; First, the preprocessed time series is input into the TimesNet-Crossformer-LSTM model. The TimesNet module decomposes the original sequence into a one-dimensional time series through fast Fourier decomposition, and quickly calculates the periodicity of the sequence. The input data is then converted to the frequency domain to analyze the time series. 2D convolution is then used to extract information from the two-dimensional tensor, and then information is aggregated back to one dimension. The time series is then segmented into patches using the DWG in the Crossformer module. The obtained patch embedding serves as the input for the two-stage attention. After the two-stage attention is performed between the sequence and time, and between features, it is input into the LSTM module for prediction and correction, and the final result is output. The TimesNet module is a two-dimensional time series tensor obtained under the condition of selected period and frequency. When the selected period or frequency is different, the obtained two-dimensional time series tensor is also different. The TimesNet module performs information mining on the decomposed time series through two-dimensional convolution; After the Crossformer module splits the time series into patches, it divides the time series of each variable into multiple blocks according to a certain window size. Each block is mapped through a full connection, and then a two-stage attention layer is used to focus on two stages of the time series, namely the time dimension attention and the variable dimension attention, to capture and predict the intra-cycle and inter-cycle relationships in the time series; The LSTM module corrects the prediction results and outputs the intra-cycle and inter-cycle relationships in the corrected time series; Step S4: Use the Adam algorithm to adjust the parameters of the neural network, use back propagation to train the network, and evaluate the training effect of the model parameters by observing the loss value on the validation set; Step S5: Perform an activation function ablation experiment on the TimesNet-Crossformer-LSTM model and select tanshark as the activation function of the model; Step S6: Input the test set into the trained model to obtain the point prediction result, and evaluate the model prediction effect using four evaluation indicators.

2. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM according to claim 1, characterized in that: In step S1, the feature selection is performed using the Pearson correlation coefficient analysis method, and the formula of the Pearson correlation coefficient analysis method is as follows: In the above formula, {x i ,,i=1,2,···,n} and {y i ,,i=1,2,···,n} are two sets of sequences with length n, and is the average of two series of length n, r xy is the variable correlation coefficient; The selected features are normalized, and the normalization formula is as follows: In the above formula, z is the original data in a certain dimension; z min is the minimum value of this dimension in the dataset; z max is the maximum value of this dimension in the data set; z′ is the data after z normalization.

3. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM according to claim 1, characterized in that: In step S2, the training set and the validation set are input into the model for training. The input of the model is represented as X = [x1···x i-1 ,x i ···x t ] T , t represents the t-th training sample, t>i; and set batch size = 516.

4. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM according to claim 1, characterized in that: In step S5, an activation function ablation experiment is performed on the TimesNet-Crossformer-LSTM model. The activation functions include handtanh, silu, tanh, tanhshrink, gelu, and relu. The ablation experiment shows that tanhshrink has the highest accuracy in the model prediction task.

5. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM according to claim 1, characterized in that: In step S6, the prediction effect of the model is evaluated by four evaluation indicators, namely, mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), and determination coefficient (R). 2 ,in: In the above formula, where: y s is the predicted value at time s; y rs is the actual value at time s, y average is the average value of the true value, and N is the number of samples.

Images