Short-term power load prediction method based on Times Net-Crossform-LSTM

Abstract

The invention discloses a TimeNet-Crossform-LSTM-based short-term power load prediction method, and the method comprises the steps: 1) carrying out the feature screening of an imported data set, and carrying out the normalization processing of the selected features; 2) dividing a training set, a verification set and a test set, and inputting the training set and the verification set into the model for training; (3) the model carries out information extraction on the input time sequence, and the relation between the period and the period in the time sequence is extracted; 4) adjusting parameters of the neural network by adopting an Adam algorithm, training the network by utilizing back propagation, and evaluating a training effect of model parameters by observing a loss value on a verification set; 5) according to the ablation experiment result, selecting tanshak as the activation function of the model, and 6) inputting the test set into the trained model to obtain the point prediction result, and evaluating the model prediction effect through four evaluation indexes. Compared with the existing method, the short-term power load prediction method has higher precision in the short-term power load prediction task.

Description

Technical Field

[0001] The present invention belongs to the technical field of short-term power load probability prediction, and specifically relates to a short-term power load prediction method based on TimesNet-Crossformer-LSTM. Background Art

[0002] Due to the limitations of existing technologies, electric energy cannot be stored in large quantities, and the generation, transmission, distribution and consumption of electric energy are almost carried out simultaneously. Therefore, the power supply and demand sides of the power system should always be balanced. Therefore, efficient and accurate load forecasting can not only facilitate the smooth dispatch of the power system, but also improve economic and social benefits, and provide better power guarantee for people's normal life and production. Usually, power load forecasting can be divided into short-term, medium-term and long-term forecasts according to time span and operation decisions, and their corresponding responsibilities are also different. Long-term forecasting is for the planning of the power system, medium-term forecasting is for better maintenance of the power system, and the responsibility of short-term forecasting is to provide more accurate load information for the daily operation of the power system.

[0003] In the past, many single models have been applied to short-term power load forecasting. Among them, multivariate linear regression completes the forecasting task by fitting the load curve. By continuously fitting the input variables and output variables, future variables are obtained to achieve short-term load forecasting. The time series analysis method is a classic method for processing load forecasting. The random process of the historical load series is analyzed, the corresponding parameters are set, and the model is established, and then the size of the parameters in the model is estimated. With the development of machine learning technology, it has been widely used in load forecasting, including support vector machines (SVMs) and random forests. The emergence of recursive neural networks (RNNs) has derived a series of models that can be used for time series forecasting. RNNs can remember short sequence changes very well, but when processing long sequences, problems such as gradient disappearance and gradient explosion are prone to occur; unlike RNNs that repeatedly superimpose all input information, long short-term memory networks (LSTMs) can forget unimportant time series information. While solving the RNN gradient explosion, it also has advantages in processing long sequence memory tasks. Therefore, it has been widely used in time series power load forecasting. The gated recurrent neural network (GRU) merges the forget gate and input gate in the LSTM model, and merges the unit state and hidden state, which makes its structure simpler than LSTM. The bidirectional long short-term memory neural network (BiLSTM) implements two LSTM trainings on the time series, forward and backward, to further improve the globality and integrity of feature extraction. The convolutional neural network (CNN) can share convolution kernels and has certain advantages in processing multi-dimensional feature vectors. Through convolution and pooling, it can well mine and extract hidden information in historical data. With the emergence of Transformer, the self-attention mechanism is used to weight different positions in the sequence, which makes it have obvious advantages in processing time series prediction and has been widely used. Crossformer extracts the relationship between features and time, and between features through two-stage attention, and has achieved good results in processing high-dimensional time series prediction. Autoformer uses the season-trend method to decompose the time series into seasonal components and trend components, and then uses the autocorrelation coefficient of the time series to find the most relevant fragments of the time series. Informer and Pyraformer reduce complexity through sparse attention, but do not take into account the multi-component nature of the time series. The time two-dimensional variational model (TimesNet) is not limited to the one-dimensionality of time series. It expands the one-dimensional time series data to two-dimensional space for analysis and extracts features within and between cycles, making it effective in long- and short-term predictions. However, it is easy to ignore the relationship between features and time, and between features. Summary of the invention

[0004] The purpose of the present invention is to provide a short-term power load forecasting method based on TimesNet-Crossformer-LSTM in view of the deficiencies in the prior art. The TimesNet module converts a one-dimensional time series into a two-dimensional space for analysis, extracts information from the two-dimensional tensor through 2D convolution, and then performs two-stage attention on the sequence after aggregating it into a one-dimensional tensor, so as to maximize the relationship between the sequence and time, and between features. Finally, the prediction result is corrected through the LSTM module, and to a certain extent, it helps to solve the problems of gradient vanishing and gradient explosion in the overall model in the process of processing long sequences, so as to have better effect and accuracy in the task of short-term power load forecasting.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions.

[0006] A short-term power load forecasting method based on TimesNet-Crossformer-LSTM includes the following steps:

[0007] Step S1, feature screening is performed on the imported data set, and feature selection is performed using the Pearson correlation coefficient analysis method; and the selected features are normalized;

[0008] Step S2, dividing the screened and processed data set into a training set, a validation set, and a test set in a ratio of 8:1:1, and inputting the training set and the validation set into the model for training;

[0009] Step S3, using the TimesNet-Crossformer-LSTM model for training, transforming the input one-dimensional time series into a two-dimensional space through the special structure of the model for information extraction, and extracting the relationship between cycles and periods in the time series;

[0010] Step S4: Use the Adam algorithm to adjust the parameters of the neural network, use back propagation to train the network, and evaluate the training effect of the model parameters by observing the loss value on the validation set;

[0011] Step S5, performing an activation function ablation experiment on the TimesNet-Crossformer-LSTM model, and selecting tanshark as the activation function of the model;

[0012] Step S6: Input the test set into the trained model to obtain the point prediction result, and evaluate the model prediction effect through four evaluation indicators.

[0013] Furthermore, in step S1, the feature selection is performed using the Pearson correlation coefficient analysis method, and the formula of the Pearson correlation coefficient analysis method is as follows:

[0014]

[0015] In the above formula, {x i ,,i=1,2,···,n} and {y i ,,i=1,2,···,n} are two sets of sequences with length n, and is the average of two series of length n, r xy is the variable correlation coefficient.

[0016] The selected features are normalized, and the formula for normalization is as follows:

[0017]

[0018] In the above formula, z is the original data in a certain dimension; min is the minimum value of this dimension in the data set; z max is the maximum value of this dimension in the data set; z′ is the data after z normalization.

[0019] Furthermore, in step S2, the training set and the validation set are input into the model for training, and the input of the model is represented as X = [x1···x i-1 ,x i ···x z ] T ; and set batch size = 516.

[0020] Furthermore, the process of extracting the intra-cycle and inter-cycle relationships in the time series in step S3 is as follows:

[0021] First, the preprocessed time series is input into the TimesNet-Crossformer-LSTM model. The TimesNet module decomposes the original sequence into a one-dimensional time series through fast Fourier decomposition, and quickly calculates the periodicity of the sequence. Then, the input data is converted to the frequency domain to analyze the time series. Then, the two-dimensional tensor is extracted for information through 2D convolution, and then the information is aggregated back to one dimension. After that, the time series is divided into patches through DWG in the Crossformer module. The obtained patch embedding is used as the input of the two-stage attention. After the two-stage attention is performed between the sequence and time, and between features, it is input into the LSTM module for prediction and correction, and the final result is output.

[0022] The TimesNet module is a two-dimensional time series tensor obtained under the condition of selected period and frequency. When the selected period or frequency is different, the obtained two-dimensional time series tensor is also different. The TimesNet module mines information on the decomposed time series through two-dimensional convolution;

[0023] After the Crossformer module divides the time series into patches, it divides the time series of each variable into multiple blocks according to a certain window size. Each block is mapped through full connection, and then through a two-stage attention layer, it focuses on the two stages of the time series, namely, time dimension attention and variable dimension attention, to capture and predict the intra-cycle and inter-cycle relationships in the time series;

[0024] The LSTM module corrects the prediction results and outputs the intra-cycle and inter-cycle relationships in the corrected time series.

[0025] Furthermore, in step S5, an ablation experiment of activation functions is performed on the TimesNet-Crossformer-LSTM model, and the activation functions include handtanh, silu, tanh, tanhshrink, gelu, and relu. The ablation experiment shows that tanhshrink has the highest accuracy in the model prediction task.

[0026] Furthermore, in step S6, the prediction effect of the model is evaluated by four evaluation indicators, which are: mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), and determination coefficient (R). 2 ,in:

[0027]

[0028]

[0029] In the above formula, where: y s is the predicted value at time s; y rs is the actual value at time s, y average is the average value of the true value, and N is the number of samples.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. The TimesNet module provided by the present invention decomposes the one-dimensional time series through fast Fourier decomposition, quickly calculates the periodicity of the series, converts the input data to the frequency domain to analyze the time series, cleverly converts the one-dimensional time series to two-dimensional space for analysis, and then extracts information from the two-dimensional tensor through 2D convolution, and then aggregates the information back to one dimension, which largely explores the relationship between each time period and has better accuracy in short-term series prediction than the existing technology.

[0032] 2. The Crossformer module provided by the present invention first passes through DWG to divide the time series into patch forms, and the obtained patch embedding is used as the input of the subsequent steps. Then, it passes through the two-stage attention to perform attention on the time dimension and attention between the feature dimensions, and outputs the relationship between the sequence and time, and between features. After the LSTM module corrects the prediction results, it solves the problems of gradient vanishing and gradient explosion in the overall model in the process of processing long sequences to a certain extent, making the overall model more robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present disclosure and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 The overall flow chart of a short-term power load forecasting method based on TimesNet-Crossformer-LSTM of the present invention;

[0035] Figure 2 It is a heat map of characteristic correlation coefficients in an embodiment of the present invention;

[0036] Figure 3 The influence of various functions on the model accuracy in the embodiment of the present invention

[0037] Figure 4 It is a time series of the TimesNet two-dimensional structure and a two-dimensional kernel capture time variation diagram in an embodiment of the present invention;

[0038] Figure 5 is a structural diagram of a Crossformer module in an embodiment of the present invention;

[0039] Figure 6 Schematic diagram of the LSTM module in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to facilitate those of ordinary skill in the art to understand and implement the present invention, the following detailed description is given to each step of the method proposed by the present invention, and it should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of the present application.

[0041] Example

[0042] This embodiment uses a dataset from a certain region in Australia and a public dataset from the United States. The Australian dataset selected 30 months of power load data from January 1, 2006 to June 1, 2008, with 48 points collected per day and a time interval of 30 minutes. The public dataset from the United States selected 60 months of power load data from June 1, 2003 to June 1, 2008, with 24 points collected per day and a time interval of 1 hour.

[0043] like Figure 1 As shown, the present invention provides a short-term power load forecasting method based on TimesNet-Crossformer-LSTM, comprising the following steps:

[0044] Step S1, feature screening is performed on the imported data set, and feature selection is performed using the Pearson correlation coefficient analysis method; and the selected features are normalized;

[0045] The formula of the Pearson correlation coefficient analysis method is as follows:

[0046]

[0047] In the above formula, {x i ,,i=1,2,···,n} and {y i ,,i=1,2,···,n} are two sets of sequences with length n, and is the average of two series of length n, r xy is the variable correlation coefficient.

[0048] like Figure 2 Shown is the correlation between each eigenvector and load of the Australian data set used in this embodiment;

[0049] The selected features are normalized, and the formula for normalization is as follows:

[0050]

[0051] In the above formula, z is the original data in a certain dimension; min is the minimum value of this dimension in the data set; z max is the maximum value of this dimension in the data set; z′ is the data after z normalization.

[0052] Step S2: divide the screened and processed data set into training set, validation set and test set in a ratio of 8:1:1, and input the training set and validation set into the model for training. The input of the model is represented as X = [x1···x i-1 ,x i ···x z ] T ; and set batch size = 516;

[0053] Step S3, using the TimesNet-Crossformer-LSTM model for training, transforming the input one-dimensional time series into a two-dimensional space through the special structure of the model for information extraction, and extracting the relationship between cycles and periods in the time series;

[0054] First, the preprocessed time series is input into the TimesNet-Crossformer-LSTM model. The TimesNet module decomposes the original sequence into a one-dimensional time series through fast Fourier decomposition, and quickly calculates the periodicity of the sequence. Then, the input data is converted to the frequency domain to analyze the time series. Then, the two-dimensional tensor is extracted for information through 2D convolution, and then the information is aggregated back to one dimension. After that, the time series is divided into patches through DWG in the Crossformer module. The obtained patch embedding is used as the input of the two-stage attention. After the two-stage attention is performed between the sequence and time, and between features, it is input into the LSTM module for prediction and correction, and the final result is output.

[0055] The TimesNet module is a two-dimensional time series tensor obtained under the condition of selected period and frequency. When the selected period or frequency is different, the obtained two-dimensional time series tensor is also different. The TimesNet module mines information on the decomposed time series through two-dimensional convolution, such as Figure 4 Shown is the time series of TimesNet’s two-dimensional structure and the two-dimensional kernel capturing time variation graph;

[0056] like Figure 5As shown in the figure, after the Crossformer module divides the time series into patches, it divides the time series of each variable into multiple blocks according to a certain window size. Each block is mapped through full connection, and then through the two-stage attention layer, it pays attention to the two stages of the time series, namely, the time dimension attention and the variable dimension attention, to capture and predict the intra-cycle and inter-cycle relationships in the time series;

[0057] like Figure 6 As shown in the figure, the LSTM module corrects the prediction results and outputs the relationship between the cycle and the cycle in the corrected time series. The LSTM module strengthens the ability of TimesNet to capture long-term dependencies through its special deconstruction, and also reduces the occurrence of gradient explosion of the overall model to a certain extent;

[0058] Step S4: Use the Adam algorithm to adjust the parameters of the neural network, use back propagation to train the network, and evaluate the training effect of the model parameters by observing the loss value on the validation set;

[0059] Step S5, performing an ablation experiment on the activation function of the TimesNet-Crossformer-LSTM model, wherein the activation functions include handtanh, silu, tanh, tanhshrink, gelu, and relu. The ablation experiment shows that tanhshrink has the highest accuracy in the model prediction task, so tanshark is selected as the activation function of the model;

[0060] Step S6: Input the test set into the trained model to obtain the point prediction result, and evaluate the model prediction effect through four evaluation indicators; the four evaluation indicators are: mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), and determination coefficient (R) 2 ,in:

[0061]

[0062]

[0063] In the above formula, where: y s is the predicted value at time s; y rs is the actual value at time s, y average is the average value of the true value, and N is the number of samples.

[0064] The following is a comparison of the prediction results of different models under different data sets to further illustrate the scientific nature of the method of the present invention and its advancement compared to the prior art.

[0065] Note: The bold fonts in the following table represent the best indicators among all models.

[0066] ① The comparison of point prediction results of different models under different data sets is shown in Tables 1 and 2.

[0067] Table 1. Point predictions of different models on the Australian dataset

[0068]

[0069] Table 2. Point predictions of different models on the US dataset

[0070]

[0071]

[0072] ② The comparison of multi-step prediction results of different models under different data sets is shown in Tables 3 and 4.

[0073] Table 3 Multi-step predictions of different models on the Australian dataset

[0074]

[0075] Table 4. Multi-step predictions of different models on the US dataset

[0076]

[0077] From the comparison of the experimental results of the above models between different data sets, it can be seen that the TimesNet-Crossformer-LSTM model proposed in the present invention has the best performance and the highest accuracy. When the data is changed, it can still maintain strong robustness, has higher accuracy and greater potential.

[0078] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM, characterized in that: The following steps are involved: Step S1, feature screening is performed on the imported data set, and feature selection is performed using the Pearson correlation coefficient analysis method; and the selected features are normalized; Step S2, dividing the screened and processed data set into a training set, a validation set, and a test set in a ratio of 8:1:1, and inputting the training set and the validation set into the model for training; Step S3, using the TimesNet-Crossformer-LSTM model for training, transforming the input one-dimensional time series into a two-dimensional space through the special structure of the model for information extraction, and extracting the relationship between cycles and periods in the time series; Step S4: Use the Adam algorithm to adjust the parameters of the neural network, use back propagation to train the network, and evaluate the training effect of the model parameters by observing the loss value on the validation set; Step S5, performing an activation function ablation experiment on the TimesNet-Crossformer-LSTM model, and selecting tanshark as the activation function of the model; Step S6: Input the test set into the trained model to obtain the point prediction result, and evaluate the model prediction effect through four evaluation indicators.

2. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM according to claim 1, characterized in that: In step S1, the Pearson correlation coefficient analysis method is used for feature selection. The formula of the Pearson correlation coefficient analysis method is as follows: In the above formula, {x i, ,i=1,2,···,n} and {y i, ,i=1,2,···,n} are two sets of sequences with length n, and is the average of two series of length n, r xy is the variable correlation coefficient. The selected features are normalized, and the formula for normalization is as follows: In the above formula, z is the original data in a certain dimension; min is the minimum value of this dimension in the data set; z max is the maximum value of this dimension in the data set; z′ is the data after z normalization.

3. According to a TimesNet-Crossformer-LSTM-based short-term power load forecasting method according to claim 1, it is characterized in that: In step S2, the training set and the validation set are input into the model for training. The input of the model is represented as X = [x1···x i-1 ,x i ···x z ] T ; and set batch size = 516.

4. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM according to claim 1, characterized in that: The process of extracting the intra-cycle and inter-cycle relationships in the time series in step S3 is as follows: First, the preprocessed time series is input into the TimesNet-Crossformer-LSTM model. The TimesNet module decomposes the original sequence into a one-dimensional time series through fast Fourier decomposition, and quickly calculates the periodicity of the sequence. Then, the input data is converted to the frequency domain to analyze the time series. Then, the two-dimensional tensor is extracted for information through 2D convolution, and then the information is aggregated back to one dimension. After that, the time series is divided into patches through DWG in the Crossformer module. The obtained patch embedding is used as the input of the two-stage attention. After the two-stage attention is performed between the sequence and time, and between features, it is input into the LSTM module for prediction and correction, and the final result is output. The TimesNet module is a two-dimensional time series tensor obtained under the condition of selected period and frequency. When the selected period or frequency is different, the obtained two-dimensional time series tensor is also different. The TimesNet module mines information on the decomposed time series through two-dimensional convolution; After the Crossformer module divides the time series into patches, it divides the time series of each variable into multiple blocks according to a certain window size. Each block is mapped through full connection, and then through a two-stage attention layer, it focuses on the two stages of the time series, namely, time dimension attention and variable dimension attention, to capture and predict the intra-cycle and inter-cycle relationships in the time series; The LSTM module corrects the prediction results and outputs the intra-cycle and inter-cycle relationships in the corrected time series.

5. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM according to claim 1, characterized in that: In step S5, an activation function ablation experiment is performed on the TimesNet-Crossformer-LSTM model, and the activation functions include handtanh, silu, tanh, tanhshrink, gelu, and relu. The ablation experiment shows that tanhshrink has the highest accuracy in the model prediction task.

6. A short-term power load forecasting method based on TimesNet-Crossformer-LSTM according to claim 1, characterized in that: In step S6, the prediction effect of the model is evaluated by four evaluation indicators, which are: mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), and determination coefficient (R). 2 ,in: In the above formula, where: y s is the predicted value at time s; y rs is the actual value at time s, y average is the average value of the true value, and N is the number of samples.

Images