A method for predicting electricity load based on the periodicity of time series data

Abstract

The present invention discloses a method for predicting electricity load based on the periodicity of time series data. The model involved in the present invention includes two modules: a trend modeling module and a periodic attention module, which respectively capture the trend and periodic characteristics of the time series. Among them, the trend modeling module uses the improved TCN to be responsible for capturing the long-term and short-term dependencies of the time series and modeling the trend of the time series; the periodic attention module consists of convolutional operations and BiLSTM. This module extracts the periodic characteristics of the time series by cross-periodically paying attention to the similar positions of the prediction points in the historical periods. Using the underlying principle of the gated recurrent neural network, while enabling the time series information to be transmitted across periods, it avoids the problem of gradient disappearance in the recurrent neural network when processing long sequence inputs. Experiments prove that explicitly paying attention to the periodic characteristics of the time series not only makes the present invention have better interpretability, but also improves the accuracy in the task of predicting electricity load data.

Description

Technical Field

[0001] The present invention belongs to the field of time series prediction. Aiming at the prediction task of fixed-cycle power consumption load data, a power consumption load prediction method based on the periodicity of time series data is designed. Background Art

[0002] A time series refers to a set of statistical data arranged in chronological order observed or recorded for the same phenomenon. Time series data is ubiquitous in daily life, from power consumption load, traffic flow, temperature change to oil price, etc. In practical applications, users usually predict new development trends or potential dangerous events based on the observation and analysis of historical time series signals. For example, predicting traffic flow to plan better driving routes, or predicting residential power consumption load to formulate more appropriate power supply plans.

[0003] Traditional time series prediction methods are mainly divided into two types: statistical methods and traditional machine learning methods. Representative ones in statistical methods include autoregressive moving average (ARMA), autoregressive integrated moving average model (ARIMA), linear regression method, etc. Representative ones in traditional machine learning methods include random forest, support vector machine, and Bayesian method, etc. However, statistical methods can only capture the linear relationship of time series data, with poor accuracy, and are generally only applicable to short-term prediction and not applicable to power consumption load prediction. And traditional machine learning is prone to the problem of falling into local optimal solutions, resulting in inaccurate prediction results of the model.

[0004] In recent years, the progress of applying deep neural networks in the direction of time series prediction has received more and more attention. Deep learning models can capture the nonlinear relationship of time series. A representative model is the recurrent neural network (RNN). The recurrent neural network predicts time series by combining historical information. Later, aiming at the problems of gradient explosion and gradient disappearance during the training of recurrent neural models, a gated recurrent neural network represented by LSTM and GRU was proposed, and the gating mechanism is used to screen important historical information; although the gated recurrent network is carefully designed to remember the long-term dependence of time series, there will still be problems of gradient disappearance and gradient explosion when dealing with too long sequences.

[0005] Most existing methods are designed for short-term time series prediction. However, in real-world electricity load forecasting, it is necessary to predict the load data for a relatively long period in the future, that is, long sequence time series forecasting (LSTF). Long sequence time series forecasting requires the model to have better prediction ability and be able to effectively capture the long-range correlation of training data and prediction sequences. The Transformer model based on the self-attention mechanism focuses on time points in historical data and captures the long-term and short-term dependencies of time series simultaneously, showing more robust performance and stronger long-term memory than recurrent neural network models. However, the self-attention calculation method of the original Transformer is insensitive to local information, making the model vulnerable to outliers and bringing potential optimization problems. To address this issue, the conTrans model enhances the attention to local context information according to the characteristics of time series prediction tasks, making the prediction more accurate.

[0006] Although neural networks have achieved good results in time series prediction tasks, few existing deep learning models explicitly consider the periodicity of time series. Recurrent neural networks and convolutional neural networks do not pay attention to the periodicity of time series. The attention mechanism focuses on important time points in historical data and does not involve the research on sequence periodicity. Electricity consumption data has a fixed cycle, and the change trends in each cycle are relatively stable. Therefore, when predicting data for a period in the future, the historical data in the same period of the previous cycle can be referred to. By utilizing the periodic characteristics of electricity consumption data, the accuracy of electricity consumption prediction is effectively improved in the present invention. Summary of the Invention

[0007] The object of the present invention is to provide a method for predicting electricity load based on the periodicity of time series for the prediction task of electricity consumption data with a fixed cycle. By fully integrating the trend and periodicity of time series, the prediction accuracy of the electricity consumption prediction task is improved.

[0008] According to the characteristics of electricity load data with a fixed cycle, the present invention proposes a jump cycle attention model based on TCN and BiLSTM. The model involved in this method consists of two modules: a trend modeling module and a cycle attention module (Cycle-BiLSTM). Among them, the trend modeling module uses the improved TCN to capture the long-term and short-term dependencies of time series and model the trend of time series. The cycle attention module extracts the periodic properties of data through convolutional operations and BiLSTM. Based on the prediction results of the trend modeling module, the module adds information containing historical cycles, combines the underlying principles of recurrent neural networks, and improves the LSTM prediction method to achieve cross-cycle transmission of time series information.

[0009] The technical solution adopted by the present invention is as follows:

[0010] Step 1: Collect residents' daily electricity consumption data.

[0011] Step 2: Preprocess the collected electricity consumption data. Remove outliers, and by analyzing the original data, assign additional features to the data.

[0012] Normalize each dimensional feature of the data separately, and divide the data into a training set, a validation set, and a test set. Take one week's electricity consumption data as the training sample, and the electricity load value of the next day as the label.

[0013] Step 3: Perform a one-dimensional convolution operation on the data preprocessed in Step 2 in the time dimension to extract the local correlation features between adjacent time steps. Divide the convolution operation results by period, and each period is used as the input of a BiLSTM unit and input into the BiLSTM unit in sequence.

[0014] Step 4: Improve the residual connection of the TCN. Cut off the covariates in the TCN residual block, and only retain the historical electricity consumption data in the residual block; input the data preprocessed in Step 2 into the improved TCN model.

[0015] Step 5: Input the results in Steps 3 and 4 into a fully connected module for full fusion.

[0016] Step 6: Introduce a residual connection, and linearly add the result of Step 5 and the residual block.

[0017] Step 7: Use the data output in Step 6 as the final prediction result.

[0018] The technical solution provided by the present invention will produce the following beneficial effects:

[0019] (1) The present invention improves the residual block of the TCN, making the residual connection in the TCN have a better convergence effect. By using TCN-modified to capture the long-term and short-term patterns of the time series, the trend features of the electricity load data are extracted.

[0020] (2) The present invention aggregates the local feature trends of the time series through convolution operations, and adds the local context information of this point on the basis of a single time point, making the prediction more accurate.

[0021] (3) The present invention divides the convolution operation results by period and uses BiLSTM to process the convolution operation results divided by period, which has the following two advantages: First, it converts the multi-point single-dimensional prediction method into a single-point multi-dimensional prediction, overcoming the problem that the inference speed of LSTM drops sharply when facing long sequence predictions; Second, it utilizes the underlying principle of LSTM to achieve cross-period transmission of time series information, enhances the attention to the corresponding positions of the prediction points in the historical periods, and improves the prediction effect of electricity load data.

[0022] The method proposed by the present invention effectively improves the prediction accuracy of electricity consumption data by capturing the periodic characteristics and long-term and short-term dependencies of electricity load data, helps the power supply department to grasp the changing trend of residents' electricity consumption demand and the supply-demand balance in real time, optimize the power rationing plan, and solves the power supply-demand contradiction to a certain extent. Brief Description of the Drawings

[0023] Figure 1 is the flow chart related to the invention;

[0024] Figure 2 is the overall framework diagram of the model;

[0025] Figure 3 is the schematic diagram of the Cycle-BiLSTM jump period attention module. Detailed Embodiment

[0026] The following combines the drawings to further describe the detailed embodiments of the present invention in detail. Its specific process is described as Figure 1 shown, where:

[0027] Step 1: Collect residents' daily electricity consumption data.

[0028] The information collected includes:

[0029] (1) Electricity load value;

[0030] (2) Covariates: weather, temperature, humidity.

[0031] Step 2: Preprocess the collected electricity data. Remove outliers, analyze the original data, and assign additional features to the data. Normalize each dimensional feature of the data separately, and divide the data into a training set, a validation set, and a test set.

[0032] (1) The additional features are obtained by analyzing the date features in the collected electricity data. The additional features include: month, day of the week today, and whether it is a holiday. Add the additional features to the end of each original data.

[0033] (2) The normalization method adopted is to constrain the feature values using the maximum and minimum values of the current feature. The formula is as follows:

[0034]

[0035] (3) When dividing the data into training set, validation set and test set, shuffle the data. After shuffling, the data shows a random distribution, enabling the model to learn the commonalities of time series during learning and enhancing the generalization ability of the model.

[0036] Step 3: Perform a one-dimensional convolution operation on the data preprocessed in Step 2 in the time dimension to extract the local correlation features between adjacent time steps, and process the single time point information of the original data into local trend features that integrate multi-step timestamps centered on this time point. Divide the convolution operation results by period, and each period is used as the input of a BiLSTM unit and input into the BiLSTM unit in sequence.

[0037] This step corresponds to the period attention module of the model. Considering the deficiencies of gated recurrent neural networks in long sequence prediction tasks, this module combines the characteristics of the underlying logic formula of gated recurrent neural networks and the advantages of modeling medium and short-term time series, and designs a period attention module that focuses on the context information of the historical period position of the prediction point. This module uses convolution operations to extract the local trend features of the context of each time point; by combining the underlying principle of gated recurrent neural networks, the results of the convolution operation are input into the BiLSTM unit by period, enabling the time series information to be transmitted across periods at the time point level. This step realizes cross-period attention while avoiding the problem of gradient disappearance when the recurrent neural network processes long sequence inputs.

[0038] (1) The convolution operation extracts the local correlation features between adjacent time steps, and processes the single time point information of the original data into continuous trajectory features that integrate multi-step timestamps centered on this time point. The scanning formula of the k-th convolution kernel is:

[0039] R k = ReLU(W k *X + b k ) (1.2)

[0040] where the output R k is a vector, ReLU is the activation function f(x) = max(0, x), Wk is the k-th convolution kernel, and bk is the bias. The convolution kernel size is w*n, w is the stride in the time dimension, and n is the number of features corresponding to a single time step. The output dimension of this convolution operation is d, where d is the number of convolution kernels, and d different time features of the time series are extracted.

[0041] (2) After dividing the result of the convolution operation by period and inputting it into the BiLSTM for training, there are the following two advantages: First, it converts the multi-point single-dimensional prediction method into a single-point multi-dimensional prediction, overcoming the problem that the inference speed of the LSTM drops sharply when facing long sequence predictions; second, it utilizes the underlying principle of the LSTM to achieve cross-period transmission of time series information, enhancing the attention to the corresponding positions of the prediction points in the historical periods and improving the prediction effect of the electricity load data. The calculation formula for the forward LSTM hidden state of the BiLSTM unit corresponding to the m-th period is (the formula for the backward LSTM is the same):

[0042] F m = σ(P m W xf + H m-1 W hf + b f ) (1.3)

[0043] I m = σ(P m W xi + H m-1 W hi + bi ) (1.4)

[0044] O m = σ(P m W xo + H m-1 W hO + b O ) (1.5)

[0045]

[0046]

[0047] H m = O m ⊙ tanh(C m ) (1.8)

[0048] Among them, P m is the input period data of the t-th LSTM unit, F m , I m and O m are the forget gate, input gate and output gate respectively, is the candidate memory unit, and C m is the memory unit.

[0049] From formulas (1.3 - 1.8), it can be seen that the matrix operations in the hidden layer of LSTM all adopt the form of Hadamard product, so the structure and order of the hidden layer do not change during training. Assuming that the hidden layer is divided into 24 parts, then the output of the third part of the hidden layer of the \(t\) - th LSTM cell (denoted as \(H\) t [3]) is calculated from the third part of the hidden layer of the \((t - 1)\) - th cell (denoted as \(H\) t-1 [3]) and the third part of the current input cycle (denoted as \(P\) t [3]), that is

[0050] \(H\) t [3]=func t (\(P\) t [3], \(H\) t-1 [3]) (1.9)

[0051] Similarly, the calculation formula of \(H\) t-1 [3] can be recorded as:

[0052] \(H\) t-1 [3]=func t-1 (\(P\) t-1 [3], \(H\) t-2 [3]) (1.10)

[0053] From the above - mentioned process of reasoning, the output of the hidden layer at the \(t\) - th time point extracts the information of these time positions \(t - p\), \(t - 2p\), …, \(t - np\) (\(p\) is the cycle), that is

[0054] \(H\) t [3]=func(\(P\) i [3], \(P\) 2 [3],..., \(P\) t [3]) (1.11)

[0055] The calculation formula for the forward LSTM hidden state of the BiLSTM cell corresponding to the \(t\) - th time point is:

[0056] \(F\) t =\(\sigma\)(\(P\) t \(W\) xf +\(H\) t-p \(W\) hf +\(b\) f ) (1.12)

[0057] \(I\) t =\(\sigma\)(\(P\) t \(W\) xi +\(H\) t-p \(W\) hi +\(b\) i ) (1.13)

[0058] \(O\) t= σ(P t W xo + H t-p W ho + b o )(1.14)

[0059]

[0060]

[0061] H t = O t ⊙ tanh(C t )(1.17)

[0062] where X t is a certain time point in the input queue, and p is the period.

[0063] Step 4: Improve the residual connection of the TCN. Cut off the covariates in the TCN residual block and only retain the historical electricity consumption data in the residual block; input the data preprocessed in Step 2 into the improved TCN model.

[0064] Modification of the TCN residual block: Modify the residual connection between layers of the TCN to a linear regression of the main feature (electric load), which improves the prediction accuracy of the model in the prediction task of multi-variable samples with single-variable labels.

[0065] The TCN adopts a multi-layer residual structure and can effectively extract the long-term and short-term features of the data. In the traditional TCN model, the residual block is obtained by performing a convolution calculation with a convolution kernel size of 1*1 on the original input matrix. However, auxiliary variables such as weather and season that implicitly affect the value of the electric load implicitly represent the information of the time series. Adding this part of knowledge back to the trained data will instead affect the extraction state of the auxiliary features of the model, causing the data to diverge and making it difficult to better utilize the residual connection structure to assist the model in predicting the main feature values. Here, the residual block of the TCN is modified by cutting off the auxiliary features in the original input matrix and using a fully connected process to restore the size of the residual block matrix for the retained main features.

[0066] The comparison of the prediction effects between the improved TCN and the original TCN is shown in Table 1 as follows:

[0067] Table 1 Comparison of prediction effects

[0068]

[0069] Step 5: Input the results in Steps 3 and 4 into the fully connected module for full fusion.

[0070] Transpose the matrices output in Steps 3 and 4 and then perform matrix concatenation. The size of the concatenated matrix is [batch_size, hidden_size, 2*pre_len], where pre_len is the prediction length. Input the concatenated matrix into the fully connected module, which is divided into two layers. The first layer extracts the common features of the matrices output in Steps 3 and 4 to generate joint features of periodic attention and long-term and short-term trend dependencies; the second layer is responsible for adjusting the matrix dimensions and aggregating the hidden layer information.

[0071] Step 6: Introduce a residual connection and perform linear addition of the result of Step 5 and the input linear transformation. When the input and output feature dimensions are equal, the residual block adopts the form of an identity residual block; when the input and output feature dimensions are not equal, the residual block adopts the form of a convolutional residual block.

[0072] The residual connection forcibly breaks the symmetry of the network, adds the linear regression result of the original data on the basis of the non-linear transformation, and avoids the problem of the degradation of the deep neural network. The core idea of the residual connection is to perform linear addition of the output result of the non-linear transformation and the result of the input linear transformation. In the present invention, the input non-linear transformation is the result of the periodic attention module and the trend modeling module extracting and processing the features of the original data.

[0073] The residual connection formula for the identity residual block is as follows:

[0074] output = f(x) + x (1.18)

[0075] The residual connection formula for the convolutional residual block is as follows:

[0076] output = f(x) + conv1d(x) (1.19)

[0077] Step 7: Use the data output in Step 6 as the final prediction result.

[0078] The performance evaluation of the present invention uses the hourly load data ENTSO-E of the European interconnected power grid from January 2015 to May 2017, a total of 20,086 pieces of data. The prediction method is to use one week of data to predict one day of data.

[0079] The performance evaluation indicators adopted by the present invention are three indicators: MSE, RMSE, and MAE.

[0080] MSE (Mean Squared Error), the mean squared error. The mean squared error refers to the expected value of the square of the difference between the parameter estimate value and the parameter true value. MSE can evaluate the degree of change of the data. The smaller the MSE value, the better the accuracy of the prediction model.

[0081]

[0082] Where p is the predicted value and a is the true value.

[0083] RMSE (Root Mean Squared Error) is the square root of the mean squared error and is a commonly used formula for measuring the error rate of a regression model.

[0084]

[0085] MAE (Mean Absolute Error): The mean absolute error is the average of the absolute errors and can better reflect the actual situation of the prediction value error.

[0086]

[0087] The comparison of the prediction effects of the model of the present invention and other models on the above dataset is shown in Table 2:

[0088] Table 2 Comparison of Model Prediction Effects

[0089]

[0090] The experimental results show that by explicitly focusing on the periodic characteristics of the electricity load data, the present invention has higher accuracy in the electricity consumption prediction task compared with other advanced methods.

Claims

1. A method for predicting electricity load based on the periodicity of time-series data, characterized in that, the method comprises the following steps: Step 1: Collect daily electricity consumption data of residents, including electricity load and covariates, where the covariates include humidity, temperature, and weather; Step 2: Preprocess the collected electricity consumption data; remove outliers, and assign additional features to the data by analyzing the original data; Perform normalization processing on each dimensional feature of the data, and divide the data into a training set, a validation set, and a test set; Take one week's electricity consumption data as training samples and the electricity load value of the next day as labels; Step 3: Perform a one-dimensional convolution operation on the data preprocessed in Step 2 in the time dimension to extract local correlation features between adjacent time steps; divide the convolution operation results by period, and each period is used as the input of a BiLSTM unit and sequentially input into the BiLSTM unit; Step 4: Improve the residual connection of TCN, remove the covariates in the residual block of the TCN, and only retain the historical electricity consumption data in the residual block; input the data preprocessed in Step 2 into the improved TCN model; Step 5: Input the results in Steps 3 and 4 into a fully connected module for full fusion; Step 6: Introduce a residual connection, and linearly add the result of Step 5 and the residual block; Step 7: Use the data output in Step 6 as the final prediction result; Regarding the improvement of TCN in Step 4, the covariates in the residual block between layers of the TCN are deleted and replaced with a linear regression of the electricity load, which increases the prediction accuracy of the model in the prediction task of multi-variable samples with single-variable labels; The fully connected module in Step 5 is divided into two layers. The first layer extracts the common features of the output matrices in Steps 3 and 4 to generate joint features of periodic attention and long-term and short-term trend dependencies; the second layer is responsible for adjusting the matrix dimension and aggregating the hidden layer information.

2. A method for predicting electricity load based on the periodicity of time-series data according to claim 1, characterized in that: The additional features described in Step 2 are obtained by analyzing the date features in the collected electricity consumption data.

3. A method for predicting electricity load based on the periodicity of time-series data according to claim 1, characterized in that: When using the convolution operation in Step 3 to extract local correlation features between adjacent time steps, the scanning formula of the k-th convolution kernel is: R k = ReLU(W k * X + b k ) where the output R k is a vector, ReLU is the activation function f(x) = max(0, x), and W k is the k-th convolutional kernel, and b k is the bias; the convolutional kernel size is w*n, where w is the stride in the time dimension and n is the number of features corresponding to a single time step; the output dimension of this convolutional operation is d, and d represents the convolutional kernel, which is designed to extract d-dimensional different temporal features.

4. A method for predicting electricity load based on the periodicity of time-series data according to claim 1, characterized in that: The residual connection in Step 6 forcibly breaks the symmetry of the network, adds the linear regression result of the original data on the basis of the non-linear transformation, and avoids the problem of deep neural network degradation; The residual connection formula using an identity residual block is as follows: output = f(x) + x The residual connection formula using a convolutional residual block is as follows: output = f(x) + conv1d(x).

Images