Marketized user short-term electricity consumption prediction method based on VAE-BiLSTM

Abstract

The invention discloses a marketization user short-term electricity consumption prediction method based on VAE-BiLSTM, belongs to the field of electric power system electricity consumption prediction, solves the technical problem of electricity consumption prediction accuracy, and is a process of processing collected marketization data to realize short-term electricity consumption prediction. A node weight and an offset parameter obtained after training prediction of the current batch are used as initial training parameters of the next batch, and regularization is carried out on a loss function of data of each batch after model training on the basis of the initial training parameters of the next batch. The beneficial effects of the invention are that the method collects the marketization power consumption data, divides the training data into several batches according to the correlation and internal behaviors, eliminates the influence of random abnormal points on the Bi-LSTM prediction accuracy, and improves the prediction precision.

Description

Technical Field

[0001] The present invention belongs to the field of power system electricity consumption prediction, and in particular relates to a method for predicting short-term electricity consumption of market-oriented users based on VAE-BiLSTM. Background Art

[0002] Electricity load forecasting for market-oriented users is a key link for power grid companies to maximize economic benefits. So far, many solutions have been adopted to forecast electricity load. However, most of these methods cannot simulate the time series state of data or eliminate data noise. Therefore, in most cases, the accuracy of load forecasting results is usually not high.

[0003] In order to accurately predict future electricity consumption and demand, some scholars have proposed prediction models, such as support vector machine (SVM), least square support vector machine (LSSVM), long short-term memory (LSTM), convolutional neural network (CNN), etc. However, few existing models propose data preprocessing methods, resulting in low accuracy of prediction models.

[0004] For example, when using global LSTM to create an estimation model for large time series data, some data information will be discarded due to the training characteristics of LSTM. The discarded information may have a greater impact on the prediction model. In this case, the prediction accuracy coefficient of LSTM will be reduced. Summary of the invention

[0005] In order to solve the technical problem of predicting the accuracy of electricity consumption, the present invention divides the long-term prediction method into multiple short-term prediction methods. As long as each short-term load prediction is accurate, the long-term prediction can be made more accurate by superposition. Based on this factor, the present invention designs a short-term electricity consumption prediction method for market-oriented users based on VAE-BiLSTM, and adopts the VAE method to preprocess and reconstruct the electricity consumption data of market-oriented users. The main data and valid data are considered, and the BiLSTM hybrid method based on batch training is adopted for training, which improves the performance and effectiveness of the model.

[0006] The technical solution of the present invention is a method for predicting short-term electricity consumption of market-oriented users based on VAE-BiLSTM, which is a process of processing collected market-oriented data to realize short-term electricity consumption prediction. The prediction method is trained in multiple batches after collecting market-oriented data, and the node weights and bias parameters obtained after the training and prediction of the current batch are used as the initial training parameters of the next batch. On this basis, the loss function of each batch of data after model training is regularized.

[0007] Before training any batch, the collected market data is first reconstructed. The specific steps are as follows:

[0008] a. For the current batch of market data, the market data is mapped to the latent space Z according to the encoder layer of the variational autoencoder VAE, generating the average error μ and variance σ 2 , and then get the error function F of VAE,

[0009]

[0010] In formula 1, θ represents the VAE model parameters, represents the lower bound of the variation, σ 2 zd Represents the market data matrix x d The variance matrix, μ zd Represents the market data matrix x d The average error matrix, μ xd represents the average error generated by the encoder, x represents the market data variable, α and β represent the weights associated with the optimization objective,

[0011] b. According to the decoder layer of the variational autoencoder VAE, the information of the data sample is decoded in the latent space Z and the sample data x is reconstructed. The calculation formula is as follows:

[0012] p θ (x) = ∫p θ (z)p θ (x|z)dz Formula 2,

[0013] Among them, θ represents the calculated model parameters, p θ (x) is the likelihood function, p θ (z) is the distribution function of the latent variable z in the latent space Z, p θ (x|z) represents the degree difference with q.

[0014] The training process for any batch is as follows:

[0015] c. Use a hybrid bidirectional long memory and short-term memory model BiLSTM to train the reconstructed data samples in batches and time periods. The hybrid bidirectional long memory and short-term memory model BiLSTM is based on the input gate i of the LSTM network model. t 、Forget gate t and output gate o t , and calculate a candidate unit at each time step Based on the increase of the current storage cell c t and the hidden state h t , and is defined as:

[0016]

[0017] h t =o t *tanh(c t ) Formula 4,

[0018] At time t, the hidden layer and output layer are calculated in two directions as follows:

[0019]

[0020]

[0021] In the hybrid bidirectional long memory and short-term memory model BiLSTM, with the help of the hidden state h t The information from the back layer flows to the front layer or the information from the front layer flows to the back layer, and the predicted value y t The calculation formula is:

[0022]

[0023] In Formula 3 to Formula 7, W i , W f , W o and W c Represents the weight matrix associated with each gate, b i 、b f 、b f 、b c Represents the deviation value associated with each gate, h t-1 and x t The hidden state at the previous time step and the input value at time t are demonstrated respectively, y t represents the predicted value of the load in period t, and σ represents the activation function.

[0024] The specific steps for regularizing the loss function of each batch of data after model training are as follows:

[0025] d. For y in step c t Regularization is performed, and the loss function f in regularization is loss The formula is as follows:

[0026]

[0027] In formula 8, t represents the time step (t <L),y i is the true value, represents the estimated output value, λ and Θ represent the penalty coefficient and all structural parameters in the model respectively.

[0028] The beneficial effect of the present invention is that the present invention collects market-based electricity consumption data and divides the training data into several batches according to correlation and intrinsic behavior to eliminate the influence of random anomalies on the prediction accuracy of Bi-LSTM, thereby improving the prediction accuracy.

[0029] Secondly, the training samples are reconstructed by VAE to complete the training steps in the best way. VAE is used to evaluate the probability distribution rules associated with each batch of training samples. Then, VAE reconstructs them according to the probability distribution of these samples. The reconstructed samples can reflect the load status, and their predicted values ​​are very close to the actual values. One of the most important advantages of VAE is to minimize noise interference and anomalies during data preprocessing and prevent the loss of key data information.

[0030] The present invention uses a BiLSTM model to train load forecasting input data, which not only learns the positive laws of the data, but also learns the negative laws of the data. Batch training technology can ensure that accurate information of the current data is retained and used during the training process. During the training process, in order to retain important state information, the structural parameters related to the local Bi-LSTM will be shared as the initial structural parameters of the next local Bi-LSTM model.

[0031] In order to avoid overfitting problems when training the VAE-BiLSTM model, adjustment parameters are proposed in the loss function. The BiLSTM method based on batch training is used to improve the performance and effectiveness of the model. DETAILED DESCRIPTION

[0032] In the specific implementation of the present invention, the first step is to collect electricity consumption data of market users and divide the training data into multiple batches, wherein the number of training samples related to each batch is calculated by the formula L=T / n, where T represents an integer of training samples and n represents the number of batches.

[0033] Then, the dataset associated with each batch is considered as M i ={(x i1 ,y i1 ),(x i2 ,y i2 ),L,(x iL ,y iL}, where x and y are input and output respectively. The VAE model is applied to the dataset to reduce the noise interference and outliers in the training data, thereby reconstructing the training samples in batches. The reconstructed samples are represented by M′ i It is used to train the BiLSTM model. The trained local model extracts data features, identifies its behavior patterns, and can describe the current situation as comprehensively as possible. At this time, by inputting the query sample n i ={(m i1 ,ni1 ),(m i2 ,n i2 ),L,(m ik ,n ik} as the input of the network, and output Compared with the global BiLSTM prediction result, the local BiLSTM prediction result at the same time node is closer to M i , with higher accuracy. The same local BiLSTM for N i After the prediction, the parameters such as node weights and biases related to the local BiLSTM structure will be shared and used as the initial network structure parameters of the next batch of local BiLSTM. This process retains important state information and is an improvement on the network structure. Then, the next batch of training data sets M j To train a local BiLSTM.

[0034] In addition, since the number of training data samples is reduced by using back propagation through time in local BiLSTM training, the model may face the problem of overfitting. In the model, this problem is solved by introducing a regularization parameter in the loss function. Finally, the loss function is expressed as follows:

[0035]

[0036] In formula 8, t represents the time step (t <L),y i is the true value, Represents the estimated output value. The second term in the equation is the regularization term, which is used to prevent overfitting. λ and Θ represent the penalty coefficient and all structural parameters in the model, respectively.

[0037] The following is the theoretical research process of the short-term electricity consumption prediction method proposed by the present invention:

[0038] First, the market-based electricity consumption data is collected, and the training data is divided into several batches according to the correlation and intrinsic behavior to eliminate the impact of random outliers on the prediction accuracy of Bi-LSTM. Secondly, the training samples are reconstructed by VAE to complete the training steps in the best way. VAE is used to evaluate the probability distribution rules associated with each batch of training samples. Then, VAE reconstructs these samples according to their probability distribution. The reconstructed samples can reflect the load state, and their predicted values ​​are also very close to the actual values. One of the most important advantages of VAE is to minimize noise interference and anomalies during data preprocessing and prevent the loss of key data information. Finally, the reconstructed samples are used for local BiLSTM training in batches and time series. Batch training technology ensures that accurate information of the current data is retained and used during the training process. During the training process, in order to retain important state information, the structural parameters related to the local Bi-LSTM will be shared as the initial structural parameters of the next local Bi-LSTM model. Finally, in order to avoid overfitting problems when training the VAEBiLSTM model, adjustment parameters are proposed in the loss function. The following steps are included:

[0039] S1: Variational Autoencoder VAE;

[0040] S2: Bidirectional long memory and short-term memory BiLSTM;

[0041] S3: Variational Autoencoder Bidirectional Long Short-Term Memory VAEBiLSTM.

[0042] Among them, S1 includes the following steps: Variational Autoencoder VAE is one of the deep learning applications that adopts unsupervised learning strategy. The network is based on a fully connected neural network with an encoder layer and a decoder layer, and its structure is similar to the multi-layer perceptron MLP neural network structure. The encoder layer structure is used to transmit and map the input data to samples in a low-dimensional latent space, learn the smooth latent state representation of the input data, and describe the observations in the latent space in a probabilistic manner, which is the outstanding advantage of this network.

[0043] According to the data distribution rules, the information of the data sample is decoded in the latent space through the decoder layer to reconstruct the sample. Therefore, the data variable x is displayed in the latent space Z, and the latent variable in Z is displayed as z. Considering that the distribution function z is p(z), the calculation formula of the data x is as follows:

[0044] p θ (x) = ∫p θ (z)p θ (x|z)dz Formula 2,

[0045] Where θ is the calculated model parameter. The likelihood function p based on the variational inference principle is θ (x) is calculated as follows:

[0046]

[0047] in, Indicates q θ (z|x) and p θ The difference between (z|x) is called KL divergence. It should be noted that Equation 9 is obtained by optimizing To achieve optimization. As shown in Formula 10, since optimization must be performed, it can be written as Formula 11. After optimizing the objective function, the initial values ​​of some distributions can be reset and the constant terms can be deleted, which can be expressed as Formula 12:

[0048]

[0049]

[0050]

[0051] KL(q θ (z|x)p θ (z|x)) represents q θ (z|x) and p θ The difference between (z|x) and is called the KL divergence. The variational lower bound is shown, and the ^ symbol represents the estimated state.

[0052] VAE has a structure based on a neural network, and the optimization of the index shown in Formula 12 is performed by a neural network with a special structure. The objective function optimized when training the VAE network is:

[0053]

[0054] Among them, α and β represent the weights associated with the optimization target and are constants between (0,1). F is the error function associated with VAE. The error function iterates between the encoder layer and the decoder layer to repeatedly adjust the weights and biases. The error function iteration process has two stages: the first stage is p θ (z|x), the second stage is q θ (z|x). The first stage is to map the original market data x to the latent space Z through the encoder, generating the mean μ and variance σ 2 ,The second stage is the processing of the latent variables z in the latent space Z, and reconstructs the market data variables x through the decoder layer, where x′ represents the inverse matrix of x.

[0055] The S2 includes the following steps: There are three gates in the LSTM structure: input gate i t、Forget gate t and output gate o t Each gate has a specific function. The input gate determines whether the memory cell can be updated, the forget gate controls whether the memory cell is kept or deleted, and the output gate controls the input of information, the retention of the cell's historical state information, and the output of information. All of these gates are also activated by a sigmoid activation function. The rewriting of the memory cell in the LSTM structure is done by calculating a candidate cell at each time step. The mathematical formula related to the LSTM structure is defined as:

[0056] i t =σ(W i [h t-1 ,x t ]+b i ) Formula 13,

[0057] f t =σ(W f [h t-1 ,x t ]+b f ) Formula 14,

[0058] o t =σ(W o [h t-1 ,x t ]+b o ) Formula 15,

[0059]

[0060]

[0061] h t =o t *tanh(c t ) Formula 4,

[0062] Where σ represents the activation function. i , W f , W o and W c Represents the weight matrix associated with each gate. i 、b f 、b f 、b c Denotes the deviation value associated with each gate. t-1 and x t The hidden state at the previous time step and the input value at time t are respectively demonstrated. Finally, c t and h t Describes the current storage cell and hidden state respectively.

[0063] Unlike traditional LSTM, in BiLSTM network, hidden states are used to flow information from the back layer to the front layer and vice versa. At time t, the hidden layer and output layer are calculated in two directions as follows:

[0064]

[0065]

[0066]

[0067] Comparing BiLSTM and LSTM models, BiLSTM has some inherent advantages in structure, such as learning forward rules for data information and backward rules for data information, which highlights the high performance of the network. Therefore, the BiLSTM network can use the features extracted in the past during the training process to improve the training process in time periods related to different time series. Load forecasting data is a time series, during which there will be some changes in load consumption patterns based on sudden changes in weather and other unforeseen problems. In this study, this significant feature of Bi-LSTM helps the network identify power consumption patterns throughout the training data during the forecasting process, thereby improving the performance and accuracy of network forecasting.

[0068] The prediction results of the VAE-BiLSTM based model are compared with other methods.

[0069] The model is better than the power consumption prediction models such as SVR and LSTM. The predicted value obtained by the model of the present invention is closer to the actual value, with a smaller error value and lower than the previous model. The technical comparison of the effects and contributions of the present invention with the recent research results is shown in Table 1.

[0070] Table 1 Comparison with similar studies in the literature

[0071]

Claims

1. A method for predicting short-term electricity consumption of market users based on VAE-BiLSTM is a process of processing collected market data to realize short-term electricity consumption prediction, characterized by: The prediction method is trained in multiple batches after collecting market data, and the node weights and bias parameters obtained after training and predicting the current batch are used as the initial training parameters of the next batch. On this basis, the loss function of each batch of data after model training is regularized.

2. According to claim 1, a method for predicting short-term electricity consumption of market users based on VAE-BiLSTM is characterized by: Before training any batch, the collected market data is first reconstructed. The specific steps are as follows: a. For the current batch of market data, the market data is mapped to the latent space Z according to the encoder layer of the variational autoencoder VAE, generating the average error μ and variance σ 2 , and then get the error function F of VAE, In formula 1, θ represents the VAE model parameters, represents the lower bound of the variation, σ 2 zd Represents the market data matrix x d The variance matrix, μ zd Represents the market data matrix x d The average error matrix, μ xd represents the average error generated by the encoder, x represents the market data variable, α and β represent the weights associated with the optimization objective, b. According to the decoder layer of the variational autoencoder VAE, the information of the data sample is decoded in the latent space Z and the sample data x is reconstructed. The calculation formula is as follows: p θ (x) = ∫p θ (z)p θ (x|z)dz Formula 2, Among them, θ represents the calculated model parameters, p θ (x) is the likelihood function, p θ (z) is the distribution function of the latent variable z in the latent space Z, p θ (x|z) represents the degree difference with q.

3. According to claim 2, a method for predicting short-term electricity consumption of market users based on VAE-BiLSTM is characterized by: The training process for any batch is as follows: c. Use a hybrid bidirectional long memory and short-term memory model BiLSTM to train the reconstructed data samples in batches and time periods. The hybrid bidirectional long memory and short-term memory model BiLSTM is based on the input gate i of the LSTM network model. t 、Forget gate t and output gate o t , and calculate a candidate unit at each time step Based on the increase of the current storage cell c t and the hidden state h t , and is defined as: h t =o t *tanh(c t ) Formula 4, At time t, the hidden layer and output layer are calculated in two directions as follows: In the hybrid bidirectional long memory and short-term memory model BiLSTM, with the help of the hidden state h t The information from the back layer flows to the front layer or the information from the front layer flows to the back layer, and the predicted value y t The calculation formula is: In Formula 3 to Formula 7, W i , W f , W o and W c Represents the weight matrix associated with each gate, b i 、b f 、b f 、b c Represents the deviation value associated with each gate, h t-1 and x t The hidden state at the previous time step and the input value at time t are demonstrated respectively, y t represents the predicted value of the load in period t, and σ represents the activation function.

4. According to claim 1, a method for predicting short-term electricity consumption of market users based on VAE-BiLSTM is characterized by: The specific steps for regularizing the loss function of each batch of data after model training are as follows: d. For y in step c t Regularization is performed, and the loss function f in regularization is loss The formula is as follows: In formula 8, t represents the time step (t <L),y i is the true value, represents the estimated output value, λ and Θ represent the penalty coefficient and all structural parameters in the model respectively.

Images