Carbon emission prediction method based on LSTM network model

Abstract

The invention relates to a carbon emission prediction method based on an LSTM network model. The method comprises the following steps: decomposing an original carbon emission sequence into a series of VMF modal components with limited bandwidth and residual errors; white noise weighting and multi-time reconstruction are introduced to obtain IMF components; the Luong attention mechanism is added into the LSTM long and short term memory network model; and inputting the VMF modal component and the IMF component into an improved LSTM (Long Short Term Memory) network model, and carrying out superposition reconstruction on each prediction result to obtain an attention output vector. According to the method, the complexity of an original time sequence is reduced through variational mode decomposition VMD, then residual errors of the variational mode decomposition VMD are decomposed again through ensemble empirical mode decomposition EEMD, features extracted after decomposition are predicted through an LSTM model combined with a Luong attention mechanism, and finally prediction of the carbon dioxide emission of a region is completed. The method solves the problems that the complexity is high and the information amount cannot be extracted in the variational mode decomposition residual term, so as to improve the accuracy and stability of the prediction model.

Description

Technical Field

[0001] The present invention relates to the technical field of carbon emission prediction, and in particular to a carbon emission prediction method based on an LSTM network model. Background Art

[0002] Excessive greenhouse gas emissions caused by human activities are the main cause of global warming, causing rising temperatures on the earth's surface, continued deterioration of the global climate and loss of biodiversity, which have serious negative impacts on the environment, ecosystems and social economy. Energy consumption, industrial production, transportation and other fields are the main sources of carbon dioxide emissions and are also one of the important factors in climate change. Accurately predicting carbon emissions is crucial to mitigating climate change and formulating emission reduction measures.

[0003] In recent years, common time series prediction models include prediction methods based on statistical models, gray models, and machine learning. Among them, the method based on statistical models uses historical data and economic changes to predict future carbon emissions, but the nonlinearity and volatility of carbon emission data make it difficult for statistical models to reflect complex real-world situations; gray prediction is only suitable for short-term predictions and exponential growth predictions, and predictions for long-term series will result in large errors; currently, the carbon emission prediction model based on machine learning has attracted attention because of its excellent performance in processing nonlinear data. However, it is difficult to obtain accurate and reliable results by relying solely on a single machine learning model. Researchers tend to use signal decomposition methods and deep learning theory to build a combined prediction model to improve prediction accuracy. For example, wavelet transform, empirical mode decomposition and other methods have been applied. Although the decomposition method is effective, the accuracy needs to be improved, and it is only based on short-term data predictions or does not consider other countries and regions. Summary of the invention

[0004] In order to solve the problems of insufficient processing of long-term dependency of carbon emission data and inaccurate capture of nonlinear characteristics, the purpose of the present invention is to provide a carbon emission prediction method based on LSTM network model, which can perform high-precision prediction, significantly improve the prediction effect, and improve the accuracy and stability of the prediction model.

[0005] To achieve the above purpose, the present invention adopts the following technical solution: a carbon emission prediction method based on LSTM network model, comprising:

[0006] (1) The original carbon emission sequence is processed using variational mode decomposition, and the original carbon emission sequence is decomposed into a series of VMF mode components and residuals with limited bandwidth;

[0007] (2) Use ensemble empirical mode decomposition to perform secondary decomposition on the residual, introduce white noise weighting and multiple reconstructions to obtain the IMF component;

[0008] (3) Construct an LSTM long short-term memory network model, add the Luong attention mechanism to the LSTM long short-term memory network model, and obtain an improved LSTM long short-term memory network model;

[0009] (4) The VMF modal component and IMF component are input into the improved LSTM long short-term memory network model. The improved LSTM long short-term memory network model superimposes and reconstructs each prediction result to obtain the attention output vector

[0010] The step (1) specifically refers to:

[0011] Assume x(t) is the original carbon emission sequence, and the constrained variational model of variational mode decomposition is:

[0012]

[0013] In the formula, u k are the modal components, ω k is the center frequency, k = 1, 2, ... K, K is the number of modes to be decomposed, δ(t) is the unit impulse function, * is the convolution operation, and t is the time variable;

[0014] In order to solve the optimization problem, the Lagrangian multiplier λ(t) and the quadratic penalty factor α are introduced to transform the constrained variational problem into an unconstrained variational problem, and the augmented Lagrangian expression is obtained:

[0015]

[0016] The alternating direction multiplier method is used to continuously update each component and its center frequency, and finally the optimal solution of the unconstrained model is obtained. The updated modal components after alternating optimization iterations are Center frequencyω k The expression is as follows:

[0017]

[0018] In the formula, They are The corresponding Fourier transform; is the kth modal component of n iterations, λ n (t) is the Lagrangian operator after n iterations, is the center of gravity of the power spectrum of the current modal function, τ is the convex function optimization parameter, n is the number of iterations, ω is the frequency, and the Lagrange multiplier λ(ω) is updated to λ n+1 (ω).

[0019] The step (2) specifically comprises the following steps:

[0020] (2a) First, add white noise n with normal distribution to the original carbon emission sequence x(t) i (t) Row weighting, generating a new signal x i (t):

[0021] x i (t) = x(t) + n i (t)

[0022] In the formula, x i (t) is the time series after adding white noise for the i-th time;

[0023] (2b) For a signal x containing noise i (t) Perform empirical mode decomposition to obtain the form of the respective IMF sum:

[0024]

[0025] Among them, C i,j (t) is the jth IMF component obtained by decomposition after adding white noise for the i-th time, r i (t) is the residual function, representing the average trend of the signal;

[0026] (2c) Then repeat steps (2a) and (2b) N times, and convert each component C i,j (t) is averaged and the IMF components of the ensemble empirical mode decomposition are finally obtained:

[0027]

[0028] Where N is the number of repetitions, which is 200.

[0029] The step (3) specifically comprises the following steps:

[0030] (3a) First, the forget gate f t Determine the cell state C at time t-1 t-1 The information that needs to be forgotten in the forget gate reads the hidden layer state h at time t-1 t-1 and the input sequence x at time t t :

[0031] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0032] Where W f and b f They are respectively the forget gate f t The weight and bias of , σ is the sigmoid activation function;

[0033] (3b) Secondly, input gate i t Read the input sequence x at time t t , and determines the information stored in the neuron, then the tanh layer generates the temporary state of the memory unit at time t, and finally updates the cell state Thus, the new cell state C is obtained t :

[0034] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0035]

[0036] Among them, i t Input the gate state at time t, control x t Pass to C t The amount of information, W i 、b i are the weight matrix and bias of the input gate, W c 、b c are the weight matrix and bias term of the cell state respectively; tanh is the hyperbolic tangent activation function, For Hadamard;

[0037] (3c) Finally, the output gate selects important information from the current state to output, and the sigmoid layer determines which part of the neuron state needs to be output. Then the neuron state to be output passes through a fully connected layer and is multiplied by the output of the sigmoid layer to obtain h t ,h t It is also the input value of the next hidden layer, and the output gate o t The calculation formula is:

[0038] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0039]

[0040] Among them, t is the output gate state at time t, W o 、b o are the weight matrix and bias term of the output gate respectively.

[0041] The step (4) specifically refers to: the Luong attention mechanism first calculates the correlation score between the current encoding time step and the decoding time step, and its output is a scalar to measure the last hidden layer state h t and each source hidden state h s The degree of correlation between them, and then the attention weight α is obtained by performing Softmax normalization on these correlation scores t , s is the encoding time step;

[0042]

[0043] e ts =g(h t ,h s )

[0044]

[0045] In the formula, the intermediate variable c t is the weighted sum of the encoder hidden states, exp(S(h t ,h s )) is the source hidden state h s The attention value output at the decoding layer time t, exp(S(h t ,h s' )) is the source hidden state h at the next moment s' The attention value output at the decoding layer time t, e ts is the alignment score; g is the alignment function, which is used to calculate the correlation between the outputs of s and t; s' is the encoding time step at the next moment;

[0046] Then, the information of the encoded time step is weighted and summed according to the attention weight to obtain the intermediate variable c of the current time step t ; The last hidden layer state h t With the intermediate variable c t Fusion is performed to obtain the attention output vector

[0047] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention first reduces the complexity of the original time series through variational mode decomposition VMD, and then re-decomposes the residual of variational mode decomposition VMD through ensemble empirical mode decomposition EEMD, and uses the LSTM model combined with Luong's attention mechanism to predict the features extracted after decomposition, and finally completes the prediction of regional carbon dioxide emissions; the actual example shows that the selected data includes the period of the epidemic, and a relatively high-precision prediction can still be made. The model used in the present invention has the lowest evaluation indicators and the highest prediction accuracy. The comparative experiment verifies that the present invention is an accurate and effective model; second, The present invention proposes the idea of ​​decomposing first and then integrating, and applies the secondary decomposition to the field of carbon emission prediction. First, the original carbon emission sequence is decomposed, and then each subsequence is predicted separately through an improved LSTM long short-term memory network model, and finally the final prediction result is obtained by recombining. Compared with a single model, the prediction effect of the present invention is greatly improved. Third, the method of decomposing the residual of variational mode decomposition proposed in the present invention captures important features in the original data that may be omitted by variational mode decomposition, and solves the problems of high complexity and inability to extract information in the residual terms of variational mode decomposition, so as to improve the accuracy and stability of the prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the method of the present invention;

[0049] Figure 2 It is a schematic diagram of the structure of the LSTM long short-term memory network model in the present invention;

[0050] Figure 3 It is a schematic diagram of the structure of the Luong attention mechanism in the present invention;

[0051] Figure 4 It is a comparison diagram of the effects of the present invention. DETAILED DESCRIPTION

[0052] like Figure 1 As shown, a carbon emission prediction method based on the LSTM network model includes:

[0053] (1) The original carbon emission sequence is processed using variational mode decomposition, and the original carbon emission sequence is decomposed into a series of VMF mode components and residuals with limited bandwidth;

[0054] (2) Use ensemble empirical mode decomposition to perform secondary decomposition on the residual, introduce white noise weighting and multiple reconstructions to obtain the IMF component;

[0055] (3) An LSTM long short-term memory network model is constructed, and the Luong attention mechanism is added to the LSTM long short-term memory network model to obtain an improved LSTM long short-term memory network model; the LSTM long short-term memory network model is selected as the main framework of the model. By introducing a gating mechanism, minor information can be selectively forgotten, the characteristics of time series data can be fully captured, and the nonlinear relationship between load and input data can be accurately modeled;

[0056] (4) The VMF modal component and IMF component are input into the improved LSTM long short-term memory network model. The improved LSTM long short-term memory network model superimposes and reconstructs each prediction result to obtain the attention output vector

[0057] The step (1) specifically refers to:

[0058] Assume x(t) is the original carbon emission sequence, and the constrained variational model of variational mode decomposition is:

[0059]

[0060] In the formula, u k are the modal components, ω k is the center frequency, k = 1, 2, ... K, K is the number of modes to be decomposed, δ(t) is the unit impulse function, * is the convolution operation, and t is the time variable;

[0061] In order to solve the optimization problem, the Lagrangian multiplier λ(t) and the quadratic penalty factor α are introduced to transform the constrained variational problem into an unconstrained variational problem, and the augmented Lagrangian expression is obtained:

[0062]

[0063] The alternating direction multiplier method is used to continuously update each component and its center frequency, and finally the optimal solution of the unconstrained model is obtained. The updated modal components after alternating optimization iterations are Center frequencyω k The expression is as follows:

[0064]

[0065] In the formula, They are λ n (t), Fourier transform corresponding to x(t); is the kth modal component of n iterations, λ n (t) is the Lagrangian operator after n iterations, is the center of gravity of the power spectrum of the current modal function, τ is the convex function optimization parameter, n is the number of iterations, ω is the frequency, and the Lagrange multiplier λ(ω) is updated to λ n+1 (ω).

[0066] The step (2) specifically comprises the following steps:

[0067] (2a) First, add white noise n with normal distribution to the original carbon emission sequence x(t) i (t) Row weighting, generating a new signal x i (t):

[0068] x i (t) = x(t) + n i (t)

[0069] In the formula, x i (t) is the time series after adding white noise for the i-th time;

[0070] (2b) For a signal x containing noise i (t) Perform empirical mode decomposition to obtain the form of the respective IMF sum:

[0071]

[0072] Among them, C i,j (t) is the jth IMF component obtained by decomposition after adding white noise for the i-th time, r i (t) is the residual function, representing the average trend of the signal;

[0073] (2c) Then repeat steps (2a) and (2b) N times, and convert each component C i,j (t) is averaged and the IMF components of the ensemble empirical mode decomposition are finally obtained:

[0074]

[0075] Where N is the number of repetitions, which is 200.

[0076] like Figure 2 As shown, the step (3) specifically includes the following steps:

[0077] (3a) First, the forget gate f t Determine the cell state C at time t-1 t-1 The information that needs to be forgotten in the forget gate reads the hidden layer state h at time t-1 t-1 and the input sequence x at time t t :

[0078] f t =σ(W f ·[ht-1 ,x t ]+b f )

[0079] Where W f and b f They are respectively the forget gate f t The weight and bias of , σ is the sigmoid activation function;

[0080] (3b) Secondly, input gate i t Read the input sequence x at time t t , and determines the information stored in the neuron, then the tanh layer generates the temporary state of the memory unit at time t, and finally updates the cell state Thus, the new cell state C is obtained t :

[0081] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0082]

[0083] Among them, i t Input the gate state at time t, control x t Pass to C t The amount of information, W i 、b i are the weight matrix and bias of the input gate, W c 、b c are the weight matrix and bias term of the cell state respectively; tanh is the hyperbolic tangent activation function, For Hadamard;

[0084] (3c) Finally, the output gate selects important information from the current state to output, and the sigmoid layer determines which part of the neuron state needs to be output. Then the neuron state to be output passes through a fully connected layer and is multiplied by the output of the sigmoid layer to obtain h t ,h t It is also the input value of the next hidden layer, and the output gate o t The calculation formula is:

[0085] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0086]

[0087] Among them, t is the output gate state at time t, W o 、b o are the weight matrix and bias term of the output gate respectively.

[0088] like Figure 3 As shown, step (4) specifically refers to: the Luong attention mechanism first calculates the correlation score between the current encoding time step and the decoding time step, and its output is a scalar to measure the last hidden layer state h t and each source hidden state h s The degree of correlation between them, and then the attention weight α is obtained by performing Softmax normalization on these correlation scores t , s is the encoding time step;

[0089]

[0090] e ts =g(h t ,h s )

[0091]

[0092] In the formula, the intermediate variable c t is the weighted sum of the encoder hidden states, exp(S(h t ,h s )) is the source hidden state h s The attention value output at the decoding layer time t, exp(S(h t ,h s' )) is the source hidden state h at the next moment s' The attention value output at the decoding layer time t, e ts is the alignment score; g is the alignment function, which is used to calculate the correlation between the outputs of s and t; s' is the encoding time step at the next moment;

[0093] Then, the information of the encoded time step is weighted and summed according to the attention weight to obtain the intermediate variable c of the current time step t ; The last hidden layer state h t With the intermediate variable c t Fusion is performed to obtain the attention output vector

[0094] The improved LSTM long short-term memory network model of the present invention, namely the VMD-EEMD-LSTM-LAM model, and the LSTM long short-term memory network model without the Luong attention mechanism, namely the VMD-EEMD-LSTM model, are compared as shown in the figure. Figure 4 As shown, the blue line is the original data, the orange line is the VMD-EEMD-LSTM-LAM model, and the green line is the VMD-EEMD-LSTM model. After enlarging the solid line box part, it can be seen more clearly that the prediction result curve of the model with Luong attention mechanism, namely the VMD-EEMD-LSTM-LAM model, has a higher degree of fit with the actual carbon emission data and the fluctuation is more stable. It can be seen that the present invention has excellent performance.

[0095] In summary, the present invention first reduces the complexity of the original time series through variational mode decomposition (VMD), and then re-decomposes the residual of variational mode decomposition (VMD) through ensemble empirical mode decomposition (EEMD), and uses the LSTM model combined with Luong's attention mechanism to predict the features extracted after decomposition, and finally completes the prediction of regional carbon dioxide emissions; the actual example shows that the selected data includes the period of the epidemic, and high-precision predictions can still be made. The model used in the present invention has the lowest evaluation indicators and the highest prediction accuracy. The comparative experiment verifies that the present invention is an accurate and effective model; compared with a single model, the prediction effect of the present invention is greatly improved; the method for decomposing the residual of variational mode decomposition proposed in the present invention captures important features in the original data that may be omitted by variational mode decomposition, and solves the problem of high complexity and inability to extract information in the residual term of variational mode decomposition, so as to improve the accuracy and stability of the prediction model.

Claims

1. A carbon emission prediction method based on LSTM network model, Features: The method comprises the following steps in order: (1) The original carbon emission sequence is processed using variational mode decomposition, and the original carbon emission sequence is decomposed into a series of VMF mode components and residuals with limited bandwidth; (2) Use ensemble empirical mode decomposition to perform secondary decomposition on the residual, introduce white noise weighting and multiple reconstructions to obtain the IMF component; (3) Construct an LSTM long short-term memory network model, add the Luong attention mechanism to the LSTM long short-term memory network model, and obtain an improved LSTM long short-term memory network model; (4) The VMF modal component and IMF component are input into the improved LSTM long short-term memory network model. The improved LSTM long short-term memory network model superimposes and reconstructs each prediction result to obtain the attention output vector 2. According to the carbon emission prediction method based on the LSTM network model according to claim 1, Features: The step (1) specifically refers to: Assume x(t) is the original carbon emission sequence, and the constrained variational model of variational mode decomposition is: In the formula, u k are the modal components, ω k is the center frequency, k = 1, 2, ... K, K is the number of modes to be decomposed, δ(t) is the unit impulse function, * is the convolution operation, and t is the time variable; In order to solve the optimization problem, the Lagrangian multiplier λ(t) and the quadratic penalty factor α are introduced to transform the constrained variational problem into an unconstrained variational problem, and the augmented Lagrangian expression is obtained: The alternating direction multiplier method is used to continuously update each component and its center frequency, and finally the optimal solution of the unconstrained model is obtained. The updated modal components after alternating optimization iterations are Center frequencyω k The expression is as follows: In the formula, They are λ n (t), Fourier transform corresponding to x(t); is the kth modal component of n iterations, λ n (t) is the Lagrangian operator after n iterations, is the center of gravity of the power spectrum of the current modal function, τ is the convex function optimization parameter, n is the number of iterations, ω is the frequency, and the Lagrange multiplier λ(ω) is updated to λ n+1 (ω).

3. According to the carbon emission prediction method based on the LSTM network model according to claim 1, Features: The step (2) specifically comprises the following steps: (2a) First, add white noise n with normal distribution to the original carbon emission sequence x(t) i (t) Row weighting, generating a new signal x i (t): x i (t)=x(t)+n i (t) In the formula, x i (t) is the time series after adding white noise for the i-th time; (2b) For a signal x containing noise i (t) Perform empirical mode decomposition to obtain the form of the respective IMF sum: Among them, C i,j (t) is the jth IMF component obtained by decomposition after adding white noise for the i-th time, r i (t) is the residual function, representing the average trend of the signal; (2c) Then repeat steps (2a) and (2b) N times, and convert each component C i,j (t) is averaged and the IMF components of the ensemble empirical mode decomposition are finally obtained: Where N is the number of repetitions, which is 200.

4. According to the carbon emission prediction method based on the LSTM network model according to claim 1, Features: The step (3) specifically comprises the following steps: (3a) First, the forget gate f t Determine the cell state C at time t-1 t-1 The information that needs to be forgotten in the forget gate reads the hidden layer state h at time t-1 t-1 and the input sequence x at time t t : f t =σ(W f ·[h t-1 ,x t ]+b f ) Where W f and b f They are respectively the forget gate f t The weight and bias of , σ is the sigmoid activation function; (3b) Secondly, input gate i t Read the input sequence x at time t t , and determines the information stored in the neuron, then the tanh layer generates the temporary state of the memory unit at time t, and finally updates the cell state Thus, the new cell state C is obtained t : i t =σ(W i ·[h t-1 ,x t ]+b i ) Among them, i t Input the gate state at time t, control x t Pass to C t The amount of information, W i , b i are the weight matrix and bias of the input gate, W c , b c are the weight matrix and bias term of the cell state respectively; tanh is the hyperbolic tangent activation function, For Hadamard; (3c) Finally, the output gate selects important information from the current state to output, and the sigmoid layer determines which part of the neuron state needs to be output. Then the neuron state to be output passes through a fully connected layer and is multiplied by the output of the sigmoid layer to obtain h t ,h t It is also the input value of the next hidden layer, and the output gate o t The calculation formula is: o t =o(W o ·[h t-1 ,x t ]+b o ) Among them, t is the output gate state at time t, W o , b o are the weight matrix and bias term of the output gate respectively.

5. According to the carbon emission prediction method based on the LSTM network model according to claim 1, Features: The step (4) specifically refers to: the Luong attention mechanism first calculates the correlation score between the current encoding time step and the decoding time step, and its output is a scalar to measure the last hidden layer state h t and each source hidden state h s The degree of correlation between them is then calculated by performing Softmax normalization on these correlation scores to obtain the attention weight α t , s is the encoding time step; e ts =g(h t ,h s ) In the formula, the intermediate variable c t is the weighted sum of the encoder hidden states, exp(S(h t ,h s )) is the source hidden state h s The attention value output at the decoding layer time t, exp(S(h t ,h s' )) is the source hidden state h at the next moment s' The attention value output at the decoding layer time t, e ts is the alignment score; g is the alignment function, which is used to calculate the correlation between the outputs of s and t; s' is the encoding time step of the next moment; Then, the information of the encoded time step is weighted and summed according to the attention weight to obtain the intermediate variable c of the current time step t ; The last hidden layer state h t With the intermediate variable c t Fusion is performed to obtain the attention output vector

Images