A time series prediction method for daily natural gas load

Abstract

The present invention discloses a method for time series prediction of daily natural gas load, which includes the following steps: analyzing the time series dependence relationship of historical daily natural gas load data; using the sliding window principle, determining the window length according to the dependence relationship, and extracting the time series features within the time window; using heuristic search to determine the values of the hyperparameters in the model, and using the Adam optimization algorithm to obtain the values of the parameters in the model; analyzing the in-sample prediction error and conducting white noise test; finally, using the obtained model to predict the next-day gas load. In view of the influence of date on the load value, the relationship between date coding mining and natural gas load data is utilized, eliminating the trouble of introducing exogenous variables and its own accuracy. At the same time, heuristic search is used to determine the hyperparameters of the model, increasing the anti-interference performance of the model. The time window and the recurrent neural network can fully exploit the characteristics of the time series itself, ultimately improving the prediction accuracy of the model.

Description

Technical Field

[0001] The present invention belongs to the fields of time series analysis and energy, and particularly relates to a time series prediction method for daily natural gas load. Background Art

[0002] In recent years, the energy problem has become the top priority for discussion and solution in countries around the world. Countries and organizations around the world have actively explored technologies for new alternative energy sources. Among them, renewable energy has received great attention due to its renewable and pollution-free characteristics. The economic transformation of our country also requires the cooperation of the energy industry. In recent years, the natural gas industry has developed very rapidly, and at the same time, higher requirements have been put forward for the safety, scientificity, and rationality of natural gas.

[0003] Currently, the research methods for natural gas load prediction are mainly divided into TS (time series method) and AI (artificial intelligence method). Using TS for prediction is based on the observation of the same variables collected in the past. Its statistical method has a simple structure and is easy to model. However, the distribution characteristics of data often have a greater impact on the output of its model, and its generalization ability is low. AI (artificial intelligence method) includes machine learning methods and deep learning methods. For machine learning methods, the fitting effect is better, but in the case of high-dimensional big data samples, overfitting often occurs, and the expression ability of the model is poor. For deep learning methods, although the traditional ANN (artificial neural network) deep neural network has strong fitting ability, for time series data, the traditional ANN network will destroy the time series information between data. The RNN (recurrent neural network) can allow the neurons in the hidden layer to communicate with each other, store the output result of the previous time step in the hidden layer in the form of information, and at the next time step, the previous output also has an impact on it, which connects the process. The RNN model is more suitable for learning the characteristics of time series data. The long short-term memory network (LSTM) reduces the problems of gradient explosion and dissipation in the training of recurrent neural networks by introducing gated units. In power load prediction, there is also a combined model prediction method, which optimizes multiple hyperparameters existing in the model by combining optimization algorithms; or processes the original time series data by combining data preprocessing methods, such as empirical mode decomposition (EMD), etc., which is equivalent to denoising the original data to improve the prediction accuracy, but at the same time increases the complexity of model processing. For the scenario of daily natural gas load, the data noise is small and it is not applicable. Summary of the Invention

[0004] The present invention overcomes the deficiencies of the prior art, and the technical problem to be solved is: to provide a time series prediction method for daily natural gas load. This method first uses autocorrelation analysis to determine the range of the time window, then uses a bidirectional recurrent neural network to extract time series information for prediction, and at the same time uses SA to perform combined optimization on the hyperparameters of the model to achieve accurate prediction of short-term load.

[0005] The object of the present invention is achieved by the following technical solutions: A method for predicting the daily load time series of natural gas, the method comprising the following steps:

[0006] (1) Obtain historical natural gas daily load data, and analyze the time series dependence relationship of the historical natural gas daily load data;

[0007] (2) Determine the length of the sliding time window according to the dependence relationship, and extract the time series features within the time window;

[0008] (3) Construct a load prediction model, the input of the model is the extracted time series features, the output is the predicted value of the next load data, and use heuristic search combined with the Adam optimization algorithm to determine the parameters in the load prediction model;

[0009] (4) Use the load prediction model to make a prediction, compare with the true value to obtain the prediction error, analyze the in-sample prediction error, and perform a white noise test. If the error is white noise, return to the previous step to re-determine the model parameter value, otherwise go to the next step;

[0010] (5) Use the obtained load prediction model to predict the next natural gas daily load.

[0011] Further, in the step (1), the time series dependence relationship of the historical natural gas daily load data is determined by the autocorrelation coefficient (ACF). The autocorrelation coefficient is used to measure the correlation between the observed values every k time units (y t and y t-k ) in the time series, and is represented by the sample statistic ξ k of the historical natural gas daily load data. The statistical characteristics of the sample are calculated through the finite sample data:

[0012] where

[0013] where c k is the sample autocovariance of the interval k of y t , y t represents the observed value of the natural gas daily load data at the t-th moment, and c0 represents the sample variance. ξ k is the sample autocorrelation coefficient of y t with an interval of k, and n represents the number of samples, represents the sample mean.

[0014] Further, in the step (2), the size n of the time window is determined by the lag period number of the 5% significance range of the autocorrelation function. The time series features of the sliding window include: y t-n+1 ,..., y tOne-hot encoding representation of the timing values within the window, the maximum value, minimum value, peak-to-peak value, energy, average value, absolute average value, root mean square, variance, standard deviation, peak factor, skewness factor, gap factor, waveform factor, pulse factor, margin factor, and date within the window, as well as the one-hot encoding representing the four seasons.

[0015] Further, in the step (3), the heuristic search uses the simulated annealing algorithm (SA): The simulated annealing algorithm consists of two nested loops. Among them, the outer loop is controlled by temperature, and the temperature is determined by the initial temperature, termination temperature, and temperature decay law, and the temperature affects the Metropolis criterion; the inner loop is determined by the set number of times, mainly controlling the number of new solutions generated at each temperature, corresponding to the slow cooling process. The Metropolis criterion is as follows:

[0016] When f(x j ) ≤ f(x i ), x i = x j

[0017] When f(x j ) > f(x i ), with probability, accept x j ;

[0018] In the above formula, f represents the objective function, which is the energy function; x j is the solution randomly selected from the neighborhood, x i is the solution of the previous step, T i represents the current temperature, where the value x j represents the hyperparameter value of the load prediction model.

[0019] Further, in the step (3): The load prediction model uses Bi-LSTM:

[0020] i g = sigm(i t W ix + O t-1 W im + b i )

[0021] f g = sigm(i t W fx + O t-1 W fm + b f )

[0022] O g = sigm(i t W ox + O t-1W Om +b O )

[0023] u = tanh(i t W ux +O t-1 W um +b u )

[0024] x t = f g ·x t-1 +i g ·u

[0025] O t = O g ·tanh(u)

[0026]

[0027] where: i g , f g , O g respectively represent the input gate, forget gate, and output gate of the LSTM design. The activation function of the gating unit is the sigmoid function, which outputs values between 0 and 1, determining the degree of retention and forgetting of the cell state. i t represents the input of the model, Y t represents the predicted load value. u is the candidate state of the cell, and the activation function uses the hyperbolic tangent function, outputting values between -1 and +1. The cell output is a combination of the outputs of the forward and backward networks. x t represents the updated signal, O t is the output of the unit, Y t is the final output of the model after integrating the forward and backward directions. W ix , W fx , W Ox , W ux respectively represent the input weights of the input gate, forget gate, output gate, and the candidate state of the cell. W im , W fm , W om , W um respectively represent the output weights of the input gate, forget gate, output gate, and the candidate state of the cell. b i , b f , b o , b u respectively represent the bias vectors of the input gate, forget gate, output gate, and the candidate state of the cell.

[0028] Furthermore, in step (4): perform the Ljung_Box test on the in-sample error, and the constructed test statistic is:

[0029]

[0030] H0: The original data are all independent, that is, the overall correlation coefficient is 0, and some observed correlations are only caused by random sampling errors.

[0031] H1: The original data are not independent, that is, there is at least one Where k≤m.

[0032] Where T is the sample size, m is the number of lag periods or degrees of freedom, is the autocorrelation coefficient of the i-th order lag. Under the condition that the null hypothesis is true, Q(m) follows a chi-square distribution with m degrees of freedom. Given a significance level α, the rejection region is Accepting the original hypothesis means that the original sequence is a white noise sequence, otherwise it is believed that the sequence still has correlation.

[0033] Furthermore, during the training process of the LSTM prediction model, MSE is used as the loss function and the Adam optimization algorithm is used to train the LSTM model.

[0034] Furthermore, the training of the load forecasting model includes: building a forward LSTM network: taking various time series features after standardization as the input vector of the LSTM model, and using random initialization of the weight matrix and coefficients; building a fully connected layer: sending the bidirectional LSTM network into the fully connected layer, and the output signal is the predicted value at the current moment; iteratively training the classifier model: using Adam to continuously update the parameters to determine the LSTM prediction model.

[0035] Compared with the prior art, the present invention has the following beneficial effects: the present invention first analyzes the autocorrelation of natural gas daily load data, and selects the size of the time window accordingly, which has better interpretability. For the network model, a bidirectional recurrent neural network is selected, which can effectively mine the temporal relationship of the data, and the hyperparameters of the network are searched through SA, which overcomes the shortcomings of poor universality and high uncertainty of selection by manual experience. The white noise test of in-sample prediction is performed on the availability of the model to ensure that there is no available information in the residual sequence predicted by the current model. The present invention fully exploits the characteristics of the time series itself in the natural gas daily load forecast, and ultimately improves the universality and prediction accuracy of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the method for time series prediction of natural gas daily load of the present invention;

[0037] Figure 2 is the autocorrelation function graph;

[0038] Figure 3 It is a schematic diagram of the time window;

[0039] Figure 4 It is the structural diagram of LSTM;

[0040] Figure 5 It is the structural diagram of Bi-LSTM;

[0041] Figure 6 It is the result diagram of the daily natural gas load prediction;

[0042] Figure 7 It is the residual diagram of the result of the daily natural gas load prediction; Specific implementation manner

[0043] In order to more clearly illustrate the embodiments of the present invention, the specific implementation manners of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings, and other implementation manners can be obtained. The following examples are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.

[0044] A time series prediction method for the daily natural gas load provided by the present invention uses the daily natural gas load data of urban users in a certain city in Zhejiang Province from 2011 to 2019 for a total of 3,287 days. The flowchart of this control method is as Figure 1 shown, and the specific implementation of this method includes the following steps:

[0045] (1) Obtain historical daily natural gas load data and analyze the time series dependence relationship of the historical daily natural gas load data;

[0046] The time series dependence relationship of the historical daily natural gas load data is determined by the autocorrelation coefficient (ACF). The autocorrelation coefficient can be used to measure the correlation between the observed values every k time units (y t and y t-k ) in the time series:

[0047]

[0048] Among them, ρ k represents the correlation between the observed values of the time series itself every k time units (y t and y t-k ); Cov(X,Y) represents the covariance of the random variables X and Y; σ represents the standard deviation of the variable.

[0049] In the calculation formula of ρ k , the sample statistic ξ k is used to represent. ρ kIt is the overall statistical characteristic. In practice, the statistical characteristics of the sample can still only be calculated through limited sample data.

[0050] Among them Among them, c k is the sample autocovariance of the interval k of y t , and y t represents the observed value of the daily gas load data at the t-th moment, and c0 represents the sample variance. ξ k is the sample autocorrelation coefficient of y t with an interval of k, n represents the number of samples, represents the sample mean.

[0051] (2) Determine the sliding time window length according to the dependence relationship, and extract the time series features within the time window; The autocorrelation graph is as Figure 2 shown: Plot the ACF graph for the first-order differenced sequence. It can be seen that when the autocorrelation coefficient is at multiples of 7, it is significantly greater than other lag periods. Therefore, 7 is selected as the size of the time window. The construction of the data sample refers to Figure 3 shown, and a data dataset is constructed with a time window of 7.

[0052] The size n of the time window is determined by the lag period number containing the 5% significance limit of autocorrelation. The time series features of the sliding window include: y t-n+1 ,..., y t The time series values within the window and the maximum value, minimum value, peak-to-peak value, energy, average value, absolute average value, root mean square, variance, standard deviation, peak factor, skewness factor, gap factor, waveform factor, impulse factor, margin factor, and one-hot encoding representation of the date within the window, totaling 26 dimensions, as shown in Table 1.

[0053] Table 1 Time series features extracted for the waveform

[0054]

[0055]

[0056] (3) Construct a load forecasting model. The input of the model is the extracted time series features, and the output is the predicted value of the next load data. Use heuristic search combined with the Adam optimization algorithm to determine the parameters in the load forecasting model; The heuristic search uses the simulated annealing algorithm: Simulate the process of an object cooling in the physical world for optimization. The simulated annealing algorithm contains two nested loops. Among them, the outer loop is controlled by temperature, and the temperature is determined by the initial temperature, termination temperature, and temperature decay law. The temperature affects the Metropolis criterion; The inner loop is determined by the set number of times, mainly controlling the number of new solutions generated at each temperature, corresponding to the slow cooling process.

[0057] Metropolis criterion: When f(x j ) ≤ f(x i ), x i = x j ; When f(x j ) > f(x i ), with probability, accept x j .

[0058] In the above formula, f represents the objective function, which is the energy function; x j is the solution randomly selected in the neighborhood, x i is the solution of the previous step, T i represents the current temperature, where the solved value x j represents the hyperparameter value of the load forecasting model. It can be seen from the Metropolis rule that when the temperature is higher, the probability of accepting a worse solution is relatively large, and when the temperature is lower, the probability of accepting a worse solution is relatively small.

[0059] The principle of the algorithm is as follows: In the early search process, the search space is increased to avoid falling into local optimality; in the later search process, the search space is reduced to make the approximate optimal solution closer to the global optimal solution, that is, a combination of large-scale rough search and local fine search. By traversing the probability space, the solution of the optimization problem can be obtained. It can be proved that it converges to the global optimal solution in probability. Use SA to search for the training input batch, training rounds, number of hidden layers, and number of nodes of the model. The results are shown in Table 2.

[0060] In step (3) above, the model uses Bi-LSTM, and the structure is as Figure 4 and Figure 5 shown:

[0061] i g = sigm(i t W ix + O t-1 W im + b i )

[0062] f g = sigm(i t W fx + O t-1 W fm + b f )

[0063] O g = sigm(i t W ox + O t-1 W Om + b o )

[0064] u = tanh(i t W ux + O t-1 W um + b u )

[0065] x t = f g · x t-1 + i g · u

[0066] O t = O g · tanh(u)

[0067]

[0068] where: i g , f g , O g represent the input gate, forget gate, and output gate of the LSTM design respectively. The activation function of the gating unit is the sigmoid function, which outputs a value between 0 and 1, determining the degree of retention and forgetting of the cell state. i t represents the input of the model, Y t represents the predicted load value, u is the candidate state of the cell, and the activation function uses the hyperbolic tangent function, which outputs a value between -1 and +1, indicating that the cell state needs to be strengthened in some dimensions and weakened in some dimensions, used to improve the gradient disappearance and gradient explosion problems of the traditional RNN structure. The cell output is the synthesis of the outputs of the forward and backward networks. x t represents the update signal, O t is the output of the unit, Y t is the final output of the model after synthesizing the forward and backward directions. W ix , W fx , W Ox , W ux represent the input weights of the input gate, forget gate, output gate, and candidate state of the cell respectively. W im , W fm , W Om , W um represent the output weights of the input gate, forget gate, output gate, and candidate state of the cell respectively. b i , b f , b o , b u represent the bias vectors of the input gate, forget gate, output gate, and candidate state of the cell respectively.

[0069] (4) Use the load forecasting model for prediction, compare with the true value to obtain the prediction error, analyze the in-sample prediction error, and conduct a white noise test. If the error is white noise, return to the previous step to re-determine the model parameter values; otherwise, proceed to the next step. Specifically: Conduct a Ljung_Box test on the in-sample error. The constructed test statistic is:

[0070]

[0071] The null hypothesis H0: All original data are independent, that is, the population correlation coefficient is 0, and any observed correlations are only due to random sampling errors.

[0072] The alternative hypothesis H1: The original data are not independent, that is, there is at least one where k ≤ m.

[0073] where T is the sample size, m is the degree of freedom, is the autocorrelation coefficient of the i-th lag. Under the condition that the null hypothesis holds, Q(m) follows a chi-square distribution with m degrees of freedom. Given the significance level α, the rejection region is Accepting the null hypothesis means that the original sequence is a white noise sequence; otherwise, it is considered that the sequence still has correlations.

[0074] During the training process of the LSTM prediction model, use MSE as the loss function and the Adam optimization algorithm to train the LSTM model.

[0075] The training of the load forecasting model includes: constructing a forward LSTM network: using the standardized time series features of various types as the input vector of the LSTM model, and using randomly initialized weight matrices and coefficients; constructing a fully connected layer: sending the bidirectional LSTM network into the fully connected layer, and the output signal is the predicted value at the current moment; iteratively training the classifier model: using Adam to continuously update the parameters to determine the LSTM prediction model.

[0076] The hyperparameter values of the optimized actual model are shown in Table 2;

[0077] Table 2. Hyperparameter Value Table

[0078] Initial learning rate 1e-3 Training input batch size 32 Number of training epochs 140 Number of LSTM hidden layers 2 Number of nodes in LSTM hidden layer 16,32 Number of fully-connected hidden layers 3 Number of nodes in fully-connected hidden layer 32,16,1

[0079] (5) Use the obtained load forecasting model to predict the next natural gas daily load.

[0080] Use the model to fit 280 daily data. The results are as Figure 6 shown, and the prediction residuals are as Figure 7 shown. The test statistic can pass the white noise test, and this model can be used for natural gas daily load forecasting.

[0081] The present invention first analyzes the autocorrelation relationship of natural gas daily load data, selects the size of the time window based on this, with better interpretability. For the network model, a bidirectional recurrent neural network is selected, which can effectively mine the time series relationship of the data. And the hyperparameters of the network are searched by SA, overcoming the disadvantages of poor generality and high uncertainty in selection by manual experience. For the usability of the model, a white noise test of in-sample prediction is carried out to ensure that there is no available information in the residual sequence predicted by the current model. The present invention fully exploits the characteristics of the time series itself in natural gas daily load prediction, and finally improves the universality and prediction accuracy of the method.

[0082] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for predicting the daily load time series of natural gas, characterized in that, The method includes the following steps: (1) Obtain historical daily natural gas load data and analyze the time series dependence relationship of the historical daily natural gas load data; the time series dependence relationship of the historical daily natural gas load data is determined by the autocorrelation function ACF, and the ACF is used to measure the correlation between the observed values of y t and y t-k at every k time units in the time series. Use the sample statistic ξ k of the historical daily natural gas load data to represent and calculate the statistical characteristics of the sample through finite sample data: wherein where c k is the sample autocovariance of y t at an interval of k, y t represents the observed value of the daily gas load data at the t-th moment, c0 represents the sample variance; ξ k is the sample autocorrelation coefficient of y t at an interval of k, n represents the number of samples, represents the sample mean; (2) Determine the sliding time window length according to the dependence relationship and extract the time series features within the time window; the size n of the time window is determined by the lag period number within the 5% significance range of the autocorrelation function; the time series features of the sliding window include: y t-n+1 ,..., y t the time series values within the window and the one-hot encoding representations of the maximum value, minimum value, peak-to-peak value, energy, average value, absolute average value, root mean square, variance, standard deviation, peak factor, skewness factor, gap factor, waveform factor, impulse factor, margin factor in the window, and the one-hot encoding representing the date and the one-hot encoding representing the four seasons; (3) Construct a load forecasting model. The input of the model is the extracted time series features, and the output is the predicted value of the next load data. Use heuristic search combined with the Adam optimization algorithm to determine the parameters in the load forecasting model; (4) Use the load forecasting model for prediction, compare with the true value to obtain the prediction error, analyze the in-sample prediction error, and conduct a white noise test. If the error is white noise, return to the previous step to re-determine the model parameter values, otherwise proceed to the next step; (5) Use the obtained load forecasting model to predict the next natural gas daily load.

2. The method for predicting the daily load time series of natural gas according to claim 1, wherein In step (3), the heuristic search uses the simulated annealing algorithm SA: The simulated annealing algorithm contains two nested loops; among them, the outer loop is controlled by temperature, and the temperature is determined by the initial temperature, the termination temperature, and the temperature decay law. The temperature affects the Metropolis criterion; the inner loop is determined by the set number of times, mainly controlling the number of new solutions generated at each temperature, corresponding to the slow cooling process; the Metropolis criterion is as follows: When f(x j ) ≤ f(x i ) then x i = x j When f(x j ) > f(x i ), with probability , accept x j ; In the above formula, f represents the objective function, which is the energy function; x j is a solution randomly selected in the neighborhood, and x i is the solution of the previous step, and T i represents the current temperature, where the solved value x j represents the value of the hyperparameters of the load forecasting model.

3. A method for predicting the daily load time series of natural gas according to claim 1, characterized in that In step (3): The load forecasting model uses Bi-LSTM: i g = sigm(i t W ix + O t-1 W im + b i ) f g = sigm(i t W fx + O t-1 W fm + b f ) O g = sigm(i t W Ox + O t-1 W Om + b O ) u = tanh(i t W ux + O t-1 W um + b u ) x t = f g ·x t-1 + i g ·u O t = O g ·tanh(u) Among them: i g , f g , O g respectively represent the input gate, forget gate, and output gate of the LSTM design. The activation function of the gating unit is the sigmoid function, which outputs values between 0 and 1, determining the degree of retention and forgetting of the cell state. i t represents the input of the model, Y t represents the predicted load value. u is the candidate state of the cell, and the activation function uses the hyperbolic tangent function, which outputs values between -1 and +1. The cell output is the synthesis of the outputs of the forward and backward networks. x t represents the updated signal, O t is the output of the unit, Y t is the final output of the model after synthesizing the forward and backward directions. W ix , W fx , W Ox , W ux respectively represent the input weights of the input gate, forget gate, output gate, and the candidate state of the cell. W im , W fm , W Om , W um respectively represent the output weights of the input gate, forget gate, output gate, and the candidate state of the cell. b i , b f , b o , b u respectively represent the bias vectors of the input gate, forget gate, output gate, and the candidate state of the cell.

4. A method for predicting the daily load time series of natural gas according to claim 1, wherein In step (4): Conduct a Ljung_Box test on the in-sample error, and the constructed test statistic is: H0: All the original data are independent, that is, the population correlation coefficient is 0, and some observed correlations are only due to the error of random sampling; H1: The original data is not independent, that is, there is at least one where k ≤ m; where \(T\) is the sample size and \(m\) is the number of lag periods, i.e., the degrees of freedom, is the autocorrelation coefficient of the \(i\)th lag. Under the condition that the null hypothesis holds, \(Q(m)\) follows a chi-square distribution with \(m\) degrees of freedom; given the significance level \(\alpha\), the rejection region is Accepting the null hypothesis means that the original sequence is a white noise sequence; otherwise, it is considered that the sequence still has correlations.

5. A method for predicting the daily load time series of natural gas according to claim 3, characterized in that, During the training process of the LSTM prediction model, use MSE as the loss function and the Adam optimization algorithm to train the LSTM model.

6. A method for predicting the daily load time series of natural gas according to claim 3, characterized in that, The training of the load forecasting model includes: constructing a forward LSTM network: taking the standardized various time series features as the input vector of the LSTM model, and using randomly initialized weight matrices and coefficients; constructing a fully connected layer: sending the bidirectional LSTM network into the fully connected layer, and the output signal is the predicted value at the current moment; iteratively training the classifier model: continuously update the parameters using Adam to determine the LSTM prediction model.

Images