Urban natural gas consumption prediction method based on XGB-PSO-LSTM model

Abstract

The invention discloses a city natural gas consumption prediction method based on an XGB-PSO-LSTM model, belongs to the technical field of energy prediction, and aims to solve the problem that a traditional statistical method such as time sequence analysis and regression analysis establishes a model based on historical data to predict future, and the prediction efficiency is high. The deep learning method has the advantages of processing the nonlinear relation of sequence data and long-term dependence, but the parameter adjustment is complicated and the adaptability is relatively weak.

Description

Technical Field

[0001] The present invention relates to the technical field of energy prediction, and in particular to a method for predicting urban natural gas consumption based on an XGB-PSO-LSTM model. Background Art

[0002] With the acceleration of urbanization, natural gas has been widely used in cities as an important energy source. Accurately predicting urban natural gas consumption is of great significance for optimizing energy management and ensuring energy supply security. At present, the main methods for predicting urban natural gas consumption include: 1. Traditional statistical methods: This type of method mainly includes time series analysis (ARIMA, SARIMA, etc.) and regression analysis, etc. Their main idea is to analyze the statistical laws between historical data and establish mathematical models for prediction.

[0003] Time series analysis: Based on the autocorrelation and moving average characteristics of time series data, a model is established to predict future trends. For example, the ARIMA (Autoregressive Integrated Moving Average Model) model can effectively capture trends and seasonal characteristics in data through the combination of autoregression and moving average. However, this method is essentially an extrapolation based on the trend of historical data. When external factors change significantly, the prediction results will have a large deviation.

[0004] Regression analysis: Establish a linear regression model between natural gas consumption and other influencing factors (such as temperature, holidays, seasons, etc.) to predict future consumption. For example, the multivariate linear regression model analyzes the impact of multiple influencing factors on natural gas consumption to establish a more comprehensive prediction model. However, this method lacks the ability to process nonlinearities.

[0005] 2. Machine Learning Methods: This type of method mainly includes support vector machine SVM, gradient boosting decision tree algorithm XGBoost, etc. Their main idea is to learn the nonlinear relationship in the data and build a model for prediction.

[0006] SVM: By mapping data into a high-dimensional space, the optimal hyperplane is found for classification or regression. For example, the SVM model can effectively handle nonlinear regression problems and has good generalization ability. However, this method is difficult to process a large amount of sample data quickly and efficiently.

[0007] XGBoost: Build a strong learner by combining multiple weak learners. The core idea is to decompose the objective function into a series of additive loss functions and use decision trees to gradually fit the residuals of these loss functions. This method provides a measure of feature importance, which is very useful for feature selection and model understanding. Although this method can convert sequence data into feature inputs through feature engineering, its core algorithm is not optimized for the specific structure of sequence data, so its ability to handle natural gas prediction, which has strong sequence data characteristics, is relatively limited.

[0008] 3. Deep Learning Methods This type of method mainly includes artificial neural network ANN, recurrent neural network RNN, long short-term memory network LSTM, etc. Their main idea is to learn the nonlinear relationship in the data and build a model for prediction.

[0009] ANN: It has the ability of self-learning, self-adaptation and handling nonlinear problems, and can quickly and efficiently find the optimal solution in massive data. However, as the size of the data set continues to expand, this method is prone to falling into local extreme values ​​or even failing to converge, and it may be very sensitive to slight changes in input data. For example, slight changes in natural gas prices may lead to a large gap in the model's prediction results, which in turn leads to poor robustness of the model.

[0010] RNN: It is a neural network that takes sequence data as input, performs calculations in the direction of the sequence, and all recurrent units are connected in a chain. The RNN network remembers the previous information and applies it to the calculation of the current output, which makes RNN have certain advantages in learning the nonlinear characteristics of the sequence. However, this method has the disadvantages of gradient disappearance or gradient explosion, which makes the network unstable.

[0011] LSTM: is a special type of recurrent neural network that introduces memory units Memory cells and three door controllers input gate, forget gate, output gate To effectively solve the gradient vanishing or gradient exploding problems encountered by standard RNN when processing long sequence data. LSTM can capture long-term dependencies in sequence data, and has the ability to self-learn, adapt, and handle nonlinear problems. It can quickly and efficiently find the optimal solution in massive sequence data, especially in tasks such as time series prediction and natural language processing. However, the parameter adjustment of LSTM usually requires a lot of experiments and experience, including learning rate, number of layers, number of neurons, etc., and its adaptability is weak, and different parameters are usually required for different data sets. Summary of the invention

[0012] In view of the shortcomings of the prior art, the present invention proposes a method for predicting urban natural gas consumption based on the XGB-PSO-LSTM model, comprising the following steps: Step 1: Collect some original daily load data of natural gas and form a data set; Step 2: preprocessing the original daily load data in the data set; Step 3: Split the preprocessed data set into a training set and a test set, sample feature samples from the training set, and select feature combinations ; Step 4: Initialize the XGBoost model and specify the model’s hyperparameters; Step 5: In feature combination Train the XGBoost model and use the GridSearch method to optimize the hyperparameters to obtain the optimal hyperparameter configuration; Step 6: Retrain the XGBoost model using the optimal hyperparameter configuration; Step 7: Input the data to be tested into the trained XGBoost model for prediction and obtain the prediction results; Step 8: Calculation The importance score of each feature in is filtered according to the feature importance score to form a new feature combination ; Step 9: Add the prediction results to the feature set In the process, a new feature combination is formed As a training set for the LSTM model; Step 10: Use feature combinations Train the LSTM model and use the PSO algorithm to optimize the LSTM model to obtain the optimal LSTM hyperparameter combination; Step 11: Retrain the LSTM model using the optimal LSTM hyperparameter combination; Step 12: Input the test data into the retrained LSTM model to obtain the prediction results; Step 13: Model evaluation.

[0013] As a further improvement of the present invention, the preprocessing in step 2 includes: standardization, missing value filling and outlier removal.

[0014] As a further improvement of the present invention, the division of the training set and the test set in step 3 is specifically: the training set is divided according to a ratio of 94%, and the remaining 6% is used as the test set.

[0015] As a further improvement of the present invention, the optimal hyperparameter configuration obtained in step 5 is specifically: the hyperparameters of the XGBoost model are optimized by a grid search GridSearch method, and all possible hyperparameter combinations are systematically traversed. For each parameter combination, the XGBoost model is trained, and its performance is evaluated, and the performance of each parameter combination is recorded, and the parameter combination with the best performance is selected as the final configuration.

[0016] As a further improvement of the present invention, the step 10 of using the PSO algorithm to optimize the LSTM model includes the following steps: Step 10.1: Initialize LSTM hyperparameters; Step 10.2: Initialize all particles, their velocities and positions. Each particle represents a potential LSTM parameter combination. Step 10.3: In each iteration, calculate the fitness value of each particle and evaluate the quality of the particle by calculating the mean square error; Step 10.4: Update the individual optimal solution pbest ; Step 10.5: Update the global optimal solution gbest ; Step 10.6: For each particle j Update speed and position in the iteration; Step 10.7: Determine whether the maximum number of iterations has been reached.

[0017] If the stopping condition is met, the approximate optimal solution is output , output the optimal LSTM hyperparameter combination and the corresponding fitness value, otherwise go to step 10.3 and continue running until the conditions are met.

[0018] As a further improvement of the present invention, the data to be measured in step 7 and step 12 are all original daily load data of natural gas.

[0019] Beneficial effects of the present invention: 1. Establish an XGBoost model and eliminate the least important features by calculating the importance of features to avoid interference from irrelevant features.

[0020] 2. Combining the prediction advantages of the XGBoost model and LSTM, adding the XGBoost prediction results as features to the filtered features, and then using them as the training set of LSTM, can effectively reduce the error and uncertainty of a single method.

[0021] 3. Use the PSO algorithm to optimize the model hyperparameters. PSO is a group-based heuristic search algorithm that simulates the social behavior of groups such as bird flocks and fish schools to find the optimal solution to the optimization problem. The PSO algorithm has simple parameter settings and fast convergence speed. It can effectively optimize the parameters of the LSTM model and improve the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is an operation flow chart of the city natural gas consumption prediction method based on the XGB-PSO-LSTM model of the present invention; Figure 2 This is a graph showing the feature importance results of the XGB model of the present invention; Figure 3 It is a combined diagram of the XGB-LSTM model of the present invention; Figure 4 The operating flow chart of the PSO optimization LSTM model parameters of the present invention; Figure 5 This is a graph showing the prediction of natural gas consumption of PFD_FX127 in the past 15 days using different models of the present invention; Figure 6 This is a prediction curve of natural gas consumption of PFD_FO118 in the past 15 days by different models of the present invention; Figure 7 This is a prediction curve of the natural gas consumption of PFD_BP015 in the past 15 days using different models of the present invention. DETAILED DESCRIPTION

[0023] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. The embodiments described are given in the drawings. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thoroughly and comprehensively understood.

[0024] In this embodiment, data from three gas monitoring points in a city in Northeast China are taken as an example to record daily load data from January 1, 2020 to June 30, 2024.

[0025] like Figure 1 As shown, a method for predicting urban natural gas consumption based on the XGB-PSO-LSTM model includes the following steps: Step 1: Collect some original daily load data of natural gas and form a data set; Step 2: preprocessing the original daily load data in the data set; The interquartile range (IQR) method is used to detect outliers in the original daily load data, and the possible outliers are calculated by combining the daily load, temperature and weather conditions of the same period last year and the average of the month, thereby eliminating noise data.

[0026] First, the collected raw daily load data are standardized to eliminate the influence caused by the magnitude difference between the features. The standardization is performed according to the following formula (1):

[0027] In the formula, For the i The original daily load data collected is the minimum value in the data set, is the maximum value in the data set, is the standardized data.

[0028] After being processed by the above formula, all standardized data are mapped to the range of [0,1], which allows the original daily load data with different characteristics to be directly compared.

[0029] Then, fill in the missing values. Since temperature is a very important factor affecting natural gas consumption, but some of the temperature data of the original daily load data is missing, the missing temperature data needs to be filled. Considering that the temperature change is trending in most cases and does not conform to the normal distribution, linear interpolation is used for filling. Specifically: first locate the point where the temperature data is missing, then find two known temperature data points before and after it, then calculate the slope between the two known points, that is, the rate of change, and finally use the slope and known data points to estimate the temperature of the missing data points.

[0030] Next, outliers are removed. Since the original daily load data may contain outliers due to input errors or system failures, the appearance of these outliers may have a certain impact on data analysis. Therefore, an IQR-based method is used to detect outliers. The specific approach is: first find the first quartile Q1 and the third quartile Q3 of any original daily load data, and then calculate IQR = Q3 - Q1. Then determine the range of outliers: the lower limit is Q1 - 3IQR, and the upper limit is Q3 + 3IQR. Mark values ​​outside this range as possible outliers. Finally, combined with the characteristic values ​​of daily load, temperature and weather conditions in the same period last year and the mean of the month, these possible abnormal points are further analyzed and judged to determine whether they need to be removed.

[0031] Through the above method, the outliers in the original daily load data can be detected and combined with other characteristic values ​​for comprehensive analysis, so as to eliminate noise data and improve the accuracy and reliability of data analysis.

[0032] Step 3: Split the preprocessed data set into a training set and a test set, sample feature samples from the training set, and select feature combinations ; The preprocessed data set is split into a training set and a test set. The training set is divided into 94% for model training and learning; the remaining 6% is used as a test set to evaluate the performance of the model, that is, the first 52 months of the collected daily load data are used as the training set, and the last 2 months of the data are used as the test set.

[0033] In actual operation, the method of sampling feature samples from the training set is adopted, and these sampled samples are used for model training.

[0034] Using the results of the research literature and partial correlation analysis, the specific feature combination is selected There are 21 features; They are: sales volume of the previous day, sales volume of the previous two days, sales volume of the previous three days, highest temperature of the day, lowest temperature of the day, highest temperature of the previous day, lowest temperature of the previous day, highest temperature of the previous two days, lowest temperature of the previous two days, year, weather conditions, holiday type of the day before yesterday, holiday type of tomorrow, holiday type of the day, week attribute, pm2.5 index, daily relative humidity, season, air pressure, relative wind speed, wind speed.

[0035] Step 4: Initialize the XGBoost model and specify the model’s hyperparameters: As learning rate learning_rate =[0.01, 0.03, 0.05], the maximum depth of the tree max_depth =[4,5,6,7], the number of weak learners n_estimators =[60,70,80,90], the sample ratio used in each decision tree training subsample= [0.7, 0.8, 0.9], the ratio of features used in each decision tree construction colsample_bytree =[0.7, 0.8, 0.9].

[0036] Step 5: In feature combination Train the XGBoost model and use the GridSearch method to optimize the hyperparameters to obtain the optimal hyperparameter configuration; The hyperparameters of the XGBoost model are optimized by the GridSearch method, systematically traversing all possible hyperparameter combinations. For each parameter combination, the XGBoost model is trained and its performance is evaluated, and the performance of each parameter combination is recorded, and the parameter combination with the best performance is selected as the final configuration.

[0037] In the process of optimizing XGBoost models, hyperparameter optimization is crucial. These hyperparameters can significantly affect the model's training speed, generalization ability, and prediction accuracy.

[0038] In addition, in each training, the XGBoost model will select a set of optimal hyperparameters through the GridSearch method.

[0039] Step 6: Retrain the XGBoost model using the optimal hyperparameter configuration; Step 7: Input the data to be tested into the trained XGB model for prediction to obtain the prediction result; wherein the data to be tested refers to the daily load data of the original natural gas.

[0040] After the above steps, the XGBoost model has been used in The model is trained on , and the hyperparameters of the model are optimized through the GridSearch method, and finally the prediction results are obtained.

[0041] Step 8: Calculation The importance score of each feature in is filtered according to the feature importance score to form a new feature combination ; like Figure 2 As shown, first, the feature importance is evaluated through the trained XGBoost model to calculate The importance scores of each feature in . These scores reflect the contribution of each feature to the model prediction results. By calculation, we get m The importance scores of the features are sorted according to the scores.

[0042] In order to optimize the input of features, the XGBoost model is used to identify and eliminate features that contribute less to the prediction results. The fluctuation of natural gas consumption is affected by many factors, and the importance of each factor may be different in different data sets, and there may even be some irrelevant feature noise. These noises will have a negative effect on model prediction. Therefore, based on the feature importance score, the feature threshold with a cumulative share of 99.9% is determined. After screening, features with importance scores lower than the above threshold are excluded, and only the remaining ones are retained. n important features to form a new feature combination .

[0043] Step 9: Add the prediction results to the feature set In the process, a new feature combination is formed As a training set for the LSTM model; In this framework, the mean square error (MSE) is selected as the loss function of the model to quantify the difference between the predicted results and the actual observed values.

[0044] like Figure 3 As shown, step 10: Use feature combination Train the LSTM model and use the PSO algorithm to optimize the LSTM model to obtain the optimal LSTM hyperparameter combination; Based on the LSTM network, a city gas consumption prediction model is built. Adam is selected as the optimizer. The five hyperparameters of the LSTM model: the number of model iterations, batch size batchsize , the output neurons of LSTM, the number of hidden layer neurons, and the learning rate are used as the optimization variables of PSO particles. The PSO algorithm is used to optimize the parameters of the LSTM model to avoid overfitting and further improve the generalization ability of the model.

[0045] The particle swarm optimization algorithm requires each particle to maintain two vectors during the optimization process: the velocity vector and the position vector in j Indicates the number of the particle. The velocity of the particle determines the direction and speed of its movement, while the position reflects the position of the solution represented by the particle in the solution space, which is the basis for evaluating the quality of the solution. The algorithm also requires each particle to maintain its own historical optimal position vector pbest And the group also maintains a global optimal vector gbest .

[0046] like Figure 4 As shown in Figure 1, PSO optimization of LSTM model parameters includes the following steps: Step 10.1: Initialize LSTM hyperparameters; Step 10.2: Initialize all particles, their velocities and positions. Each particle represents a potential LSTM parameter combination. Generate 10 particles randomly, each particle represents a candidate solution in the problem space. Assign a random position and velocity to each particle. Set the individual optimal solution for each particle. pbest , which is usually the initial position of the particle at the beginning. Select the global optimal solution gbest , which can be the position of the best particle in the initial population, or a preset value.

[0047] Step 10.3: In each iteration, calculate the fitness value of each particle and evaluate the quality of the particle by calculating the mean square error; Step 10.4: Update the individual optimal solution pbest ; For each particle, its current fitness value is compared with the fitness value of its historical optimal solution. If the current fitness value is better, the individual optimal solution of the particle is updated.

[0048] Step 10.5: Update the global optimal solution gbest ; Compare the individual optimal solutions of all particles and find the global optimal solution, that is, the solution with the best fitness value among all particles.

[0049] Step 10.6: For each particle j Update speed and position in the iteration; The speed update is performed according to the following formula (2):

[0050] In the formula, It is j Particles in The speed of time, It is j Particles in time t speed. w is the inertia weight, controlling the influence of the particle's previous velocity. and is the acceleration constant, which affects the degree of acceleration of particles to the individual optimal position and the global optimal position. and is a random number in the range [0,1] to increase randomness. It is j The best position found so far for a particle. It is the global optimal position found by the entire particle swarm so far. Represents particles j exist t The position at the moment.

[0051] The location update is performed according to the following formula (3):

[0052] In the formula, It is j Particles in time t location.

[0053] Step 10.7: Determine whether the maximum number of iterations has been reached; If the stopping condition is met, the approximate optimal solution is output , output the optimal LSTM hyperparameter combination and the corresponding fitness value, otherwise go to step 10.3 and continue running until the conditions are met.

[0054] The above method updates the global optimal solution in each iteration gbest And fitness values, and record the current optimal LSTM parameter combination, that is, in each training, the LSTM model will select a set of optimal hyperparameters through the PSO algorithm.

[0055] Step 11: Retrain the LSTM model using the optimal LSTM hyperparameter combination; Step 12: Input the test data into the retrained LSTM model to obtain the prediction results; The predicted results are stored in the database for easy visualization.

[0056] Step 13: Model evaluation.

[0057] The test set is used for verification to evaluate the performance of the model, and MAPE (Mean Absolute Percentage Error) is selected as the evaluation indicator to comprehensively and objectively evaluate the predictive performance of the model.

[0058] The calculation of MAPE is based on the following formula (4):

[0059] In the formula, n is the number of days to be predicted, is the true value, For the predicted value If the error meets the requirements, that is, the MAPE index is less than 6%, the model can be used for prediction.

[0060] like Figure 5-Figure 7 As shown in the figure, in this implementation, we take the data of three gas monitoring points in a city in Northeast China as an example, and record the daily load data from January 1, 2020 to June 30, 2024. In order to evaluate the prediction performance of the constructed model, MAPE is used as an evaluation index. The size of the MAPE value reflects the accuracy of the model prediction, that is, the smaller the MAPE value, the closer the model's prediction result is to the actual observation value, and the better the prediction performance.

[0061] Table 1 shows the performance of each model in the evaluation index MAPE. It can be clearly seen from Table 1 that XGBoost performs the worst on all users, which may be because XGBoost is not specially designed for time series data, so it may not be as good as LSTM in capturing the dependencies in time series data. The results of XGB-LSTM are better than LSTM, which fully proves the feature selection ability of XGBoost. Too many irrelevant features will interfere with the prediction of LSTM model. It can also be seen that the effect of LSTM-PSO is better than LSTM, because LSTM may fall into local optimality during training, and LSTM model needs to manually adjust hyperparameters through a large number of experiments. PSO, as a global optimization algorithm, can help the model jump out of the local optimal solution, and can automatically perform parameter search work to reduce manual intervention. Therefore, the LSTM model optimized by PSO has better generalization ability. XGB-LSTM-PSO performs best on three users, proving that it is effective to use XGB for secondary feature construction and fully combine PSO's parameter search ability.

[0062]

[0063] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for predicting urban natural gas consumption based on the XGB-PSO-LSTM model, characterized in that: The following steps are involved: Step 1: Collect some original daily load data of natural gas and form a data set; Step 2: preprocessing the original daily load data in the data set; Step 3: Split the preprocessed data set into a training set and a test set, and sample feature combinations from the training set ; Step 4: Initialize the XGBoost model and specify the model’s hyperparameters; Step 5: In feature combination Train the XGBoost model and use the GridSearch method to optimize the hyperparameters to obtain the optimal hyperparameter configuration; Step 6: Retrain the XGBoost model using the optimal hyperparameter configuration; Step 7: Input the data to be tested into the trained XGBoost model for prediction and obtain the prediction results; Step 8: Calculation The importance score of each feature in is filtered according to the feature importance score to form a new feature combination ; Step 9: Add the prediction results to the feature set In the process, a new feature combination is formed As a training set for the LSTM model; Step 10: Use feature combinations Train the LSTM model and use the PSO algorithm to optimize the LSTM model to obtain the optimal LSTM hyperparameter combination; Step 11: Retrain the LSTM model using the optimal LSTM hyperparameter combination; Step 12: Input the test data into the retrained LSTM model to obtain the prediction results; Step 13: Model evaluation.

2. The method for predicting urban natural gas consumption based on the XGB-PSO-LSTM model according to claim 1 is characterized in that: The preprocessing in step 2 includes: standardization, missing value filling and outlier removal.

3. The method for predicting urban natural gas consumption based on the XGB-PSO-LSTM model according to claim 2 is characterized in that: The division of the training set and the test set in step 3 is specifically as follows: the training set is divided according to a ratio of 94%, and the remaining 6% is used as the test set.

4. The method for predicting urban natural gas consumption based on the XGB-PSO-LSTM model according to claim 2 is characterized in that: The optimal hyperparameter configuration is obtained in step 5 as follows: the hyperparameters of the XGBoost model are optimized by the grid search GridSearch method, all possible hyperparameter combinations are systematically traversed, and for each parameter combination, the XGBoost model is trained and its performance is evaluated, and the performance of each parameter combination is recorded, and the parameter combination with the best performance is selected as the final configuration.

5. The method for predicting urban natural gas consumption based on the XGB-PSO-LSTM model according to claim 2 is characterized in that: The step 10 of using the PSO algorithm to optimize the LSTM model includes the following steps: Step 10.1: LSTM hyperparameter initialization; Step 10.2: Initialize all particles, their velocities and positions. Each particle represents a potential LSTM parameter combination. Step 10.3: In each iteration, calculate the fitness value of each particle and evaluate the quality of the particle by calculating the mean square error; Step 10.4: Update the individual optimal solution pbest ; Step 10.5: Update the global optimal solution gbest ; Step 10.6: For each particle j Update speed and position in the iteration; Step 10.7: Determine whether the maximum number of iterations has been reached; If the stopping condition is met, the approximate optimal solution is output , output the optimal LSTM hyperparameter combination and the corresponding fitness value, otherwise go to step 10.3 and continue running until the conditions are met.

6. The method for predicting urban natural gas consumption based on the XGB-PSO-LSTM model according to claim 2 is characterized in that: The data to be measured in step 7 and step 12 are all original daily load data of natural gas.

Images