Biogas production prediction method based on lstm-xgb dynamic integration

Abstract

The application discloses a biogas production prediction method based on LSTM-XGB dynamic integration, and belongs to the technical field of biogas engineering. The method comprises the following steps: collecting multi-dimensional historical working condition data of anaerobic digestion reaction and performing normalization pretreatment; independently training a long short-term memory (LSTM) prediction base model and an extreme gradient boosting (XGB) prediction base model respectively; obtaining real-time biogas operation data, and calculating a real-time data fluctuation index through a sliding window; comparing the fluctuation index with a set threshold value, and dynamically allocating prediction weights to the LSTM and XGB prediction base models according to a comparison result; and finally inputting a current working condition into the two base models, combining the allocated weights, and outputting a final biogas production prediction value. The application introduces a data fluctuation degree as a soft switch, effectively considers the prediction of a smooth time sequence trend and the fitting of local fluctuations under extreme working conditions (such as temperature mutation), significantly reduces a mean absolute percentage error (MAPE), and provides high-precision decision support for biogas engineering feedforward regulation.

Description

Technical Field

[0001] This invention belongs to the field of biogas engineering technology and relates to a biogas production prediction method based on LSTM-XGB (LSTM-XGBoost-volatility) dynamic integration. Background Technology

[0002] As the global energy landscape shifts towards renewable energy, biomass energy is rapidly developing due to its resource recycling and low-carbon advantages. Anaerobic digestion and biogas production have become crucial for building a sustainable energy system, with over 13,000 biogas digesters already in operation worldwide. Biogas projects offer value in waste treatment, carbon reduction, and ecological protection. Conducting biogas forecasting accurately predicts biogas production, supports project operation and control, optimizes process parameters to improve the technical efficiency of medium and large-scale projects, and facilitates capacity planning and resource allocation through reliable forecasting. This ensures stable biogas energy output, aligns with policy and circular economy needs, promotes the release of industry potential, and provides a guarantee for energy transition and achieving dual-carbon goals.

[0003] Anaerobic digestion is the core of biogas production. Anaerobic microorganisms convert organic matter into biogas through a series of complex biochemical reactions, including hydrolysis, acidification, acetic acid production, and methanogenesis. The interplay of multiple biochemical reactions and microbial interactions makes anaerobic digestion highly nonlinear and dynamic, making it difficult for traditional physical and mathematical formulas to accurately describe the process and thus unreliably predict biogas yield. Machine learning, using neural network algorithms, can effectively capture the nonlinear relationships and dynamic changes in complex systems, overcoming the limitations of traditional methods and achieving accurate predictions of biogas yield. However, the high complexity of the anaerobic digestion process causes existing prediction methods, such as single LSTM models or fixed-weight ensemble models, to significantly degrade in predictive performance when dealing with fluctuations in operating conditions caused by changes in feed composition and sudden changes in ambient temperature. Specifically, LSTM models respond too smoothly to sudden fluctuations, while static ensemble models cannot adaptively adjust according to data characteristics, resulting in insufficient prediction accuracy and anti-interference capabilities for complex dynamic systems. How to maintain high accuracy in prediction models during both "stable periods" and "periods of severe fluctuation" is a pressing technical bottleneck that needs to be addressed. Summary of the Invention

[0004] To overcome the shortcomings of existing prediction models that show a significant decrease in prediction accuracy when biogas fermentation systems face extreme operating conditions, this invention aims to provide a biogas production prediction method based on LSTM-XGB dynamic ensemble. This method introduces real-time data volatility as a decision signal for model switching, enabling dynamic weight allocation among different algorithms.

[0005] The core of Long Short-Term Memory (LSTM) networks is to add "memory units" and "input gates, forget gates, and output gates" to the recurrent neural network. Through this gating structure, it autonomously filters key long-term information from time-series data and removes redundant content, completely overcoming the bottleneck of long-sequence dependencies. The eXtreme Gradient Boosting (XGB) algorithm is an optimization of gradient boosting decision trees, addressing their shortcomings in prediction accuracy and computational efficiency. It also incorporates L1 and L2 regularization and tree pruning to suppress overfitting, supports parallel computing to improve training efficiency, and automatically handles missing values ​​to reduce data preprocessing difficulty. SHAP analysis (SHapley Additive ex Planations) and feature importance are core tools for interpretability in machine learning models, effectively solving the "black box" problem of models. Revealing the correlation between features and model output from different dimensions helps optimize the model, serves as the basis for dynamically fusing two models, and makes the decision-making process more reliable. The core concept of this invention is to introduce a real-time data volatility index as a decision signal for model switching, thereby constructing a dynamic weighted ensemble model that can adapt to changes in operating conditions. This model is no longer a simple superposition of models, but an intelligent prediction system with self-judgment and adjustment capabilities. This invention divides biogas production into stable and fluctuating states. Based on these two states, the weights of the two algorithms (LSTM / XGB) differ. Through SHAP analysis, the underlying mapping is the change in the weight of the temperature parameter. The flexible application and integration of the two algorithms increases prediction accuracy.

[0006] This invention provides a biogas production prediction method based on LSTM-XGB dynamic integration, the steps of which are as follows:

[0007] (1) Collect data related to anaerobic digestion, conduct correlation analysis, and record them as characteristic parameters (including temperature, water, raw materials, energy consumption, total alkalinity, volatile fatty acids, gaseous methane content, biogas residue, and biogas slurry), and preprocess the data;

[0008] Preprocessing includes: removing outliers and imputing missing values ​​by taking the mean before and after the missing values;

[0009] (2) Determine the optimal range of feature parameters through grid search, standardize to (-10, 10), randomly divide the dataset into training and test sets in a ratio of 300:80, set a fixed seed, and ensure that the algorithm is reproducible;

[0010] (3) Establish the LSTM algorithm, establish the input layer, hidden layer, output layer, learning rate, select Adam optimizer, define the loss function as mse, add overfitting mechanism, and optimize the set parameters for hyperparameters.

[0011] LSTM algorithm input gate representation:

[0012]

[0013] LSTM algorithm forget gate representation:

[0014]

[0015] LSTM algorithm output gate representation:

[0016]

[0017] In the formula, H t−1 : Previous hidden layer; X t : Feature parameters; W xi W xf W xo : Weighting parameters; b i b f b o Deviation parameter; I t F represents the input gate of the LSTM model. t Represents the forget gate in LSTM models, O t This represents the output gate of the LSTM model.

[0018] In this invention, the LSTM model input gate I t Feature input vector X t This is a 9-dimensional vector, specifically represented as Xt = [x1, x2, x3, x4, x5, x6, x7, x8, x9], where x1 is the temperature-normalized value, x2 is the TAC-normalized value, x3 is the VFA-normalized value, and so on. The model uses a weight matrix W. x Different network weights are assigned to these nine physical features, and then the input gate I is calculated. t The state. Similarly, the forget gate F t The formula then removes outdated information from data changes; finally, output gate O... t The prediction results are presented. The weighting coefficients are determined by the input feature parameters and are not fixed values.

[0019] (4) Establish the XGB algorithm, set the number of XGB trees, maximum depth, learning rate, random sampling ratio, minimum split gain, regularization L1 and L2, where L1 and L2 represent coefficients. Regularization constraints are added to XGB. L1 realizes feature selection and simplifies the model, while L2 smooths the weights and stabilizes training. This solves the oscillation problem when L1 is used alone and also makes up for the shortcoming that L2 cannot select features when used alone.

[0020] The XGB algorithm is described as follows:

[0021]

[0022] The L1 regularization of the XGB algorithm is expressed as:

[0023]

[0024] The L2 regularization of the XGB algorithm is expressed as:

[0025]

[0026] In the formula: γ, α, and λ are the regularization penalty coefficients in the XGBoost algorithm, where γ is the minimum loss decrease for leaf node splitting; α is the L1 regularization penalty coefficient; λ is the L2 regularization penalty coefficient, used to control model complexity and prevent overfitting; T: the total number of leaves on the current k-th leaf; ω j : Weight of the j-th leaf; During model iterative training, an early stopping mechanism is added to monitor the performance of the test set. When the performance of the test set no longer improves (or even declines) for several consecutive rounds, training is automatically stopped, and the model parameters with the best performance at this time are retained.

[0027] (5) Perform feature importance interpretation and SHAP (SHapley Additives Planations) analysis on the LSTM and XGB algorithms, and select appropriate dynamic weights.

[0028] Set volatility and select different weights for the two algorithms based on the intensity of data fluctuations.

[0029] Volatility is expressed as:

[0030]

[0031]

[0032] Where: σ: standard deviation; x i : Data at point i; x: Average value of the data; υ: Volatility

[0033] Volatility does not directly participate in the final numerical calculation; it serves as the basis for weight transformation (i.e., system soft switching). It determines whether the system is currently in a stable period or experiencing volatility due to extreme conditions by comparing the volatility of the current operating condition with a set threshold (0.5). Its impact lies in dynamically changing the weights (W) allocated to the LSTM and XGB base models based on the determination result. lstm and W xgb This directly alters the final combined biogas production forecast value Y. final .

[0034] Volatility is only used as an intermediate criterion for dynamic weight allocation. The final biogas production forecast value Y final The following weighted formula is used to derive:

[0035] Y final = W lstm *Y lstm + W xgb *Y xgb

[0036] Y final This is the comprehensive forecast of biogas production at the current moment. All the multi-dimensional data acquisition, independent model training, volatility calculation, and dynamic weight allocation in this invention are ultimately aimed at outputting this high-precision forecast value, thereby providing decision support for feedforward regulation of biogas projects.

[0037] The currently collected multidimensional feature data is simultaneously input into the LSTM and XGB base models to obtain Y. lstm and Y xgb The system outputs the final predicted value: Y final = W lstm *Y lstm + W xgb *Y xgb .

[0038] W lstm and W xgb These two weighting coefficients are dynamically allocated based on the 'real-time data volatility index' in practical applications. When v > 0.5, W xgb Take 0.7, W lstm The weight is set to 0.3; when the system is in a stable state, the LSTM dominates the weights. The original source of these specific weight values ​​(such as 0.7, 0.3, 0.95, 0.05) is that during the model training phase, the optimal hyperparameters were determined by minimizing the root mean square error (RMSE) through a grid search method (with a step size of 0.05).

[0039] (6) Calculate the root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²). 2 The model performance is evaluated using indicators such as ( ).

[0040] RMSE is expressed as:

[0041]

[0042] MAE is expressed as:

[0043]

[0044] Coefficient of determination R2 The expression is as follows:

[0045]

[0046] In the formula, N: number of samples; y: The predicted value of the i-th predicted sample; i : The true value of the i-th predicted sample; :average value.

[0047] The beneficial effects of this invention are:

[0048] (1) Intelligent adaptive switching mechanism breaks through static bottleneck: A "real-time volatility" index υ based on a sliding window was creatively designed. Through offline SHAP analysis, it was verified that sudden changes in key parameters such as temperature are the core factors causing data fluctuations. When υ > 0.5 triggers a fluctuation state, the system automatically tilts the weights towards the XGB model, which is extremely sensitive to local extreme features (the weights are increased to 0.7), overcoming the problem of insufficient fitting of a single LSTM at extreme points.

[0049] (2) Significantly improve prediction accuracy: The LSTM-XGB-Volatility dynamic ensemble model constructed using this invention reduces the mean absolute percentage error (MAPE) from 4.54% to 3.47% compared to the optimal single LSTM model, and the coefficient of determination (R²) is as high as 0.948, achieving high-precision tracking of biogas production.

[0050] (3) Better data preprocessing range: In view of the nonlinear characteristics of biogas, the present invention innovatively sets the normalization range to (-10, 10) through grid search, making it easier for the input data to fall into the nonlinear region of the neural network activation function, while expanding the data range and effectively resisting the noise interference of field data caused by factors such as equipment maintenance and extreme weather. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the dynamic integrated prediction method for biogas production of the present invention.

[0052] Figure 2 This is a schematic diagram of the operation of the model structure of the present invention;

[0053] Figure 3 This is a data recording diagram of the feature parameters in the embodiment;

[0054] Figure 4 This is a Spearman correlation matrix diagram showing the relationship between feature parameters in the embodiment.

[0055] Figure 5The SHAP analysis peak plot (a) and feature importance comparison plot (b) of the XGB model;

[0056] Figure 6 The SHAP analysis peak plot (a) and feature importance comparison plot (b) of the LSTM model;

[0057] Figure 7 This is a comparison chart of the test set performance of the LSTM-XGB-Volatility dynamic ensemble model of this invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings and embodiments. Those skilled in the art should understand that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0059] Example 1: The overall process of the method of the present invention is as follows Figure 1 This embodiment provides a biogas production prediction method based on LSTM-XGB dynamic integration. The specific implementation steps are as follows:

[0060] Step S1: Data Acquisition and Feature Engineering

[0061] The data comes from the operation records of an industrial-grade wet anaerobic digester. Through 13 months of recording and monitoring, 395 days of data were collected, with each day serving as a sample node.

[0062] (1) Variable selection: Nine key input characteristics of the fermentation system were collected using sensors and laboratory methods: anaerobic fermentation temperature in the fermenter (recorded by a real-time thermometer, taking the daily average temperature), material quantity (daily cow manure and cow urine), daily water consumption, daily electricity consumption (recorded by an electricity meter), total alkalinity (TAC, recorded monthly and interpolated), volatile fatty acids (VFA), methane content, and the biogas residue and biogas slurry discharged after fermentation were recorded separately after solid-liquid separation; the target variable was biogas production (recorded by a flow meter after desulfurization and impurity removal). All recorded parameter data are as follows: Figure 3 As shown: This intuitively reflects the trends and fluctuations of various input indicators such as temperature, TAC, and VFA during the data collection period; Figure 4This is a Spearman correlation matrix diagram showing the relationships between the feature parameters in the example. The diagram visually verifies the strong (linear) correlation between the selected 9-dimensional features and the target variable (biogas production) through correlation coefficients. As shown in the diagram, temperature has a high correlation coefficient of 0.75 with biogas production, exhibiting a strong positive correlation. This indicates that temperature fluctuations are a key external factor affecting anaerobic digestion biogas production efficiency. Simultaneously, internal environmental indicators such as volatile fatty acids (VFA, correlation coefficient 0.63) show a strong positive correlation, while total alkalinity (TAC, correlation coefficient -0.66) shows a strong negative correlation. This correlation analysis result demonstrates the scientific validity and rationality of selecting the above features as inputs to the prediction model, and also provides a basis for subsequent sensitivity analysis. Figure 5 and 6 This provides data support for revealing the algorithm's unique learning logic for parameters (which is not simply linear).

[0063] (2) Data cleaning: Biogas production roughly follows a normal distribution, using formula

[0064]

[0065] Outliers were removed using the established 95% confidence interval, resulting in the removal of 15 outliers. For a small number of missing values, imputation was performed by averaging the five data points before and after the missing value. After processing, 380 days of valid data were retained and divided into training and test sets at a ratio of 300:80.

[0066] (3) Normalization: Min-Max scaling is used to normalize the features to the (-10, 10) interval. Compared with the conventional (0,1) interval, the (-10,10) interval allows a larger proportion of the data to fall in the non-linear region when passing through the activation function, enhancing the model's sensitivity to fluctuations and effectively mitigating the gradient vanishing problem in deep learning.

[0067] Step S2: Single Model Construction and Explanation of the SHAP Mechanism

[0068] Two prediction base models, LSTM and XGB, were constructed. The optimal hyperparameters were determined using grid search: the LSTM model had an input dimension of 9, a hidden layer dimension of 64, a dropout rate of 0.3, and a learning rate of 0.003; the XGB model had 100 weak learners, a maximum depth of 3, and a learning rate of 0.03. Both models were trained independently until convergence on the training set. The dynamically assigned volatility threshold (0.5) and the corresponding weight coefficients for each state (e.g., 0.7, 0.3, etc.) were all optimized during the model training phase using grid search (e.g., step size set to 0.05~0.1) with the goal of minimizing the root mean square error (RMSE), thus ensuring optimal prediction accuracy.

[0069] To establish a scientific basis for dynamic weights, the inventors introduced the SHAP analysis method. For example... Figure 5 and Figure 6 As shown, the analysis revealed that all tree models (including XGB) are extremely sensitive to the "temperature" index, with their SHAP values ​​clustering sharply at both ends of -2 and +2. This reveals the physical mechanism: abrupt temperature changes are the most direct disturbance causing drastic fluctuations in local fermentation gas volume. The LSTM network, however, exhibits a more balanced capture capability, assigning higher feature weights (VFA, TAC) to the pH environment indicators characterizing the long-term stability of the system (VFA weight reaches 0.42). This physical characteristic analysis provides theoretical and technical support for the subsequent "fluctuation switching" analysis.

[0070] Step S3: Real-time volatility calculation and dynamic weight allocation

[0071] This is the core mechanism of the invention. During prediction, the system uses a sliding window to calculate the degree of fluctuation in the current operating conditions.

[0072] Set the sliding window size n=5 and the sliding step size to 1. Calculate the standard deviation σ of historical biogas production within this window and normalize it to obtain the volatility υ. Set the decision threshold to 0.5.

[0073] The system makes real-time judgments:

[0074] (1) If υ > 0.5, it indicates that the system fluctuates violently due to external disturbances (such as sudden changes in feed or cold wave). At this time, the system greatly activates the XGB model which is sensitive to extreme temperature values, dynamically assigns a weight coefficient of 0.7 to the XGB prediction base model, and the LSTM obtains a weight coefficient of 0.3 for local accurate fitting correction.

[0075] (2) If υ ≤ 0.5, it indicates that the system is running smoothly. At this time, the regression LSTM model dominates and uses its "memory gate" mechanism to capture the long-term trend of time series changes.

[0076] Step S4: Weighted output of results

[0077] The currently collected multidimensional feature data is simultaneously input into the LSTM and XGB base models to obtain Y. lstm and Y xgb The system outputs the final predicted value: Y final = W lstm *Y lstm + W xgb *Y xgb The result is then output to the central control system for adjusting subsequent feedforward controls such as heating and feeding rate. The internal structure operates as follows: Figure 2 As shown, W at this time lstm Take 0.95, W xgb Take 0.05.

[0078] The model's decision-making mechanism includes: calculating the volatility of the current input data in real time with a sliding window and making decisions accordingly; when the current data is determined to be in a smooth state, the prediction result relies on the LSTM model; when the current data is determined to be in a volatile state, the prediction weight of the XGBoost model is increased to utilize its powerful local feature capture and decision-making capabilities to promptly correct the bias of the LSTM model. Through the iterative operation of the above mechanism, the dynamic ensemble model can achieve more accurate biogas production prediction.

[0079] Example 2: Effect Evaluation and Comparative Verification

[0080] To verify the effectiveness of this invention (LSTM-XGB-Volatility), the inventors compared and evaluated it on a test set with a single LSTM model and an ensemble model based on static fixed weights (LSTM-XGB-Static, where the weights of LSTM and XGB are fixed at 0.65:0.35). The evaluation metrics were root mean square error (RMSE), mean absolute error (MAE), coefficient of determination (R²), and mean absolute percentage error (MAPE).

[0081] The following is a comparison of the evaluation metrics of the model of this invention with the performance of a single algorithm model. The verification data comparison results are as follows:

[0082] Single LSTM model: RMSE is 1204.89 m³, MAPE is 4.54%, R² is 0.9043;

[0083] LSTM-XGB-Static static model: RMSE is 1122.57 m³, MAPE is 4.22%;

[0084] This invention (LSTM-XGB-Volatility) results in a sharp decrease in RMSE to 880.62 m³, MAE to 741.97 m³, MAPE to 3.47%, and R² to 0.9480.

[0085] Combination Figure 7 The curve comparison shows that traditional single or static ensemble models deviate significantly from the true values ​​when data experiences sharp jumps (peaks and troughs in the graph). In contrast, this invention, relying on a real-time intelligent switching and allocation mechanism based on volatility, effectively compensates for the shortcomings of deep learning models in their sensitivity to extreme boundary values. It perfectly matches the data trends during periods of rapid change while maintaining accuracy during stable periods. Compared to the best single model, this invention reduces the prediction error by approximately 23.6%, demonstrating outstanding engineering application value.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A biogas production prediction method based on LSTM-XGB dynamic integration, characterized in that, Includes the following steps: Step S1: Obtain real-time operating data of the anaerobic digestion process and construct a time series feature vector. The feature vector includes parameters such as temperature, feed rate, water consumption, power consumption, total alkalinity, volatile fatty acids, methane content, biogas residue amount, and biogas slurry amount. Step S2: Input the historical time series feature vectors into the LSTM and XGB models respectively for independent iterative training to obtain the converged LSTM prediction base model and XGB prediction base model; Step S3: Obtain historical biogas production data within a set time window before the current moment, calculate the standard deviation of the data within the window, and generate a real-time data volatility index υ after normalization. Step S4: Compare the real-time data volatility index υ with the preset judgment threshold, and dynamically allocate the output weight coefficients of the LSTM prediction base model and the XGB prediction base model according to the comparison result; Step S5: Input the current operating condition feature vector into the LSTM prediction base model and the XGB prediction base model respectively to obtain independent prediction values. Combine the output weight coefficients allocated in step S4 to perform weighted fusion and output the comprehensive prediction value of biogas production at the current time.

2. The biogas production prediction method based on LSTM-XGB dynamic integration according to claim 1, characterized in that, Before obtaining and constructing the time series feature vector as described in step S1, a data preprocessing step is also included: Outliers in the original data were identified and removed using 95% confidence intervals. For data with missing values, average the five data points before and after the missing value and then fill in the missing values; The Min-Max normalization method is used to scale all feature parameters to a set range (-10, 10).

3. The biogas production prediction method based on LSTM-XGB dynamic integration according to claim 1, characterized in that, The calculation method for the real-time data volatility index υ mentioned in step S3 is as follows: Set the sliding window size to 5 and the step size to 1; the calculation formula is as follows: In the formula, n is the size of the sliding window, here n=5; x i For each data point within the sliding window; σ is the average value of the data within the sliding window; σ is the standard deviation of the current window; min and σ max These are the minimum and maximum values ​​of the standard deviation of the historical data, respectively.

4. The biogas production prediction method based on LSTM-XGB dynamic integration according to claim 1, characterized in that, The specific logic for dynamically allocating prediction weight coefficients in step S4 is as follows: The threshold value is set to 0.

5. When υ > 0.5, the system is determined to be in a fluctuating state, the XGB prediction base model is assigned a first weight coefficient of 0.7, and the LSTM prediction base model is assigned a second weight coefficient of 0.

3. When υ≤0.5, the system is determined to be in a stationary state, and the LSTM prediction base model is assigned dominant weights to output a long-term stationary trend.

5. The biogas production prediction method based on LSTM-XGB dynamic integration according to claim 1, characterized in that, The hyperparameter settings for the LSTM prediction base model in step S2 include: a hidden layer dimension of 64, a dropout rate of 0.3, and a learning rate of 0.003; the hyperparameter settings for the XGB prediction base model include: a learner count of 100, a maximum depth of 3, and a learning rate of 0.

03.

6. The biogas production prediction method based on LSTM-XGB dynamic integration according to claim 1, characterized in that, Biogas production forecast Y final The following weighted formula is used to derive the following: Y final = W lstm *Y lstm + W xgb *Y xgb The currently collected multidimensional feature data is simultaneously input into the LSTM and XGB base models to obtain Y. lstm and Y xgb The system outputs the final predicted value: Y final = W lstm *Y lstm + W xgb *Y xgb .

Images