Long short-term memory network industrial soft-sensing method for desulfurization process flue gas

Abstract

The present application relates to the technical field of industrial soft measurement, especially a long and short term memory network industrial soft measurement method for desulfurization process flue gas, comprising: collecting input and output data in the desulfurization process of a power plant to form a historical sample database; preprocessing the collected sample data; establishing a robust LSTM model with Huber loss and l1 regularization and performing outlier detection on the data; traversing the historical data with a sliding window, taking the difference between the data after the window and the statistics in the reference window to obtain a time series S1; calculating the upper and lower edges of the S1 box plot as the normal range; if the difference between the current value and the statistics in the reference window is not within the normal range, it is abnormal; applying the above LSTM model to SO2 concentration prediction in the desulfurization process of a thermal power plant. The simulation results show that the proposed model has good anti-interference ability for outliers and non-Gaussian noise, reduces the influence of redundant variables, thereby improving the prediction performance of LSTM, and has good economy and applicability.

Description

Technical Field

[0001] This invention relates to the field of data-driven industrial soft measurement technology, specifically to an industrial soft measurement method using long short-term memory networks for desulfurization process flue gas. Background Technology

[0002] In modern industrial processes, it is essential to monitor and control key performance variables online in real time to ensure process safety, production efficiency, and product quality. However, some performance variables, such as slurry composition, fluid viscosity, and gas concentration, are difficult to measure directly using hardware sensors. Data-driven soft measurement uses historical data collected by process automation systems to build mathematical models, providing a fast, low-cost, and easily implemented alternative for the real-time monitoring and control of key variables in industrial processes, and has therefore received widespread research and application.

[0003] However, the following problems still exist in the current soft measurement modeling of industrial processes: (1) Due to the complex physicochemical reactions and structures, industrial processes usually exhibit strong nonlinear characteristics; (2) There are often many redundant variables in industrial processes, which makes the data-driven soft measurement model too complex and may lead to overfitting; (3) The different position distribution of each sensor in the process will lead to a large time delay between the measurement results of each variable; (4) The actual industrial process will be affected by environmental interference, sensor drift and system failure, and the measurement data will have non-Gaussian noise and outliers.

[0004] In the desulfurization process of thermal power plants, the SO2 concentration in the flue gas emitted from the secondary absorption tower is a key performance indicator related to pollutant emission standards and energy consumption. However, this indicator is often unstable and difficult to measure directly using hardware sensors. Furthermore, specialized online analyzers are typically expensive and time-consuming. Therefore, it is necessary to design soft sensors to provide calibration information and improve system reliability. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides an industrial soft measurement method for long short-term memory networks for desulfurization process flue gas.

[0006] This invention provides the following technical solution: an industrial soft measurement method for long short-term memory networks for desulfurization process flue gas, comprising the following steps:

[0007] Step 1: Collect input and output data from the power plant's desulfurization process to form a historical sample database;

[0008] Step 2: Preprocess the collected sample data:

[0009] Step 3: Use a robust LSTM model with Huber loss and L1 regularization to detect outliers in the data. Use a sliding window to traverse the historical data, and subtract the statistic in the reference window from the next data point in the window to obtain a new time series S1. Next, calculate the upper and lower edges of the S1 box plot as the normal range. Finally, if the difference between the current value and the statistic in its reference window is not within the normal range, it is considered an outlier.

[0010] Step 4: Apply the above LSTM model to predict SO2 concentration in the desulfurization process of thermal power plants.

[0011] The LSTM model described is a robust LSTM model with Huber loss and l1 regularization. First, to improve the LSTM's resistance to non-Gaussian noise and outliers, Huber loss is used instead of the classic LSTM's MSE loss. Second, to reduce the complexity of the LSTM model and improve its generalization performance, l1 regularization is added to the loss function.

[0012] In step 2, for the original time series Using a sliding window m, construct a multi-step time series for LSTM. Where L is the length of the original time series, x t Let d represent the input data at time t. t denoted by t, p represents the prediction step size, and m determines the memory capacity of the LSTM.

[0013] I. Algorithm Basics

[0014] 1. LSTM Neural Network Algorithm

[0015] LSTM was proposed to overcome the gradient vanishing and gradient exploding problems of standard RNNs. It controls the information flow by designing three gate units. Under the control of these three gates, LSTM can forget previously useless information and retain useful information in the memory unit, thus possessing the ability to predict time series data. Due to these advantages, LSTM has been extensively studied in the field of soft sensors. The structure of LSTM in the research is as follows... Figure 1 As shown.

[0016] The specific process of information processing: First, the memory unit c from the previous moment... t-1 Through the forgetting gate f t Forget some useless information. Secondly, based on the current input x... t and the external state h of the previous time step t-1 Calculate the new information at the current moment By inputting gate i t Control, write it into the internal state c tThen activate it. Finally, by output gate o t Control, based on the information currently accumulated t Select relevant information and pass it to the external state h. t The LSTM computation process is represented as follows:

[0017] f t =σ(W f ·[h t-1 x t ]+b f (1)

[0018] i t =σ(W i ·[h t-1 x t ]+b i (2)

[0019] o t =σ(W o ·[h t-1 x t ]+b o (3)

[0020]

[0021]

[0022] h t =o t ⊙tanh(c t (6)

[0023] In the formula W c W o W f , and b c b o b f , Here, represents the weight matrix and vector bias, respectively; s and u represent the number of input and hidden layer neurons, respectively. ⊙ represents the vector dot product, and σ(·) and tanh(·) are the sigmoid and tanh activation functions, respectively. Given the output h... t The final estimate is obtained by the following formula:

[0024]

[0025] w v and b v These are the final regression coefficient vector and the bias, respectively.

[0026] 2. L1 regularization

[0027] LASSO reduces or avoids overfitting by compressing model parameters. A linear regression model is given and introduced into ordinary least squares (OLS) estimation:

[0028]

[0029] w T = [w1, w2, ..., w q ] is the model coefficient vector. x = [x1, x2, ..., x] q ] T d represents the vector of input variables, and d represents the output variable. and These are the input and output data, respectively. N represents the total number of samples, and λ represents the regularization coefficient.

[0030] In equation (8), the first term is the residual constraint term, used to control the approximation degree of the solution, and the second term is called the L1 penalty, used to compress the weight vector. λ is a non-negative adjustable parameter used to balance accuracy and noise sensitivity. When the value of λ is large enough, the coefficients of some variables with low correlation are compressed to 0, achieving variable selection. When λ = 0, it no longer has the effect of constraint, and the weight vector... Keep the initial estimate the same as that of OLS. Therefore, the accuracy of the optimal model is largely affected by λ.

[0031] 3. Robust estimation of Huber loss

[0032] Huber loss is a robust adaptive loss function that combines quadratic and linear loss. The expression for Huber loss is:

[0033]

[0034] The choice of δ determines the emphasis on quadratic or linear loss. The function uses quadratic loss for small residuals to ensure the model more accurately obtains the global optimum, and linear loss for large residuals to suppress sensitivity to non-Gaussian noise and outliers. Therefore, Huber loss combines the advantages of both quadratic and linear losses, being robust to outliers and differentiable everywhere. The hyperparameter δ can be determined using cross-validation.

[0035] II. Designed Soft Measurement Algorithm

[0036] 1. l1-HM-LSTM Dynamic Soft Measurement Model

[0037] Using traditional LSTM as the basic model for handling nonlinearity and time delay, a novel LSTM loss function is designed. This function combines the excellent resistance of Huber loss to non-Gaussian noise with the excellent model reduction capability of L1 regularization. In other words, the loss function in a robust LSTM model with Huber loss and L1 regularization is:

[0038]

[0039]

[0040] λ controls the shrinkage strength of the weight matrix, and δ determines the quadratic or linear nature of the Huber loss.

[0041] 2. A method for updating and calculating l1-HM-LSTM networks based on BPTT

[0042] Backpropagation gradient balancing (BPTT) is a method for calculating parameter gradients in recurrent neural networks. Parameter gradients can be calculated and stored sequentially, improving computation speed. In this research, a specific BPTT training algorithm was developed for the proposed l1-HM-LSTM.

[0043] First, regarding the output gradient:

[0044]

[0045] The external state of the LSTM at time t is h. t The error term at time t is defined as

[0046]

[0047] f t i t , o t The corresponding error terms are defined as follows:

[0048] θ f，t =W f [h t-1 x t ]+b f，t =W fh h t-1 +W fx x t +b f，t (13)

[0049] θ i，t =W i [h t-1 x t ]+b i，t =W ih ht-1 +W ix x t +b i，t (14)

[0050]

[0051] θ o，t =W o [h t-1 x t ]+b o，t =W oh h t-1 +W ox x t +b o，t (16)

[0052] here

[0053] At time t-1, the error term propagating backwards along time is defined as follows:

[0054]

[0055] From equations (1)-(4), we can see that f t i t , o t All are h t-1 The function. According to the chain rule:

[0056]

[0057] Next, calculate each partial derivative of equation (18). Using the output gate o t For example, according to formula (6), The partial derivatives are

[0058]

[0059] Similarly, because o t =σ(θ) o，t We get

[0060] From equation (16), we can obtain

[0061]

[0062] By calculating all the partial derivatives in equation (18), we obtain

[0063]

[0064] By definition, δ o，tδ f，t δ i，t , It can be represented as

[0065]

[0066]

[0067]

[0068]

[0069] Formulas (22)-(25) are the formulas for the backward propagation of the error at time t. Substituting into formula (21), we obtain the error term. for

[0070]

[0071] Note that we only provide the changes applied to the output gates of the LSTM network; the changes to other gates can be derived in a similar manner.

[0072] W oh The weight gradient is the sum of the gradients at each time step. At time t, W oh，t The gradient can be expressed as

[0073]

[0074] here

[0075]

[0076] Ultimately W oh The gradient is

[0077]

[0078] For W ox Weight gradient:

[0079]

[0080]

[0081] For b o bias gradient

[0082]

[0083]

[0084] According to W oh W ox and b oFollowing the same derivation process, the gradient update equations for other weights and biases can be obtained as follows:

[0085]

[0086]

[0087]

[0088] After completing the BPTT algorithm derivation for the proposed LSTM, Adam is used to perform gradient updates.

[0089] 3. Solving the Huber loss using the sliding window cross-validation method and the λ value in l1 regularization.

[0090] The algorithm has two types of hyperparameters. The first type is the conventional training hyperparameters of the LSTM itself, such as the learning rate, which can be determined through experience and some experiments. The second type is the high-level hyperparameters that determine the robustness and regularization of the algorithm, namely δ in Huber loss and λ in L1 regularization. These are selected through grid search and moving window cross-validation (MWCV).

[0091] Grid search: This is a method to optimize model performance by iterating through all candidate parameters. The process can be divided into two steps. (1) Set the hyperparameters δ and λ in the given ranges [0.1, 1] and [0, λ], respectively. ub Uniformly distributed within λ ub It is a sufficiently large value that causes the model to underfit. (2) Find all possible combinations of hyperparameters from the defined range. (3) Find the best combination of hyperparameters through two loops, where the inner loop uses the MWCV method to evaluate the average model performance and the outer loop iterates through each possible hyperparameter.

[0092] MWCV: Cross-validation, as a performance evaluation method, is often used to optimize the hyperparameters of a model. Considering that the order of data is crucial for time series problems, MWCV is used to partition the training and validation datasets to determine the optimal δ and λ for a robust LSTM. A schematic diagram of the algorithm is shown below. Figure 2 As shown, multiple windows that move synchronously along a subset of the dataset with a step size of 1 are constructed to segment the validation dataset Z. validate =Z k+1 The union of the subsets prior to the validation dataset is used as the training dataset Z. train = [Z1, Z2...Z k After calculating all possible combinations of δ and λ using the above method, the combination with the minimum validation loss is selected as the optimal [δ, λ] for l1-HM-LSTM. The principle of this algorithm is as follows: Figure 4 As shown.

[0093] Based on the above description, the overall process of this solution is as follows: 1. Use the moving regression method on the initial dataset. Preprocessing is performed to construct a multi-step time series dataset D′={{X1,X2,...,X n-p},{d p+m d p+m+1 , ..., d L}}, The entire dataset is divided into two parts: the first 80% is training data and the rest is test data. 2. Considering the influence of redundant variables, outliers and non-Gaussian noise, Huber loss and l1 regularization are introduced into the standard LSTM, and the loss function of LSTM is redefined to improve the robustness and prediction accuracy of the algorithm. 3. In order to obtain a reliable and stable model, the optimal [δ, λ] is determined by moving window cross-validation to adapt to the error distribution of the desulfurization process dataset of thermal power plants and reduce the complexity of the LSTM model, thereby improving the generalization performance of the model. 4. The proposed LSTM is trained using a specific BPTT algorithm, and the calculation formulas are shown in (27)-(36). Based on the corresponding error terms, the Adam optimization algorithm is used to update the network weights.

[0094] In summary, the robust LSTM soft measurement algorithm based on Huber loss and L1 regularization can effectively handle short-term and long-term dependencies between input and output. It also has good anti-interference ability against outliers and non-Gaussian noise during model training and helps to reduce the influence of redundant variables, thereby improving the prediction performance of LSTM. It has good economy and applicability. Attached Figure Description

[0095] Appendix Figure 1 This is a diagram of the LSTM cell structure.

[0096] Appendix Figure 2 This is a diagram illustrating the principle of cross-validation for moving windows.

[0097] Appendix Figure 3 This is a process flow diagram of a dual-tower desulfurization system.

[0098] Appendix Figure 4 This is an outlier detection graph.

[0099] Appendix Figure 5 This invention provides a soft-sensor modeling method for predicting SO2 concentration in desulfurized flue gas emissions. Detailed Implementation

[0100] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of specific embodiments. Obviously, the described specific embodiments are merely one specific embodiment of the present invention, and not all specific embodiments. Based on the specific embodiments of the present invention, all other specific embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0101] The attached diagram illustrates the industrial soft measurement method using long short-term memory networks for desulfurization process flue gas, which includes the following steps:

[0102] Step 1: Collect input and output data from the power plant's desulfurization process to form a historical sample database; a candidate input variable set consisting of 35 auxiliary variables is shown in Table 1, with SO2 concentration being the key response variable. A total of 2676 sets of process data were collected at one-minute sampling intervals.

[0103] Table 1 Candidate Input Variables for Desulfurization System

[0104]

[0105]

[0106] Step 2: Preprocess the collected sample data:

[0107] When building the model, a moving regression method needs to be introduced to preprocess the data. For the original time series... Using a sliding window m, construct a multi-step time series for LSTM, such as... Where L is the length of the original time series, x t Let d represent the input data at time t. t denoted by t, p represents the prediction step size, and m determines the memory capacity of the LSTM.

[0108] Step 3: Design a novel loss function for LSTM: This invention discloses a robust LSTM model with Huber loss and l1 regularization to predict SO2 concentration. First, to improve the LSTM's resistance to non-Gaussian noise and outliers, Huber loss is used instead of the classic LSTM's MSE loss. Second, to reduce the complexity of the LSTM model and improve its generalization performance, l1 regularization is added to the loss function.

[0109] A specific backpropagation through time (BPTT) training algorithm was developed: In this invention, a specific BPTT training algorithm was developed for the proposed LSTM. The BPTT algorithm is used to calculate the chain derivative, and the gradient update is performed using adaptive moment estimation (Adam).

[0110] In determining δ in Huber loss and λ in l1 regularization, grid search and moving window cross-validation are used to obtain a reliable and stable model.

[0111] Outliers are detected using the Anomaly Detection Toolkit (ADTK). First, a sliding window is used to traverse the historical data, and the difference between the next data point in the window and the statistic in the reference window is calculated to obtain a new time series S1. Second, the upper and lower edges of the box plot of S1 are calculated as the normal range. Finally, if the difference between the current value and the statistic in its reference window is not within the normal range, it is considered an anomaly.

[0112] The loss function in a robust LSTM model with Huber loss and L1 regularization is:

[0113]

[0114]

[0115] Perform parameter gradient training on the loss function.

[0116] For the output gradient:

[0117]

[0118] The external state of the LSTM at time t is h. t The error term at time t is defined as

[0119]

[0120] f t i t , o t The corresponding error terms are defined as follows:

[0121] θ f，t =W f [h t-1 x t ]+b f，t =W fh h t-1 +W fx x t +b f，t

[0122] θ i，t =W i [h t-1 x t ]+b i，t =W ih h t-1 +W ix x t +b i，t

[0123]

[0124] θ o，t =W o [h t-1 x t ]+b o，t =W oh h t-1 +W ox x t +b o，t

[0125] here

[0126] At time t-1, the error term propagating backwards along time is defined as follows:

[0127]

[0128] f t i t , o t All are h t-1 The function. According to the chain rule:

[0129]

[0130] The partial derivatives of the above equation are calculated to obtain

[0131]

[0132] δ o，t δ f，t δ i，t , The formula for the backward propagation of the error at time t is expressed as follows:

[0133]

[0134]

[0135]

[0136]

[0137] Obtain the error term for

[0138]

[0139] W oh The weight gradient is the sum of the gradients at each time step. At time t, W oh，t The gradient can be expressed as

[0140]

[0141] here

[0142]

[0143] Ultimately W oh The gradient is

[0144]

[0145] For W ox Weight gradient:

[0146]

[0147]

[0148] For b o bias gradient

[0149]

[0150]

[0151] According to W oh W ox and b o Following the same derivation process, the gradient update equations for other weights and biases are obtained as follows:

[0152]

[0153]

[0154]

[0155] After completing the gradient derivation of the proposed LSTM model, Adam is used to perform gradient updates.

[0156] Step 4: Apply the above LSTM model to predict SO2 concentration in the desulfurization process of thermal power plants.

[0157] The thermal power plant employs a dual-absorption tower structure and limestone-gypsum wet flue gas desulfurization (FGD) technology. The entire FGD process mainly includes a limestone slurry preparation system, a reaction absorption system, and a flue gas system. A schematic diagram of this process is shown below. Figure 3 As shown in the diagram, the SO2 concentration in the flue gas emitted from the secondary absorption tower is a key performance indicator in this system. Strict adherence to national environmental standards is required during production. However, this indicator fluctuates frequently and with large amplitudes, making it generally unstable. Online analyzers require regular maintenance and calibration, and their performance degrades due to aging or accumulated faults. Therefore, it is necessary to design a soft sensor to continuously provide SO2 concentration predictions. This not only temporarily replaces the function of the online analyzer but also allows for the verification and fault identification of the online analyzer.

[0158] Figure 4 The data shows the outliers captured in ADTK, totaling 71, which may affect prediction accuracy. To demonstrate the superiority of the l1-HM-LSTM algorithm, this paper compares its performance with LSTM, LASSO-LSTM, and NNG-MLP soft measurement algorithms. This invention uses two model performance evaluation metrics:

[0159] 1) Root Mean Square Error (RMSE):

[0160]

[0161] 2) Mean Absolute Error (MAE)

[0162]

[0163] Where d i and These are the true value and the predicted value of the i-th test sample, respectively, and N is the true value and the predicted value. test This represents the total number of samples in the test set.

[0164] Table 2 shows the best experimental results for each model after ten runs.

[0165] Table 2 Simulation results of different algorithms for SO2 prediction

[0166]

[0167] The comparison results show that the traditional LSTM model performs significantly worse due to the presence of numerous irrelevant variables and non-Gaussian noise. Although NNG-MLP and LASSO-LSTM can select input variables and simplify the model, their performance is still not ideal. The proposed l1-HM-LSTM outperforms other algorithms on all criteria, indicating that model redundancy and non-Gaussian noise in the measurement data are well handled during training. Our method exhibits better generalization performance and higher robustness in industrial processes with non-Gaussian distributions and outlier characteristics. Its prediction results are as follows: Figure 5 As shown in the figure, the fitting curves demonstrate that the proposed algorithm can accurately follow the dynamic changes in SO2 concentration, exhibiting good estimation performance and verifying the superiority of the proposed algorithm.

[0168] Considering that soft sensing in actual industrial processes is often affected by multivariates, time series, and non-Gaussian noise, the algorithm proposed in this invention fully utilizes LSTM to effectively capture the short-term and long-term dependencies between inputs and outputs, establishes temporal correspondences between variables, and enhances the model's dynamic information processing capabilities. This is particularly useful for predicting output responses in relatively continuous processes. Furthermore, the Huber loss, which integrates linear and quadratic losses, exhibits better adaptability to error distributions. Applying L1 regularization can reduce model complexity, thereby improving model performance.

[0169] In summary, this invention discloses an efficient soft-sensing model for SO2 concentration in flue gas emitted during desulfurization processes. Simulation results show that the proposed algorithm can accurately predict SO2 concentration during the process. Comparison with other methods verifies the generalization performance and robustness of this approach. Furthermore, l1-HM-LSTM can screen out key control variables consistent with on-site plant operations, demonstrating the model's practicality and providing a reference for the optimization of desulfurization systems.

[0170] Although specific embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these specific embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An industrial soft sensing method using long short-term memory networks for desulfurization process flue gas, characterized in that... Includes the following steps: Step 1: Collect input and output data from the power plant desulfurization process to form a historical sample database; Step 2: Preprocess the collected sample data: Step 3: Build an LSTM model for outlier detection. Use a sliding window to traverse the historical data and subtract the statistic in the reference window from the next data point in the window to obtain a new time series S1. Secondly, the upper and lower edges of the S1 box plot are calculated as the normal range; finally, if the difference between the current value and the statistic in its reference window is not within the normal range, it is considered abnormal. Step 4: Apply the above LSTM model to predict SO2 concentration in the desulfurization process of thermal power plants; The LSTM model described is one with Huber loss and The regularized robust LSTM model firstly employs Huber loss instead of the classic LSTM's MSE loss to improve the LSTM's resistance to non-Gaussian noise and outliers; secondly, to reduce the complexity of the LSTM model and improve its generalization performance, a new loss function is added to the loss function. Regularization; After completing the gradient derivation of the proposed LSTM model, Adam is used to perform gradient updates; With Huber loss and The loss function in a regularized robust LSTM model is ； LSTM in The external state at time t is ,exist The error term at time is defined as , , , , The corresponding error terms are defined as follows: here ; When the output gate is hour, The weight gradient is the sum of the gradients at each time step. t time, The gradient can be expressed as here final The gradient is This refers to the external state at the previous moment. For input gate, The internal state at the current moment. For output gate, The external state at the current moment. , , , and , , , Here, represents the weight matrix and vector bias, respectively, and s and u represent the number of input and hidden layer neurons, respectively. For vector dot product, The tanh activation function; For the current input, and These are the final regression coefficient vector and the bias, respectively.

2. The industrial soft sensing method for long short-term memory networks for desulfurization process flue gas according to claim 1, characterized in that, In step 2, for the original time series Using sliding windows To construct multi-step time series for LSTM, , ,in The length of the original time series. Indicates the first Input data at any time, Indicates the first Real-time response output, Indicates the prediction step size. This determines the memory capacity of the LSTM.

Images