SO2 concentration prediction method based on two-stage attention and double-flow memory regulation GRU

Abstract

The application discloses a SO2 concentration prediction method based on a two-stage attention and a two-flow memory adjusted GRU, belongs to the field of industrial process soft measurement, and comprises the following steps: after a DS-MRGRU network is constructed, feature attention is introduced in a feature extraction stage to capture key features and inhibit irrelevant features and noises; and in a prediction output stage, timing attention is introduced to capture long time sequence dependency relationship, so that an FTA-DS-MRGRU soft measurement model with enhanced key information extraction capability is constructed for predicting the SO2 content of net flue gas. The application extracts historical information and key features of current input by means of the DS-MRGRU to generate complementary information flow, enhances the generalization capability and prediction performance of the model, inhibits redundant information by fusing feature attention and timing attention, improves the model interpretability, mines important time information in history, improves the long time sequence prediction effect, and realizes accurate prediction of the SO2 content of net flue gas.

Description

Technical Field

[0001] This invention relates to a method for predicting SO2 concentration based on a two-stage attention and two-stream memory-regulated GRU, belonging to the field of modern industrial process soft measurement. Background Technology

[0002] In complex industrial processes, it is necessary to measure certain key quality variables that are difficult to measure directly or in real time but are closely related to product quality and production safety in real time to meet process control requirements. Soft sensor technology can use easily measurable auxiliary variables in industrial processes to estimate difficult-to-measure dominant variables in real time. Due to its economical and reliable online detection, rapid response, and ease of achieving real-time monitoring and control of product quality, it has been widely used in process manufacturing and process control fields. For example, taking the desulfurization process in a thermal power plant as an example, this process requires the measurement of SO2 flue gas emission concentration using a soft sensor model to ensure that emissions comply with national air pollutant emission standards.

[0003] Soft measurement modeling methods can generally be divided into three categories: mechanism-based modeling, data-driven modeling, and hybrid mechanism-data-driven modeling. Mechanism-based modeling methods rely heavily on a deep understanding of process mechanisms, resulting in complex modeling processes and difficult solutions. Data-driven modeling methods, on the other hand, do not overly depend on process mechanisms; they only need to extract key feature information from process data to build a soft measurement model. Among these, soft measurement methods based on artificial neural networks have become a research hotspot in the field due to their excellent modeling performance. In recent years, deep neural networks have developed rapidly, and their feature representation capabilities have become increasingly enhanced. Gated recurrent neural networks (GRNNs), by introducing temporal feedback mechanisms and threshold unit structures, can not only effectively describe the dynamic temporal characteristics between data but also alleviate the gradient vanishing and exploding problems present in standard RNNs. Common GRNNs include Long Short-Term Memory (LSTM) neural networks and Gated Recurrent Unit (GRU) neural networks. Among them, the GRU network is developed based on the LSTM network and is an important variant of the Long Short-Term Memory network. It not only alleviates the long-term dependency problem of RNN, but also has a simple internal structure and can improve the network's computational efficiency, which has attracted widespread attention from researchers in the field.

[0004] The similarity between GRU and LSTM lies in that each neuron is a processing unit containing several threshold controllers, each used to determine the usefulness of input information and control the inflow of relevant information. The difference is that each GRU processing unit contains only two threshold controllers and one timing output, thus having fewer parameters and a simpler network structure. Figure 2 As shown. The GRU network threshold mechanism consists of two threshold controllers, a reset gate and an update gate, and candidate hidden states. The reset gate determines whether the candidate hidden state at the current time depends on the hidden state at the previous time and the degree of dependence; the update gate controls the update of the hidden state from time t-1 to time t; and the candidate hidden state controls the retention of relevant input information. The expressions for GRU are shown in equations (1) to (3) below:

[0005] r t =σ(W r x t +U r h t-1 +b r (1)

[0006] z t =σ(W z x t +U z h t-1 +b z (2)

[0007]

[0008] Where, x t and h t-1 These represent the current input and the previous implicit state output, respectively; r t and z t These are the current time step reset gate and update gate output, respectively, and their activation function σ(·) is the sigmoid function; The output of the candidate hidden state at the current time step has an activation function tanh(·) that is a hyperbolic tangent function; W r W z and U is the input weight vector to the network. r U z and b is the hidden weight vector. r b z and Here, is the bias vector; ⊙ represents the Hadamard product operation. The hidden state h at the current time step. t The expression is shown in formula (4):

[0009]

[0010] As mentioned above, the GRU neural network achieves information updating and dynamic memory, and persists information retention, through its threshold controller and temporal feedback mechanism. By introducing a temporal feedback mechanism and threshold unit structure, the GRU neural network can not only effectively describe the dynamic temporal characteristics between data, but also alleviate the gradient vanishing and exploding problems existing in standard RNNs. Moreover, the GRU neural network is developed based on the LSTM neural network, and its internal structure is simpler, which can effectively improve the network's computational efficiency.

[0011] However, while GRU networks can effectively uncover long-term dependencies between time-series data, they struggle to capture the characteristics of different variables at different time steps. Furthermore, the single-gating mechanism used in GRU networks to update hidden states leads to linear constraints during computation, limiting the full flow of information and thus impacting model prediction performance. Simultaneously, with the continuous development and application of industrial big data technology, it has been found that in actual industrial process modeling, information redundancy and temporal information decay weaken the model's ability to capture key information and long-term dependencies, resulting in poor prediction performance. Summary of the Invention

[0012] To address the problem that existing soft measurement technologies fail to adequately consider the nonlinearity, dynamism, information redundancy, and temporal information decay of complex industrial processes, which weakens the soft measurement model's ability to capture key information and long-term dependencies, leading to decreased accuracy of soft measurement results, this invention provides an SO2 concentration prediction method based on a two-stage attention and dual-stream memory-regulated GRU. The technical solution is as follows:

[0013] As one aspect of the present invention, a method for predicting SO2 concentration based on a GRU regulated by two-stage attention and dual-stream memory is provided, comprising:

[0014] S100. After obtaining historical data of process variables and target variables in the flue gas desulfurization process of thermal power plants, preprocessing is performed to obtain several time series of historical data, and these time series are divided into training set and test set; among them, the target variable is the net SO2 concentration of flue gas at the outlet of the secondary absorption tower, and the process variables are other parameters in the flue gas desulfurization process of thermal power plants.

[0015] S200. Construct a DS-MRGRU network for extracting the temporal and causal relationships between current and historical information, including a parallel temporal correlation flow information extraction structure and a dynamic causal correlation flow information extraction structure;

[0016] S300. A feature attention module is introduced in the feature extraction stage of the DS-MRGRU network to capture key features and suppress irrelevant features and noise. A temporal attention module is introduced in the prediction output stage of the DS-MRGRU network to capture long-term sequence dependencies, thus constructing an FTA-DS-MRGRU soft measurement model with enhanced key information extraction capabilities.

[0017] S400: The FTA-DS-MRGRU soft measurement model is trained using the training set in S100 through the backpropagation algorithm, and the FTA-DS-MRGRU soft measurement model is tested using the test set in S100 to obtain the target FTA-DS-MRGRU soft measurement model.

[0018] S500: Obtain real-time data of process variables in the flue gas desulfurization process of thermal power plants, input the target FTA-DS-MRGRU soft measurement model, and obtain the SO2 content of the clean flue gas after the flue gas desulfurization process of thermal power plants.

[0019] Furthermore, the time-related stream information extraction structure includes a first time-gate-based TGRU layer, a Dropout layer, and a second time-gate-based TGRU layer connected in sequence.

[0020] The expressions for the first and second time-gate-based TGRU layers are:

[0021]

[0022] in, This represents the input of the TGRU layer at time t. This represents the time-dependent hidden state output at time t-1 in the input time series of the TGRU layer, where σ(·) represents the sigmoid activation function. The output of the candidate time-related hidden state at time t in the input time series of the TGRU layer is represented by ⊙, which represents the Hadamard product operation. T U represents the input weight vector of the TGRU layer network. T b represents the hidden weight vector of the TGRU layer. T This represents the bias vector of the TGRU layer. This indicates the update gate corresponding to the TGRU layer. This represents the input weight vector of the update gate corresponding to the TGRU layer. This represents the hidden weight vector of the update gate corresponding to the TGRU layer. This represents the bias vector of the update gate corresponding to the TGRU layer.

[0023] Furthermore, the dynamic causal correlation flow information extraction structure includes a first causal gate-based CGRU layer, a Dropout layer, and a second causal gate-based CGRU layer connected in sequence.

[0024] The expressions for the first causal gate-based CGRU layer and the second causal gate-based CGRU layer are:

[0025]

[0026] in, This represents the input to the CGRU layer at time t. This represents the causal correlation of the hidden state output at time t-1 in the input time series of the TGRU layer, where σ(·) represents the activation function, the sigmoid function. W represents the output of the candidate causally related hidden states at time t in the input time series of the CGRU layer. ⊙ represents the Hadamard product operation. c U represents the input weight vector of the CGRU layer network. c b represents the hidden weight vector of the CGRU layer. C This represents the bias vector of the CGRU layer. This indicates the update gate corresponding to the CGRU layer. This represents the input weight vector of the update gate corresponding to the CGRU layer. This represents the hidden weight vector of the update gate corresponding to the CGRU layer. This represents the bias vector of the update gate corresponding to the CGRU layer.

[0027] Furthermore, the S300 includes:

[0028] S310. After assigning feature attention weights to the input time series through the feature attention module, the time series is weighted according to the feature attention weights, and the weighted time series is output.

[0029] S320. Extract the temporal relationship in the weighted time series through the time gate in the time-related flow information extraction structure to obtain the time-related hidden state output, and extract the causal relationship in the weighted time series through the causal gate in the dynamic causal flow information extraction structure to obtain the causal hidden state output.

[0030] S330. After assigning first temporal attention weights to the time-related hidden state outputs through the first temporal attention module and then weighting them, the comprehensive temporal features of the historical key points of attention are obtained. After assigning second temporal attention weights to the causal related hidden state outputs through the second temporal attention module and then weighting them, the comprehensive causal features of the historical key points of attention are obtained.

[0031] S340. After fusing the integrated temporal features and integrated causal features through the merging layer, the final output result is obtained through the fully connected layer.

[0032] Furthermore, the expression for the feature attention module is:

[0033] q t1 =W q 1 x t +b q 1

[0034] e t i =V1 i σ(W1 i q t1 +U1 i x t i +b1 i ), 1≤i≤n

[0035]

[0036] Where, x t Let x represent the input of the feature attention module at time t. t i q represents the i-th variable at time t in the input time series. t1 e represents the query vector of the feature attention module. t i α represents the feature attention score corresponding to the i-th variable at time t in the input time series. t i This represents the feature attention weight corresponding to the i-th variable at time t in the input time series. This represents the weighted input vector after passing through the feature attention module, where n represents the number of input features, and W... q 1 b q 1 W1 i U1 i b1 i V1 i Both indicate that the feature attention parameters are continuously updated and optimized during the model training process.

[0037] Furthermore, the expressions for the first temporal attention module and the second attention module are:

[0038] q t2 =W q 2 h t +b q 2

[0039] g j =V2 j σ(W2 j q t2 +U2 j h t-T+j +b2 j ), 1≤j≤T

[0040]

[0041]

[0042] Among them, h t q represents the input of the temporal attention module at time t. t2 G represents the query vector of the temporal attention module. j β represents the temporal attention score corresponding to the hidden state at the j-th time step in the input time series. j c represents the temporal attention weight corresponding to the hidden state at the j-th time step in the input time series. t The vector represents the weighted sum of the outputs of each hidden state, where T represents the size of the chosen time step, and W represents the value of the hidden state. q 2 b q 2 W2 j U2 j b2 j V2 j Both refer to the temporal attention parameters that are updated and optimized during model training.

[0043] Furthermore, S100 includes:

[0044] S110. After obtaining the historical data of process variables and target variables in the flue gas desulfurization process of thermal power plants, perform missing value processing, outlier processing and data standardization in sequence.

[0045] S120. Using the sliding window technique, the standardized historical data is transformed into several time series of length T, and the time series are divided into training set and test set.

[0046] Furthermore, the FTA-DS-MRGRU soft measurement model in S400 is trained using the training set in S100 via backpropagation algorithm, including:

[0047] The training set in S100 is used to perform forward computation on the FTA-DS-MRGRU soft measurement model to calculate the network weights of the FTA-DS-MRGRU soft measurement model.

[0048] The corresponding error term is obtained by calculating the loss function value of the FTA-DS-MRGRU soft measurement model using the semi-root mean square error loss function. The calculation formula is as follows:

[0049]

[0050] Where N represents the total number of samples in the test set, y t This indicates the true value of SO2 concentration in the clean flue gas. This represents the predicted SO2 concentration in the clean flue gas;

[0051] Based on the corresponding error term, the Adam optimization algorithm is used to update the gradient of the network weights.

[0052] As another aspect of the present invention, a target detection method is provided, wherein the method employs the above-described method to detect a gas to be detected, the gas to be detected being a gas generated in an industrial process.

[0053] As another aspect of the present invention, a computer device is provided, including a processor and a memory, the memory being used to store computer instructions, which are loaded and executed by the processor to implement the above-described SO2 concentration prediction method based on a two-stage attention and dual-stream memory-regulated GRU. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A simplified flowchart of the SO2 concentration prediction method based on two-stage attention and dual-stream memory-regulated GRU provided by the present invention;

[0056] Figure 2 This is a structural diagram of the gated recurrent unit (GRU) in the soft measurement modeling method of the present invention;

[0057] Figure 3 A structural diagram of the MRGRU provided by this invention;

[0058] Figure 4 A structural diagram of the feature attention module provided by this invention;

[0059] Figure 5 A structural diagram of the temporal attention module provided by this invention;

[0060] Figure 6 The structural diagram of the FTA-DS-MRGRU provided by this invention;

[0061] Figure 7 A simplified schematic diagram of the flue gas desulfurization process of a thermal power plant provided by the present invention;

[0062] Figure 8 Feature attention weighting diagram for predicting SO2 concentration in the outlet flue gas of the secondary absorption tower provided by this invention;

[0063] Figure 9 The time-series attention weighting diagram for predicting SO2 concentration in the net flue gas at the outlet of the secondary absorption tower provided by this invention.

[0064] Figure 10A This is a comparison chart of the actual and predicted SO2 concentration emissions from the outlet of a secondary absorber based on GRU.

[0065] Figure 10B This is a comparison chart of the actual and predicted values ​​of SO2 concentration emissions from the outlet of the secondary absorber based on MRGRUGRU.

[0066] Figure 10C A comparison chart of the actual and predicted SO2 concentration emissions from the outlet of the secondary absorption tower based on DS-MRGRU.

[0067] Figure 10D A comparison chart of the actual and predicted SO2 concentration emissions from the outlet flue gas of the secondary absorber based on TA-DS-MRGRU.

[0068] Figure 10E A comparison chart of the actual and predicted SO2 concentration emissions from the outlet flue gas of the secondary absorber based on FA-DS-MRGRU.

[0069] Figure 10F This is a comparison chart of the actual and predicted SO2 concentration emissions from the outlet of the secondary absorber based on FTA-DS-MRGRU.

[0070] Figure 11A This is a graph showing the error distribution of SO2 concentration prediction in the net flue gas at the outlet of a secondary absorber based on GRU.

[0071] Figure 11B This is a graph showing the error distribution of SO2 concentration prediction in the net flue gas at the outlet of a secondary absorber based on MRGRU.

[0072] Figure 11C This is a graph showing the error distribution of SO2 concentration emission prediction in the net flue gas at the outlet of the secondary absorption tower based on DS-MRGRU.

[0073] Figure 11D This is a distribution diagram of the prediction error of SO2 concentration in the net flue gas at the outlet of the secondary absorption tower based on TA-DS-MRGRU.

[0074] Figure 11E This is a graph showing the error distribution of SO2 concentration prediction in the net flue gas at the outlet of the secondary absorption tower based on FA-DS-MRGRU.

[0075] Figure 11F This is a graph showing the error distribution of SO2 concentration emission prediction in the net flue gas at the outlet of the secondary absorber based on FTA-DS-MRGRU. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0077] Example 1:

[0078] This invention provides a method for predicting SO2 concentration based on a GRU regulated by two-stage attention and dual-stream memory, such as... Figure 1 As shown, it includes:

[0079] S100. After obtaining historical data of process variables and target variables in the flue gas desulfurization process of thermal power plants, preprocessing is performed to obtain several time series of historical data, and these time series are divided into training set and test set; among them, the target variable is the net SO2 concentration of flue gas at the outlet of the secondary absorption tower, and the process variables are other parameters in the flue gas desulfurization process of thermal power plants.

[0080] Specifically, the SO2 concentration prediction method based on dual-stage attention and dual-stream memory regulation GRU provided in this embodiment of the invention selects important process variables that may affect the prediction of SO2 concentration in the net flue gas at the outlet of the secondary absorber as input variables, i.e., process variables, from the industrial decentralized control system of flue gas desulfurization in thermal power plants through mechanism analysis and expert experience. The SO2 concentration in the net flue gas at the outlet of the secondary absorber is used as the output variable, i.e., target variable. The method continuously and uniformly samples the SO2 concentration at a preset time interval, and then performs missing value processing, outlier processing based on the Laida criterion (3σ criterion), and data standardization processing based on the normalization method (z-score method) in sequence to obtain several time series of historical data.

[0081] S200. Construct a DS-MRGRU (Dual-Stream MRGRU) network for extracting the temporal and causal relationships of current and historical information, including a parallel temporal correlation stream information extraction structure and a dynamic causal correlation stream information extraction structure;

[0082] This invention introduces time gates and causal gates into the hidden states of a traditional GRU network to optimize the network structure, constructing a Memory Regulation Gated Recurrent Unit (MRGRU) to collaboratively capture key feature information from historical information and the current input using the original gating strategy. Furthermore, based on the MRGRU, Time-Aware Gated Recurrent Unit (TGRU) and Causal-Aware Gated Recurrent Unit (CGRU) with only time gates are differentiated. A dual-stream information extraction network, namely the DS-MRGRU network, is then constructed, comprising a time-dependent stream information extraction structure and a dynamic causal-dependent stream information extraction structure.

[0083] S300. A feature attention module is introduced in the feature extraction stage of the DS-MRGRU network to capture key features and suppress irrelevant features and noise. A temporal attention module is introduced in the prediction output stage of the DS-MRGRU network to capture long-term sequence dependencies, thus constructing an FTA-DS-MRGRU soft measurement model with enhanced key information extraction capabilities.

[0084] Specifically, this embodiment of the invention employs a two-stage attention mechanism to extract key information. The feature attention module introduced during the model feature extraction stage is primarily used to capture key features and suppress the influence of irrelevant features and noise. The temporal attention module introduced during the prediction output stage is mainly used to adaptively focus on key time point information in historical data, capturing long-term sequence dependencies. Attention weights provide an intuitive quantitative explanation of the contribution of each input data point to the model's prediction results, which helps in understanding the model's decision-making process and enhances the model's transparency and interpretability.

[0085] A two-stage attention mechanism is embedded in the DS-MRGRU network to capture complex time-series features and build a soft measurement model with stronger predictive performance. First, X = {x} t-T+1 ,x t-T+2 ,…,x t The input data is used as the feature attention module. Here, the input data is a time series with a time step of T, x. t This represents the historical data of n process variables at time t in the time series. The feature attention module assigns different weights α = {α1, α2, ..., α} to the n process variables through a weighting mechanism. n}, obtain the data after processing by the feature attention module. Next for The time-related features and dynamic causal features are learned independently from two information streams, generating T hidden state information with different representations. and Then, through the temporal attention module respectively... and The weights for hidden states are assigned as follows: Specifically, the weights for hidden states in the temporal flow and causal flow are assigned as follows: and And generate two combined vectors c by summation. t1 and c t2 Then merge and integrate them into the final vector c. t Finally, the network continuously optimizes its parameters through backpropagation, and based on c... t Get the output variable Final vector c t The expression for is shown in the following formula (5):

[0086]

[0087] S400: The FTA-DS-MRGRU soft measurement model is trained using the training set in S100 through the backpropagation algorithm, and the FTA-DS-MRGRU soft measurement model is tested using the test set in S100 to obtain the target FTA-DS-MRGRU soft measurement model.

[0088] S500: Obtain real-time data of process variables in the flue gas desulfurization process of thermal power plants, input the target FTA-DS-MRGRU soft measurement model, and obtain the SO2 content of the clean flue gas after the flue gas desulfurization process of thermal power plants.

[0089] In summary, this invention designs two auxiliary memory adjustment gates—a time gate and a causal gate—based on the existing gating strategy of GRU, respectively, for fine-tuning of past and current information. Furthermore, it constructs a dual-stream information extraction structure with temporal and causal correlations to generate complementary information streams, significantly enhancing its generalization ability and predictive performance in different industrial scenarios. Simultaneously, considering that information redundancy and temporal information decay weaken the model's ability to capture key information and long-term dependencies in nonlinear dynamic industrial process modeling, a two-stage attention mechanism is embedded in the DS-MRGRU network. This leads to a proposed method for predicting SO2 concentration in net flue gas based on a two-stage attention and dual-stream memory-adjusted GRU. This method adaptively extracts key feature variables using a feature attention mechanism in the feature extraction stage, suppressing the influence of redundant information and improving the model's interpretability. In the prediction output stage, a temporal attention mechanism is used to mine information from historically important moments, improving the model's prediction performance over long time series. This achieves accurate prediction of the SO2 content in the net flue gas after the flue gas desulfurization process in thermal power plants—a key quality variable.

[0090] Example 2:

[0091] S100 further includes:

[0092] S110. After obtaining the historical data of process variables and target variables in the flue gas desulfurization process of thermal power plants, perform missing value processing, outlier processing and data standardization in sequence.

[0093] Specifically, the SO2 concentration prediction method based on a two-stage attention and dual-stream memory-regulated GRU provided in this embodiment of the invention, through mechanism analysis and expert experience, selects important process variables that may affect the prediction of SO2 concentration in the net flue gas at the outlet of the secondary absorber from the industrial decentralized control system of flue gas desulfurization in thermal power plants as input variables, i.e., process variables, and the net flue gas SO2 concentration at the outlet of the secondary absorber as output variables, i.e., target variables. These variables are continuously and uniformly sampled at 5-minute intervals to obtain the input-output variable dataset, denoted as (X). all ,Y all After that, the collected (X) all ,Y all The missing value handling, outlier handling, and data standardization are performed sequentially. This includes: for variables containing only partial time points, if there is a large amount of missing data that cannot be supplemented, these variables are deleted; variables where all data in the sample are constant are also deleted. For variables with partially null values, the null value is replaced by the average of the two preceding and following data. Secondly, based on process requirements and operational experience, the operational range of the original data variables is summarized. Then, the maximum and minimum amplitude limiting methods are used to remove some samples outside this range, and outliers are removed according to the 3σ criterion. The 3σ criterion assumes that a set of test data contains only random errors, calculates the standard deviation, determines an interval with a certain probability, and considers any error exceeding this interval as gross error, not random error, and data containing such errors should be removed. First, the measured variables are measured with equal precision to independently obtain x1, x2, ..., x... N Find its arithmetic mean. and residual error (i = 1, 2, ..., N), and calculate the standard deviation σ, if a certain measured value x i The residual error v t (1≤t≤N), if the following formula (6) is satisfied:

[0094]

[0095] This error is considered a gross error, and the data x containing this error is considered gross. t It should be eliminated;

[0096] Finally, the input and output variables are standardized using the z-score method.

[0097] S120. Using the sliding window technique, the standardized historical data is transformed into several time series of length T, and the first 80% of the time series is used as the training set, and the remaining 20% ​​of the time series is used as the test set.

[0098] S200. Construct a DS-MRGRU network for extracting the temporal and causal relationships of current and historical information, including a parallel temporal correlation flow information extraction structure and a dynamic causal correlation flow information extraction structure;

[0099] Specifically, a time gate T is introduced into the hidden state of a traditional GRU network. t And the causal gate C t Optimize the network structure and retain z t and 1-z t As mutually exclusive gates, such as Figure 3 As shown, MRGRU is constructed. The time gate filters key information in the hidden state of the previous time step, thereby realizing the memory regulation of historical information and effectively capturing time-related information features. The causal gate is used to select important information in the current candidate state, regulate the influence of the current information on the hidden state, and thus capture the potential dynamic causal relationship between information, significantly enhancing the flexibility of the gating strategy. Based on this, MRGRU effectively changes the linear constraint relationship while retaining the mutual exclusion relationship. It can cooperate with the original gating strategy to capture key feature information in historical information and current input, enrich the transmitted information, finely adjust the information transmission, improve the information processing capability of the model, and more effectively adapt to the dynamic changes in the industrial environment. The state update equation of MRGRU is shown in the following formulas (7) to (9):

[0100] T t =σ(W T x t +U T h t-1 +b T (7)

[0101] c t =σ(W C x t +U C h t-1 +b C (8)

[0102]

[0103] Among them, T t Represents a time gate, C t W represents a causal gate. T W represents the input weight vector of the time-gated network. c U represents the input weight vector of a causal gate network.T U represents the time gate implicit weight vector. c b represents the implicit weight vector of the causal gate. T b represents the time gate bias vector. C This represents the causal gate bias vector.

[0104] Based on MRGRU, a time-gated TGRU layer and a causal-gated CGRU layer are derived. A time-dependent flow information extraction structure is constructed based on the time-gated TGRU layer, and a dynamic causal-dependent flow information extraction structure is constructed based on the causal-gated CGRU layer, resulting in the dual-stream information extraction network DS-MRGRU. The time-dependent flow information extraction structure uses a time gate T... t It regulates the memory and forgetting of historical information, thereby focusing on extracting temporal relationships in time series; the dynamic causal correlation flow information extraction structure uses causal gates C t Controlling the impact of current input information on prediction results aims to capture the dynamic causal relationship between current input and prediction results, ensuring that the model can accurately respond to changes in the current environment.

[0105] Specifically, such as Figure 6 As shown, the time-related stream information extraction structure includes a first time-gate-based TGRU layer, a Dropout layer, and a second time-gate-based TGRU layer connected in sequence.

[0106] The expressions for the first time-gate-based TGRU layer and the second time-gate-based TGRU layer are shown in Equations (10) and (11):

[0107]

[0108] in, This represents the input of the TGRU layer at time t. This represents the time-dependent hidden state output at time t-1 in the input time series of the TGRU layer, where σ(·) represents the sigmoid activation function. The output of the candidate time-related hidden state at time t in the input time series of the TGRU layer is represented by ⊙, which represents the Hadamard product operation. T U represents the input weight vector of the TGRU layer network. T b represents the hidden weight vector of the TGRU layer. T This represents the bias vector of the TGRU layer. This indicates the update gate corresponding to the TGRU layer. This represents the input weight vector of the update gate corresponding to the TGRU layer. This represents the hidden weight vector of the update gate corresponding to the TGRU layer. This represents the bias vector of the update gate corresponding to the TGRU layer.

[0109] like Figure 6 As shown, the dynamic causal correlation flow information extraction structure includes a first causal gate-based CGRU layer, a Dropout layer, and a second causal gate-based CGRU layer connected in sequence.

[0110] The expressions for the first causal gate-based CGRU layer and the second causal gate-based CGRU layer are shown in equations (12) and (13):

[0111]

[0112] in, This represents the input to the CGRU layer at time t. This represents the causal correlation of the hidden state output at time t-1 in the input time series of the TGRU layer, where σ(·) represents the activation function, the sigmoid function. W represents the output of the candidate causally related hidden states at time t in the input time series of the CGRU layer. ⊙ represents the Hadamard product operation. c U represents the input weight vector of the CGRU layer network. c b represents the hidden weight vector of the CGRU layer. C This represents the bias vector of the CGRU layer. This indicates the update gate corresponding to the CGRU layer. This represents the input weight vector of the update gate corresponding to the CGRU layer. This represents the hidden weight vector of the update gate corresponding to the CGRU layer. This represents the bias vector of the update gate corresponding to the CGRU layer.

[0113] S300. A feature attention module is introduced in the feature extraction stage of the DS-MRGRU network to capture key features and suppress irrelevant features and noise. A temporal attention module is introduced in the prediction output stage of the DS-MRGRU network to capture long-term sequence dependencies, thus constructing an FTA-DS-MRGRU soft measurement model with enhanced key information extraction capabilities.

[0114] like Figure 6 As shown, the feature attention module aims to dynamically assign attention weights to input features before the input data enters the dual information stream, thereby uncovering features that have a significant impact on the model's prediction results. This avoids the problem of losing feature association information in traditional correlation analysis methods, accelerates the training convergence process, and effectively improves the model's learning efficiency.

[0115] As the length of the input sequence increases, some key information may be lost. Therefore, a temporal attention mechanism is introduced to directly focus on long-distance dependencies without needing to pass information through multiple time steps, effectively mitigating the problem of key information loss as the input sequence length increases. The FTA-DS-MRGRU soft sensor model uses a temporal attention mechanism after the dual information flow structure to capture temporal dependencies, adaptively assigning weights to the received hidden state information, evaluating the importance of different historical time points to the time to be predicted, and thus selecting information from key time points.

[0116] The S300 further includes:

[0117] S310. After assigning feature attention weights to the input time series through the feature attention module, the time series is weighted according to the feature attention weights, and the weighted time series is output.

[0118] Specifically, such as Figure 4 As shown, the expressions for the feature attention module are given by formulas (14) to (17):

[0119] q t1 =W q 1 x t +b q 1 (14)

[0120] e t i =V1 i σ(W1 i q t1 +U1 i x t i +b1 i ), 1≤i≤n (15)

[0121]

[0122] Where, x t Let x represent the input of the feature attention module at time t. t i q represents the i-th variable at time t in the input time series. t1 e represents the query vector of the feature attention module. t i α represents the feature attention score corresponding to the i-th variable at time t in the input time series. t i This represents the feature attention weight corresponding to the i-th variable at time t in the input time series. This represents the weighted input vector after passing through the feature attention module, where n represents the number of input features, and W...q 1 b q 1 W1 i U1 i b1 i V1 i Both indicate that the feature attention parameters are continuously updated and optimized during the model training process.

[0123] S320. Extract the temporal relationship in the weighted time series through the time gate in the time-related flow information extraction structure to obtain the time-related hidden state output, and extract the causal relationship in the weighted time series through the causal gate in the dynamic causal flow information extraction structure to obtain the causal hidden state output.

[0124] S330. After assigning first temporal attention weights to the time-related hidden state outputs through the first temporal attention module and then weighting them, the comprehensive temporal features of the historical key points of attention are obtained. After assigning second temporal attention weights to the causal related hidden state outputs through the second temporal attention module and then weighting them, the comprehensive causal features of the historical key points of attention are obtained.

[0125] Specifically, such as Figure 5 As shown, the expressions for the first and second temporal attention modules are as shown in formulas (18) to (21) below:

[0126] q t2 =W q 2 h t +b q 2 (18)

[0127] g j =V2 j σ(W2 j q t2 +U2 j h t-T+j +b2 j ), 1≤j≤T (19)

[0128]

[0129] Among them, h t q represents the input of the temporal attention module at time t. t2 G represents the query vector of the temporal attention module. j β represents the temporal attention score corresponding to the hidden state at the j-th time step in the input time series. j c represents the temporal attention weight corresponding to the hidden state at the j-th time step in the input time series. tThe vector represents the weighted sum of the outputs of each hidden state, where T represents the size of the chosen time step, and W represents the value of the hidden state. q 2 b q 2 W2 j U2 j b2 j V2 j Both refer to the temporal attention parameters that are updated and optimized during model training.

[0130] S340. After fusing the integrated temporal features and integrated causal features through the merging layer, the final output result is obtained through the fully connected layer.

[0131] This invention employs a two-stage attention mechanism to extract key information: a feature attention module is introduced during the model feature extraction stage to capture key features and suppress the influence of irrelevant features and noise; a temporal attention module is introduced during the prediction output stage to adaptively focus on key time point information in historical data and capture long-term sequence dependencies. The attention weights provide an intuitive quantitative explanation of the contribution of each input data point to the model's prediction results, which helps in understanding the model's decision-making process and enhances the model's transparency and interpretability.

[0132] S400: The FTA-DS-MRGRU soft measurement model is trained using the training set in S100 through the backpropagation algorithm, and the FTA-DS-MRGRU soft measurement model is tested using the test set in S100 to obtain the target FTA-DS-MRGRU soft measurement model.

[0133] The S400 specifically includes:

[0134] S410, Forward computation: Calculate the weights of each gating unit and the two-stage attention mechanism, and the calculation formulas are shown in formulas (10) to (21).

[0135] S420, Backward Calculation: Calculate the loss function value. The model loss function is the half-mean squared error (MSE), and the calculation method is shown in formula (22):

[0136]

[0137] Where N represents the total number of samples in the test set, y t This indicates the true value of SO2 concentration in the clean flue gas. This represents the predicted SO2 concentration in the clean flue gas;

[0138] S430, Gradient Update: Based on the corresponding error terms, the Adam optimization algorithm is used to update the network weights. The Adam optimization algorithm is a first-order optimization algorithm that can replace the traditional stochastic gradient descent algorithm. It has higher computational efficiency and better convergence performance in the same training cycle, while requiring less computational space.

[0139] S500: Obtain real-time data of process variables in the flue gas desulfurization process of thermal power plants, input the target FTA-DS-MRGRU soft measurement model, and obtain the SO2 content of the clean flue gas after the flue gas desulfurization process of thermal power plants.

[0140] In summary, this invention designs two auxiliary memory adjustment gates—a time gate and a causal gate—based on the existing gating strategy of GRU, respectively, for fine-tuning of past and current information. Furthermore, it constructs a dual-stream information extraction structure with temporal and causal correlations to generate complementary information streams, significantly enhancing its generalization ability and predictive performance in different industrial scenarios. Simultaneously, considering that information redundancy and temporal information decay weaken the model's ability to capture key information and long-term dependencies in nonlinear dynamic industrial process modeling, a two-stage attention mechanism is embedded in the DS-MRGRU network. This leads to a proposed method for predicting SO2 concentration in net flue gas based on a two-stage attention and dual-stream memory-adjusted GRU. This method adaptively extracts key feature variables using a feature attention mechanism in the feature extraction stage, suppressing the influence of redundant information and improving the model's interpretability. In the prediction output stage, a temporal attention mechanism is used to mine information from historically important moments, improving the model's prediction performance over long time series. This achieves accurate prediction of the SO2 content in the net flue gas after the flue gas desulfurization process in thermal power plants—a key quality variable.

[0141] To verify the effectiveness and superiority of the SO2 concentration prediction method based on dual-stage attention and dual-stream memory regulation GRU provided in this embodiment of the invention, a specific experiment is conducted. The experimental data comes from the desulfurization process data acquisition system of a thermal power plant. The purpose is to perform soft measurement modeling of the SO2 concentration in the clean flue gas at the outlet of the secondary absorption tower in this process.

[0142] The flowchart of the flue gas desulfurization process in the thermal power plant is as follows: Figure 7 As shown in Table 1, after studying the desulfurization process of the thermal power plant and performing data analysis and preprocessing, a candidate input variable set consisting of 18 auxiliary variables was finally determined.

[0143] Table 1 Candidate Input Variables for Flue Gas Desulfurization Process in Coal-fired Power Plants

[0144]

[0145] The FTA-DS-MRGRU soft sensor model dynamically adjusts attention weights using a two-stage attention mechanism, enabling the assessment of the importance of spatial and temporal dimensions. By visualizing the distribution of attention weights within the model, the specific contributions of each feature and time step to the model's predictive performance are quantitatively analyzed.

[0146] The attention weight of process variables accounts for, for example Figure 8 As shown, this quantifies the importance of each input variable in predicting the SO2 concentration in the net flue gas at the outlet of the secondary absorber. Variables 2, 3, 11, and 13 are particularly prominent, each with an attention weight exceeding 10%, corresponding to the SO2 concentration in the flue gas at the outlet of the primary absorber, the pH value of the gypsum slurry in the primary absorber, the net flue gas pressure at the furnace chimney inlet, and the median desulfurization efficiency of the unit, respectively. These variables have a significant impact on the desulfurization process: the SO2 concentration in the flue gas at the absorber outlet directly affects the desulfurization effect; the pH value of the gypsum slurry in the absorber relates to the chemical reaction environment of the gypsum slurry and the desulfurization efficiency; the net flue gas pressure at the furnace chimney inlet reflects the flue gas flow state and the workload of the desulfurization system; and the median desulfurization efficiency of the unit directly reflects the real-time effect of the desulfurization process. Therefore, these four variables have a crucial impact on the prediction results of SO2 concentration and are important considerations for optimizing the desulfurization process and improving prediction accuracy.

[0147] like Figure 9 As shown, the temporal attention weighting mechanism effectively quantifies the importance of hidden states at each historical time point in predicting SO2 concentration. The horizontal axis represents the current time point (0) and the time point furthest from the current time point (6). It is clearly observed from the figure that hidden states closer to the current time point receive higher attention weights, indicating that they carry more key information and have a greater impact on the prediction results. This result not only verifies the effectiveness of the temporal attention weighting mechanism in capturing key information in time series data but also demonstrates that sampling data from recent times during power plant desulfurization processes more accurately reflects the current dynamic changes and operating status of the system.

[0148] To further verify the superiority of the proposed model, corresponding ablation experiments were conducted. Figures 10A to 10F , Figures 11A to 11F Tables 1 and 2 present the prediction curves, error frequency distributions, and performance index data for each model for SO2 concentration. Specifically, mean squared error (MSE), mean absolute error (MAE), and correlation index (R²) are used. 2 As a performance evaluation index for the prediction model, the mean squared error is specifically shown in formula (22), and the mean absolute error (MAE) and correlation index (R) are also included. 2 The calculation formulas are shown in formulas (23) to (24) respectively:

[0149]

[0150] Where N represents the total number of samples in the test set, y t This represents the true value of SO2 concentration in the net flue gas. This represents the predicted value of SO2 concentration in net flue gas. Indicates y t The average value.

[0151] Table 2. SO2 concentration prediction results using different prediction methods

[0152]

[0153] in, Figures 11A to 11B σ represents the standard deviation, reflecting the dispersion of the prediction error; a larger value indicates higher model stability and reliability. Compared to the traditional GRU network, MRGRU achieves precise control over information transmission by introducing time gates and causal gates, effectively reducing interference from redundant information and thus improving the model's prediction performance. Building upon this, DS-MRGRU significantly enhances the model's feature extraction capability by constructing a dual information flow structure: parallel temporal correlation flow and dynamic causal correlation flow. FTA-DS-MRGRU further introduces a two-stage attention mechanism, where feature attention assigns higher weights to important auxiliary variables, while temporal attention focuses on extracting hidden state information from historically important time points, significantly improving the model's prediction accuracy and robustness. Experimental results fully demonstrate the performance advantages of this method.

[0154] The effectiveness and superiority of the SO2 concentration prediction method based on dual-stage attention and dual-stream memory regulation GRU provided in this invention, compared with other methods, were verified by soft measurement of SO2 concentration in the flue gas emitted during the flue gas desulfurization process of a certain power plant. The method can effectively improve the prediction accuracy of the model.

[0155] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0156] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A SO2 concentration prediction method based on two-stage attention and two-flow memory regulation GRU, characterized in that, The method comprises the following steps: S100, obtaining historical data of process variables and target variables in a flue gas desulfurization process of a thermal power plant, preprocessing the historical data to obtain a plurality of time series of the historical data, and dividing the plurality of time series into a training set and a test set; wherein the target variable is the SO2 concentration of the clean flue gas at the outlet of the secondary absorption tower, and the process variable is other parameters in the flue gas desulfurization process of the thermal power plant; S200, constructing a DS-MRGRU network for extracting the time relationship and causal relationship between the current information and the historical information, comprising a parallel time correlation flow information extraction structure and a dynamic causal correlation flow information extraction structure, the time correlation flow information extraction structure comprising a first time gate-based TGRU layer, a Dropout layer and a second time gate-based TGRU layer connected in sequence, and the dynamic causal correlation flow information extraction structure comprising a first causal gate-based CGRU layer, a Dropout layer and a second causal gate-based CGRU layer connected in sequence; S300, introducing a feature attention module in the feature extraction stage of the DS-MRGRU network to capture key features and suppress irrelevant features and noise, and introducing a time sequence attention module in the prediction output stage of the DS-MRGRU network to capture long time sequence dependency, to construct an FTA-DS-MRGRU soft measurement model with enhanced key information extraction capability; S400, training the FTA-DS-MRGRU soft measurement model by using the training set in S100 through a back propagation algorithm, and testing the FTA-DS-MRGRU soft measurement model by using the test set in S100, to obtain a target FTA-DS-MRGRU soft measurement model; S500, obtaining real-time data of process variables in the flue gas desulfurization process of the thermal power plant, inputting the target FTA-DS-MRGRU soft measurement model, and obtaining the SO2 content of the clean flue gas after the flue gas desulfurization process of the thermal power plant; The S300 comprises: S310, assigning feature attention weights to the input time series through the feature attention module, weighting the time series according to the feature attention weights, and outputting the weighted time series; S320, extracting the time sequence relationship in the weighted time series through the time gate in the time correlation flow information extraction structure to obtain a time correlation hidden state output, and extracting the causal relationship of the weighted time series through the causal gate in the dynamic causal correlation flow information extraction structure to obtain a causal correlation hidden state output; S330, weighting the time correlation hidden state output by assigning first time sequence attention weights to the time correlation hidden state output through the first time sequence attention module, to obtain a comprehensive time feature of the historical key points of interest, and weighting the causal correlation hidden state output by assigning second time sequence attention weights to the causal correlation hidden state output through the second time sequence attention module, to obtain a comprehensive causal feature of the historical key points of interest; S340, fusing the comprehensive time feature and the comprehensive causal feature through a merging layer and outputting through a fully connected layer to obtain a final output result.

2. The method of claim 1, wherein, expressions of the first time gate-based TGRU layer and the second time gate-based TGRU layer are: wherein, denotes a TGRU layer denotes the time-dependent hidden state output at time denotes a TGRU layer denotes the time-dependent hidden state output at time denotes an activation function sigmoid function, denotes a TGRU layer denotes the candidate time-dependent hidden state output at time denotes a Hadamard product operation, denotes a TGRU layer network input weight vector, denotes a TGRU layer hidden weight vector, denotes a TGRU layer bias vector, denotes a TGRU layer corresponding update gate, denotes a TGRU layer corresponding update gate input weight vector, denotes a TGRU layer corresponding update gate hidden weight vector, denotes a TGRU layer corresponding update gate bias vector.

3. The method of claim 1, wherein, expressions of the first causal gate-based CGRU layer and the second causal gate-based CGRU layer are: wherein, denotes a CGRU layer at time t, denotes a causal recurrent hidden state output at time t, denotes an activation function sigmoid function, denotes a candidate causal recurrent hidden state output at time t, denotes a Hadamard product operation, denotes a CGRU layer network input weight vector, denotes a CGRU layer hidden weight vector, denotes a CGRU layer bias vector, denotes a CGRU layer corresponding update gate, denotes a CGRU layer corresponding update gate input weight vector, denotes a CGRU layer corresponding update gate hidden weight vector, denotes a CGRU layer corresponding update gate bias vector.

4. The method of claim 3, wherein, an expression of the feature attention module is: wherein, represents the feature attention module, at time t, represents the input time series, at time t, th variable, represents the feature attention module query vector, represents the input time series, at time t, th variable, represents the feature attention module query vector, at time t, th variable, represents the input vector after weighting by the feature attention module, represents the number of input features, , , , , , all represent the feature attention parameters that are constantly updated and optimized during the model training process.

5. The method of claim 4, wherein, expressions of the first time sequence attention module and the second attention module are: wherein, represents the time sequence attention module, the time sequence attention module, represents the time sequence attention module query vector, represents the time sequence attention score corresponding to the hidden state of the time step in the input time sequence, represents the time sequence attention weight corresponding to the hidden state of the time step in the input time sequence, represents the vector after weighted summation of each hidden state output, represents the size of the time step taken, , , , , , all represent the time sequence attention parameters updated and optimized in the model training process.

6. The method of claim 1, wherein, The S100 comprises: S110, after acquiring the historical data of the process variables and target variables in the flue gas desulfurization process of the thermal power plant, sequentially performing missing value processing, abnormal value processing and data standardization; S120, the historical data after the data is standardized is converted into a plurality of time series with a length of T and the plurality of time series are divided into a training set and a test set.

7. The method of claim 1, wherein, The FTA-DS-MRGRU soft measurement model is trained by using the training set in the S100 through the back propagation algorithm in the S400, comprising: The network weight of the FTA-DS-MRGRU soft measurement model is calculated by using the training set in the S100 to perform forward calculation on the FTA-DS-MRGRU soft measurement model; The loss function value of the FTA-DS-MRGRU soft measurement model is calculated by using the semi-mean square root error loss function to obtain the corresponding error term, and the calculation formula is: wherein, represents the total number of test set samples, represents the true value of SO2 concentration in the flue gas, represents the predicted value of SO2 concentration in the flue gas; Based on the corresponding error term, the network weight is updated by using the Adam optimization algorithm.

8. A target detection method characterized by, The method detects the gas to be detected by using the method of any one of claims 1-7, and the gas to be detected is the gas generated in the industrial process.

9. A computer device, comprising: The computer device comprises a processor and a memory, the memory is used to store computer instructions, the computer instructions are loaded and executed by the processor to realize the SO2 concentration prediction method based on the double-stage attention and double-flow memory adjustment GRU of any one of claims 1-7.

Images