Dynamic soft sensing method based on cascaded attention feature fusion and gated TCN

Abstract

The present invention discloses a dynamic soft measurement method based on cascaded attention feature fusion and gated TCN, and belongs to the technical field of industrial process control. By constructing a cascaded attention feature fusion module (CAFF), it is used to improve the residual connection part of the TCN network. The module includes two self-attention modules SAM and a multi-scale channel attention module MS-CAM. The CAFF module strengthens important information from the time and variable dimensions, suppresses the propagation of irrelevant information in the network, alleviates the loss of local dependencies between variables, and enables the network to adaptively focus on important information. At the same time, the application also proposes a gated TCN network structure, which uses gated dilated causal convolution and SELU activation function to enhance the memory and learning ability of TCN, eliminate the negative impact that unstable features may bring during the feature extraction process, and thus improve the prediction accuracy of quality variables in complex industrial processes.

Description

Technical Field

[0001] The invention relates to a dynamic soft measurement method based on cascaded attention feature fusion and gated TCN, belonging to the technical field of industrial process control. Background Art

[0002] As the scale of industry expands, production processes are becoming more complex and diversified, placing higher demands on product quality and production efficiency. The key to achieving the above goals is to measure key quality variables in complex industrial processes in real time. These variables usually refer to indicators that are closely related to product quality, such as sulfur dioxide (SO2) and hydrogen sulfide (H2S) concentrations in the tail gas of the sulfur recovery process. However, due to the limitations of measurement technology or high costs, most quality variables are obtained by offline laboratory testing, and real-time monitoring through hardware sensors becomes extremely difficult.

[0003] In order to obtain key quality indicator information online in real time, soft measurement technology was born. Soft measurement achieves online prediction and estimation of quality variables by establishing a mathematical model between quality variables that are difficult to measure in real time and process variables that are easy to measure through hardware sensors. Soft measurement modeling methods are mainly divided into white box modeling (i.e. based on process mechanism) and black box modeling (i.e. based on data drive). With the increasing scale and complexity of modern industrial processes, soft measurement models based on process mechanism are becoming increasingly difficult to build and solve, and it is also difficult to accurately and efficiently complete the online prediction of quality indicators. With the development of machine learning and deep learning, the data-driven black box modeling method does not need to consider complex process mechanisms and is currently the mainstream method of soft measurement.

[0004] Due to complex physical and chemical reactions, industrial processes usually exhibit dynamic characteristics, which means that there is a high temporal correlation between process variables within a certain period of time. However, most current data-driven soft sensing methods mainly consider spatial nonlinear relationships, which leads to poor prediction performance in processes with strong dynamic characteristics. Long short-term memory neural network (LSTM), as a deep network based on time series data, is often used for dynamic soft sensing modeling. Nevertheless, LSTM only considers the input at the current moment and the hidden layer features at the previous moment, which may make it difficult to capture the long-term dependencies between variables at distant sampling moments. In contrast, CNN extracts data features through convolution operations, has the advantages of parallel computing and weight sharing, and is more suitable for learning dynamic nonlinear relationships between variables. Bai et al. proposed a temporal convolutional network (TCN) for sequence modeling, which consists of a stack of residual blocks. TCN uses an expansion convolution strategy to insert zero values ​​in the convolution kernel to skip samples at certain moments, which gives the network a large enough receptive field to capture information at more distant moments (see "Bai S, Kolter JZ, Koltun V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling [J]. arXiv preprint arXiv: 1803.01271, 2018."). Sun et al. used TCN to extract nonlinear and long-term dynamic dependencies of variables, and used the extracted dynamic features as input to establish a soft sensor model based on the attention simple recurrent unit (ASRU) (see "Sun C, Zhang Y, Zhao H, et al. A soft sensor model forcement specific surface area based on TCN-ASRU neural network [J]. IEEE Transactions on Instrumentation and Measurement, 2023.").Yuan et al. constructed an autoregressive time convolution (AR-TCN) prediction model, introduced quality variable related information into TCN, and used it to predict key variables in debutanizer and hydrocracking processes (see "Yuan X, Qi S, Wang Y, et al. Quality variable prediction for nonlinear dynamic industrial processes based on temporal convolutional networks[J]. IEEE Sensors Journal, 2021, 21(18): 20493-20503."). However, dilated convolution also causes the network to ignore the continuity of variables, and the short-term temporal correlation between variables may be lost due to dilated convolution, which in turn leads to reduced prediction accuracy of the soft sensor model, delayed response, and inaccurate simulation of process dynamic changes. Summary of the invention

[0005] In view of the shortcomings of the above-mentioned TCN-based soft measurement model, this application proposes a dynamic soft measurement method (CAFF-GTCN) based on cascaded attention feature fusion and gated TCN for the detection of SO2 concentration in the tail gas of the sulfur recovery process, aiming to fully explore the complex dynamic characteristics and nonlinearity of the sulfur recovery process and improve the prediction accuracy of the model. This method introduces a cascaded attention feature fusion module (CAFF) to improve the residual connection in TCN, alleviate the loss of local dependencies between variables, and enable the network to adaptively focus on important information. At the same time, a gated temporal convolutional network (GTCN) structure is proposed to enhance the screening and memory capabilities of information through gated convolution, thereby eliminating the negative impact that unstable features may bring during the feature extraction process.

[0006] A dynamic soft sensing method based on cascaded attention feature fusion and gated TCN, comprising:

[0007] Step 1: Obtain historical data of process variables and quality variables from complex industrial processes. Quality variables are process parameters that need to be predicted by the soft sensor model, and process variables are other process parameters measured by hardware sensors.

[0008] The historical data set is preprocessed, including normalization, outlier and missing value removal, etc., and finally the historical data set is used as the training set X of the model. train , Y train .

[0009] Step 2: Build a gated TCN network model with cascaded attention feature fusion. First, build a TCN network model, which is composed of multiple residual blocks stacked together. Each residual block contains two layers of dilated causal convolutional layers, each of which uses weight normalization, RELU activation function, and Dropout strategy. In addition, the input and extracted features are fused through residual connections, and finally the output features of the module are obtained, where 1×1Conv can ensure that the dimensions of the input and output are consistent.

[0010] Step 3: After building the TCN network, a cascaded attention fusion module (CAFF) is designed to improve the residual connection of TCN. The CAFF module contains two self-attention modules SAM and a multi-scale channel attention module MS-CAM. The two attention mechanisms pay attention to the important information in the features from the perspective of time and variables, respectively, to further extract dynamic nonlinear features.

[0011] Step 4: Propose a gated TCN network structure, replace the dilated causal convolutional layer in TCN with a gated dilated causal convolutional layer, and use the gating signal to enhance the network's ability to filter and remember information. At the same time, replace the ReLU activation function in TCN with the SELU function, which can effectively suppress the gradient vanishing problem during network training. Finally, a gated TCN (CAFF-GTCN) soft measurement model with cascaded attention feature fusion is proposed.

[0012] Step 5: Place the X train , Y train Train the CAFF-GTCN network model and optimize the network's hyperparameters, and save the trained model parameters for online prediction;

[0013] Step 6: Acquire real-time data of process variables in the process, input the trained CAFF-GTCN prediction model, and obtain dynamic prediction values ​​of quality variables.

[0014] Optionally, the initial TCN network constructed in step 2 mainly adopts the dilated causal convolution and residual connection strategy. In most time series tasks, causality cannot be ignored, which means that the model can only rely on information before the current moment to make predictions. The causal convolution strategy is applied to TCN. For a time series x = (x1, x2, ..., x T ), then the causal convolution output value F(T) at time T is

[0015]

[0016] where f=[f0,f1,...,f k-1 ] is a convolution kernel, k represents the size of the convolution kernel, x T-iIndicates that only available historical input information participates in the convolution operation. In addition, dilated convolution is applied to the network. The dilated convolution kernel can skip certain historical moment inputs according to a fixed step size, covering a longer range of historical information under the premise of the same parameters. The dilated causal convolution at time T is:

[0017]

[0018] in,* d represents the dilated convolution operation, and d is the dilation rate used to control the jump step size. As the network deepens, the dilation rate increases exponentially, and the dilation rate of the jth hidden layer is 2 j-1 Using a larger expansion rate d can make the top-level output represent a wider range of inputs, and extract the temporal characteristics of the original data through multi-layer expansion causal convolution. As the number of network layers increases, the extracted feature information will also increase, which puts higher requirements on model performance and stability. TCN introduces residual connections, which convert the input x of the residual block into T It is superimposed and fused with the output F(T) after several stacked layers to achieve cross-level information transmission. Its mathematical expression is:

[0019] y T =F(T)+x T (3)

[0020] where y T is the final output of the current residual module, which will be passed to the next residual block as input. Residual connections make information transfer and gradient propagation between layers in the network smoother.

[0021] Optionally, in the step 3, a cascaded attention feature fusion module (CAFF) is proposed, SAM focuses on important information in the features from the perspective of time, and MS-CAM assigns corresponding attention weights according to the importance of different features from the perspective of variables, ultimately promoting better fusion of two different features X and Y.

[0022] In the initial fusion stage of features, SAM is used to assign weights to samples at different positions in the sequence. For a given input feature matrix w represents the length of the sequence, and d represents the dimension of the feature. First, a linear transformation is performed to obtain the query matrix Q, the key matrix K, and the value matrix V

[0023]

[0024] in, as well as They are three trainable parameter matrices, and is the matrix of corresponding vector queries, keys and values. Then the self-attention weight matrix M between queries and keys is calculated by scaling the dot multiplication attention. S

[0025]

[0026] The attention weight matrix M S Multiply it with V to get the SAM weighted output F S '

[0027] F S '=M S V (6)

[0028] In the CAFF module, the two features X and Y are weighted by SAM respectively, and the historical information at different times in each feature is weighted. After obtaining the weighted features X' and Y', the two features are added together to obtain the intermediate feature H

[0029]

[0030] Among them, S,X and M S,Y is the corresponding attention weight, V X 、V Y is the corresponding value matrix. SAM determines the attention weights of different positions in the sequence from a temporal perspective. In order to obtain an accurate weight matrix, the importance of different features needs to be taken into account. The intermediate feature H obtained by SAM is transposed to obtain the input feature of MS-CAM In the MS-CAM module, each dimension of the intermediate feature H is processed as a channel, d represents the number of channels of the input feature, and w×1 represents the size of the feature map. C ) and local features L(F C ), and use the Sigmoid function to calculate the channel attention weight matrix M C :

[0031]

[0032] G(F C )=Conv2(ReLU(Conv1(g(F C )))) (9)

[0033] L(F C )=Conv2(ReLU(Conv1(F C ))) (10)

[0034]

[0035] in It is a global average pooling operation. Conv1 and Conv2 have convolution kernels w / r and r 1×1 convolution, r is the reduction rate of the channel; Represents broadcast addition. MS-CAM can adjust the importance of each channel, that is, weighted aggregation of feature information of different channels in the feature map to enhance the response of important feature channels. The fusion of X and Y is realized to obtain the final fused feature Z of CAFF:

[0036] Z=M C T ⊙X+(1-M C T )⊙Y (12)

[0037] Among them, ⊙ represents the dot product operation. CAFF is used to improve the residual connection in TCN and improve the network's temporal feature extraction capability by fusing features obtained from different receptive fields.

[0038] Optionally, in the gated TCN in step 4, the gated dilated causal convolution strategy and SELU activation function can improve the model's ability to filter and memorize information. In order to implement gated dilated causal convolution, two dilated causal convolution layers with unshared convolution kernel parameter weights are selected, one for feature extraction and the other for obtaining the gating signal in combination with the Sigmoid function. The corresponding elements of the outputs of the two layers are multiplied to obtain the final output of the gated dilated causal convolution layer:

[0039]

[0040] h i =Sigmoid(h g )·(h f ),i=1,2,...,c (15)

[0041] Among them, h g As the output of the convolution kernel of the gated signal layer, h f is the convolution kernel extraction result of the feature extraction layer, h i It represents the final result after gated convolution. The convolution kernel slides in time order. The final output H obtained by gated convolution is g for

[0042]

[0043] Where c represents the number of convolution kernels and T is the length of the input history sequence. In addition, SELU is used to replace the ReLU activation function of the TCN network, introducing the self-normalization property into the network, so that the mean and variance between layers are consistent, which can speed up the convergence speed and suppress the gradient explosion problem. The expression of the SELU activation function is as follows:

[0044]

[0045] Where α and λ are two adjustable parameters of the SELU activation function.

[0046] The beneficial effects of the present invention are:

[0047] (1) A cascaded attention feature fusion module (CAFF) is designed to improve the residual connection part of the TCN network. The module contains SAM and MS-CAM. The SAM module can assign attention weights to different positions in the process variable time series, while the MS-CAM module is used to analyze the importance of different input variables or hidden layer features and assign different weights according to the relationship between the input variables. The CAFF module highlights important feature information, strengthens it from the time and variable dimensions, and suppresses the propagation of irrelevant information in the network. Therefore, it effectively alleviates the problem of feature loss in the TCN network, thereby better mining the unknown dynamic characteristics of the process.

[0048] (2) A gated TCN network structure is proposed, which uses gated dilated causal convolution and SELU activation function to enhance the memory and learning ability of TCN. The gated TCN structure can adaptively adjust the learning rate of data, which not only makes the model more stable during training, but also improves its performance in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 The present invention is a flow chart of the sulfur recovery process.

[0051] Figure 2 This is an example diagram of the dilated causal convolution with a dilation rate of d=1, 2, 4 and a convolution kernel size of 3 according to the present invention.

[0052] Figure 3 Structural diagram of the TCN model designed for this method.

[0053] Figure 4 This is a schematic diagram of the cascaded attention feature fusion module involved in the present invention.

[0054] Figure 5 This is the overall structure diagram of the CAFF-GTCN model constructed in the present invention.

[0055] Figure 6 This is a schematic diagram of the gated dilated causal convolution principle of the CAFF-GTCN model involved in the present invention.

[0056] Figure 7 The figure is a fitting curve diagram for predicting the SO2 concentration in the tail gas of the sulfur recovery industrial process using five existing benchmark models and the model provided by the present invention.

[0057] Figure 8 The absolute prediction error box plot is a diagram of the prediction of SO2 concentration in tail gas of a sulfur recovery industrial process using five existing models and the model provided by the present invention. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0059] Embodiment 1:

[0060] This embodiment provides a dynamic soft measurement method based on cascaded attention feature fusion and gated TCN, including:

[0061] Step S1, obtaining historical data of process variables and quality variables in a complex industrial process, where the quality variables are process parameters that need to be predicted by the soft sensor model, and the process variables are other process parameters measured by hardware sensors;

[0062] Step S2, constructing a gated TCN network model CAFF-GTCN with cascaded attention feature fusion, wherein the CAFF-GTCN includes a cascaded attention fusion module CAFF and a gated TCN network structure;

[0063] Step S2.1, constructing a TCN network model, wherein the TCN network model is stacked by multiple residual blocks; each residual block contains two layers of dilated causal convolutional layers, each layer adopts weight normalization, RELU activation function and Dropout strategy; at the same time, the input and extracted features are fused through residual connection, and finally the output features of the module are obtained;

[0064] Step S2.2, designing a cascaded attention fusion module CAFF, using the cascaded attention fusion module CAFF to improve the residual connection of the TCN network model constructed in step S2.1, wherein the cascaded attention fusion module CAFF includes two self-attention modules SAM and a multi-scale channel attention module MS-CAM, so as to pay attention to important information in the features from the perspectives of time and variables, respectively, and further extract dynamic nonlinear features;

[0065] Step S2.3, design the gated TCN network structure, replace the dilated causal convolution layer in TCN with the gated dilated causal convolution layer, and use the gating signal to enhance the network's ability to filter and remember information; at the same time, replace the ReLU activation function in TCN with the SELU function to suppress the gradient vanishing problem during network training; finally, the gated TCN network model CAFF-GTCN with cascaded attention feature fusion is obtained.

[0066] Step S3, using the historical data of the complex industrial process acquired in step S1 as training data to train the CAFF-GTCN model;

[0067] Step S4, obtaining real-time data of process variables in complex industrial processes, inputting the CAFF-GTCN prediction model trained in step S3, and obtaining dynamic prediction values ​​of quality variables.

[0068] Embodiment 2:

[0069] This embodiment provides a method for predicting SO2 concentration in tail gas of a sulfur recovery process based on cascaded attention feature fusion and gated TCN, including:

[0070] Step 1: Five process variables in the sulfur recovery process are selected as auxiliary variables. The relevant specific descriptions are shown in Table 1. The historical data are normalized to unify the dimensions of different variables. 8000 samples are selected from them to construct the training set X train and Y train , use the last 1500 samples to construct the test set X test and Y test .

[0071] The process flow of sulfur recovery unit is as follows: Figure 1As shown in the figure, the Sulfur Recovery Unit (SRU) is an important part of an industrial refinery. It can remove the acid gas flow before it is discharged into the atmosphere, and the sulfur element is recovered in the unit as a by-product. The SRU consists of multiple subunits, called sulfur lines. This example establishes a soft sensor model for a sulfur line in the SRU. The SRU takes two acid gases as input, one is called MEA gas, which contains a large amount of H2S gas; the other is called SWS gas, the main components of which are H2S and NH3. The input acid gas is sent to the combustion furnace (F101) for reaction. The combustion chamber is divided into two independent parts. The main combustion chamber is used to burn MEA gas. If there is insufficient air in the main combustion chamber, an appropriate amount of air will be introduced; the auxiliary combustion chamber is used to burn SWS gas. When there is insufficient air in the combustion chamber, an appropriate amount of air will be introduced. If the SWS gas flow is too low, MEA gas and air are added to keep the gas input flow in the auxiliary chamber constant. When the analyzer is offline for maintenance, the control loop cannot work and the performance of the SRU is degraded. The combustion products will be sent to the water condenser (E101), which evaporates the elemental sulfur (70%) and another part of the gas, which is sent to the catalytic reactor (R101) and then to the condenser (E102). The output of the condenser E102 then enters the second catalytic reactor (R102) and then to the second condenser (E103).

[0072] Since there are time lags and strong dynamic characteristics between process variables and quality variables in the sulfur recovery process, higher requirements are placed on the prediction performance of the model.

[0073] Table 1 Detailed description of variables in the sulfur recovery process

[0074]

[0075] Step 2: Build the TCN network model.

[0076] TCN has a strong ability to extract time series features. The network mainly uses dilated causal convolution and residual connection strategies. In most time series tasks, causality cannot be ignored, which means that the model can only rely on information before the current moment to make predictions. The causal convolution strategy is applied to TCN. For a time series x = (x1, x2, ..., x T ), then the causal convolution output value F(T) at time T is:

[0077]

[0078] where f=[f0,f1,...,f k-1 ] is a convolution kernel, k represents the size of the convolution kernel, xT-i Indicates that only available historical input information participates in the convolution operation. In addition, dilated convolution is applied to the network. The dilated convolution kernel can skip certain historical moment inputs according to a fixed step size, covering a longer range of historical information under the premise of the same parameters. The dilated causal convolution at time T is:

[0079]

[0080] in,* d represents the dilated convolution operation, and d is the dilation rate used to control the jump step size. As the network deepens, the dilation rate increases exponentially, and the dilation rate of the jth hidden layer is 2 j-1 .like Figure 2 As shown in Figure 2, using a larger expansion rate d can make the output of the top layer represent a wider range of inputs, and extract the temporal characteristics of the original data through multi-layer expansion causal convolution. Figure 2 In the figure, the expansion rates are d=1, 2, 4.

[0081] TCN is composed of multiple residual blocks stacked together, such as Figure 3 As shown in the figure, each residual block contains two layers of dilated causal convolutional layers, each of which uses weight normalization, ReLU activation function and Dropout strategy. In addition, the input and extracted features are fused through residual connection to finally obtain the output features of the module, where 1×1Conv can ensure the consistency of input and output dimensions. As the extracted feature information increases, higher requirements are placed on model performance and stability. TCN introduces residual connection to connect the input x T It is superimposed and fused with the output F(T) after several stacked layers to achieve cross-level information transmission. Its mathematical expression is:

[0082] y T =F(T)+x T (20)

[0083] where y T is the final output of the current residual module, which will be passed to the next residual block as input. Residual connections make information transfer and gradient propagation between layers in the network smoother.

[0084] Step 3: Design a cascaded attention fusion module to improve the residual connection of TCN.

[0085] In order to solve the problem of information loss caused by dilated convolution, the CAFF module is used to fuse the features extracted at two different time scales. The CAFF module contains two self-attention modules SAM and a multi-scale channel attention module MS-CAM, such as Figure 4As shown in the figure, the two attention mechanisms pay attention to the important information in the features from the perspective of time and variables respectively, and further extract dynamic nonlinear features. In the CAFF module, SAM pays attention to the important information in the features from the perspective of time, and MS-CAM gives corresponding attention weights according to the importance of different features from the perspective of variables, ultimately promoting better fusion of two different features X and Y.

[0086] In the initial fusion stage of features, SAM is used to assign weights to samples at different positions in the sequence. For a given input feature matrix w represents the length of the sequence, and d represents the dimension of the feature. First, a linear transformation is performed to obtain the query matrix Q, key matrix K, and value matrix V:

[0087]

[0088] in, as well as They are three trainable parameter matrices, and is the matrix of corresponding vector queries, keys and values. Then the self-attention weight matrix M between queries and keys is calculated by scaling the dot multiplication attention. S :

[0089]

[0090] The attention weight matrix M S Multiply it with V to get the SAM weighted output F S ':

[0091] F S '=M S V (23)

[0092] In the CAFF module, the two features X and Y are weighted by SAM respectively, and the historical information at different times in each feature is weighted. After obtaining the weighted features X' and Y', the two features are added together to obtain the intermediate feature H:

[0093]

[0094] Among them, S,X and M S,Y is the corresponding attention weight, V X 、V Y is the corresponding value matrix.

[0095] SAM determines the attention weights of different positions in the sequence from a temporal perspective. In order to obtain an accurate weight matrix, the importance of different features needs to be taken into account. The intermediate feature H obtained by SAM is transposed to obtain the input feature F of MS-CAM. C ∈R d×w×1 In the MS-CAM module, each dimension of the intermediate feature H is processed as a channel, d represents the number of channels of the input feature, and w×1 represents the size of the input feature. MS-CAM extracts the global feature G(F C ) and local features L(F C ), and use the Sigmoid function to calculate the channel attention weight matrix M C :

[0096]

[0097] G(F C )=Conv2(ReLU(Conv1(g(F C )))) (26)

[0098] L(F C )=Conv2(ReLU((Conv1(F C )))) (27)

[0099]

[0100] in It is a global average pooling operation. Conv1 and Conv2 have convolution kernels w / r and r 1×1 convolution, r is the reduction rate of the channel; Represents broadcast addition. MS-CAM can adjust the importance of each channel, that is, weighted aggregation of feature information of different channels in the feature map to enhance the response of important feature channels. The fusion of X and Y is realized to obtain the final fused feature Z of CAFF:

[0101] Z=M C T ⊙X+(1-M C T )⊙Y (29)

[0102] Among them, ⊙ represents the dot product operation. CAFF is used to improve the residual connection in TCN and improve the network's temporal feature extraction capability by fusing features obtained from different receptive fields.

[0103] Step 4: Design a gated TCN (GTCN), replace the dilated causal convolutional layer in TCN with a gated dilated causal convolutional layer, and replace the ReLU activation function with the SELU activation function to obtain the final CAFF-GTCN model. Its detailed structure is as follows Figure 5 shown.

[0104] In the gated TCN, the gated dilated causal convolution strategy and SELU activation function can improve the model's ability to filter and remember information. Figure 6 As shown in the figure, in order to realize the gated dilated causal convolution, two layers of dilated causal convolution layers with unshared convolution kernel parameter weights are selected. One layer is used for feature extraction, and the other layer combines the Sigmoid function to obtain the gating signal. The corresponding elements of the output of the two layers are multiplied to obtain the final output of the gated dilated causal convolution layer:

[0105]

[0106] h i =Sigmoid(h g )·(h f ),i=1,2,...,c (32)

[0107] Among them, h g As the output of the convolution kernel of the gated signal layer, h f is the convolution kernel extraction result of the feature extraction layer, h i It represents the final result after gated convolution. The convolution kernel slides in time order. The final output H obtained by gated convolution is g for:

[0108]

[0109] Where c represents the number of convolution kernels and T is the length of the input history sequence. In addition, SELU is used to replace the ReLU activation function of the TCN network, introducing the self-normalization property into the network, so that the mean and variance between layers are consistent, which can speed up the convergence speed and suppress the gradient explosion problem. The expression of the SELU activation function is as follows:

[0110]

[0111] Where α and λ are two adjustable parameters of the SELU activation function.

[0112] Step 5: Place the X train , Y train Train the CAFF-GTCN network model and optimize the network's hyperparameters, and save the trained model parameters for online prediction;

[0113] Step 6: Use the test set X testThe online detection values ​​of process variables in the sulfur recovery process are simulated and input into the CAFF-GTCN prediction model to obtain the dynamic prediction value of SO2 concentration in the tail gas. test Evaluate the performance of the model.

[0114] All experiments in the embodiments of the present application were performed on the same computer: i5-12400@2.50GHz CPU processor, 16GB memory, Pytorch1.11 under 64-bit Windows11 operating system.

[0115] Four benchmark models, SVR, SAE, LSTM, and TCN, are selected for comparison with the GTCN and CAFF-GTCN models proposed in this application to verify the superiority of the proposed models. The introduction of the four benchmark models can be found in:

[0116] For the SVR model, please refer to "Liu J, Wang Y, Zhang YA novel isomap-SVR soft sensor model and its application in rotary kiln calcination zone temperature prediction [J]. Symmetry, 2020, 12(1): 167."; for the SAE model, please refer to "Wang W, Wang H, Chen L, etal. A novel soft sensor method based on stacked fusion autoencoder with feature enhancement for industrial application[J].Measurement,2023,221:113491."; the LSTM model can refer to "Ma L, Zhao Y, Wang B, et al. A Multi-Step Sequence-to-Sequence Model with Attention LSTM Neural Networks for Industrial Soft SensorApplication[J]. IEEE Sensors Journal, 2023."; the TCN model can refer to "Yuan al.Quality variable prediction for nonlinear dynamic industrial processes based on temporal convolutional networks[J].IEEE Sensors Journal,2021,21(18):20493-20503.”.

[0117] The SRU process has strong dynamics and time lags, so the input sequence length in the LSTM, TCN, GTCN, and CAFF-GTCN networks is set to 15. In order to ensure the fairness of the experiment, the m-dimensional input of the static models SVR and SAE is dynamically augmented, and the first 14 time-lag samples are stacked on the current sample to form a 1-D dynamic vector:

[0118] x i (t) = [x i (t),x i (t-1),...,,x i (t-14)],i=1,2,…,m (35)

[0119] Where i = 1, 2, …, m. The experiment confirmed the hyperparameters of each model by trial and error. For the soft measurement based on SVR, the radial basis function was selected as the kernel function and the penalty factor was set to 15. For the soft measurement model based on deep network, in order to conduct comparative experiments, the hyperparameters of all networks should be kept at a similar level. For the soft measurement based on TCN, the size of the convolution kernel k = 3, the expansion rate d = 2, the number of residual modules is 3 and the number of corresponding convolution kernels is [32-16-8], the LSTM is 3 layers and each layer has 40 neurons, the network structure of SAE is [32-16-8], all networks are trained using the Adam optimizer, the loss function is MSE, the learning rate is 0.001, the size of each training batch is 32, and the number of network training iterations is set to 100.

[0120] In order to intuitively evaluate the performance of the model in practical applications, the root mean square error (RMSE), mean absolute error (MAE) and determination coefficient (R 2 ) are used to quantify the prediction effect of the model. RMSE, MAE and R 2 The definitions are as follows:

[0121]

[0122] Where n represents the number of samples, y i represents the true value of the sample, Represents the predicted value of the sample. RMSE and MAE focus on the prediction error of the sample. The smaller the RMSE and MAE, the better the model performance. 2 It represents the square correlation between the true output value and the predicted value, reflecting the ability of the model to explain the variance of the output data. 2 The closer it is to 1, the better the model performance. These three indicators can comprehensively reflect the prediction performance of the model.

[0123] Table 3 gives the evaluation indicators of the six methods on the test set. It can be concluded from Table 3 that SAE, LSTM, TCN, GTCN and CAFF-GTCN all provide larger R than SVR. 2and smaller RMSE and MAE. Although the input of the static model is dynamically expanded, the overall prediction performance is not as good as the dynamic model. This may be because the model does not have the memory ability and cannot extract dynamic features. LSTM uses a recursive structure to describe historical information, and the hidden features at the current moment only depend on the information at the previous moment. This makes LSTM insufficient in memory when dealing with long time series problems, so it cannot fully reveal the dynamic behavior of SRU. In contrast, TCN uses convolution kernels to cover a wider range in the time direction to fully learn the process dynamics, so the prediction performance of TCN is slightly better than LSTM. GTCN controls the flow of information by introducing a gating structure and reduces the weight of unstable features, so the prediction performance is significantly improved; on this basis, the CAFF-GTCN proposed in this application improves the local information loss problem of TCN through the attention mechanism, further enhances the memory ability and feature extraction ability, and thus exhibits the best prediction performance.

[0124] Table 3 Evaluation indicators of SO2 concentration prediction of six methods on the test set

[0125]

[0126] Figure 7 Visualize the prediction results of different methods on the test data set. Figure 7 It can be concluded that SVR cannot predict the concentration of SO2 very well. There is a large error between the red line representing the true value and the blue line representing the predicted value. SAE, LSTM, and TCN have different degrees of errors on some test set samples. For the GTCN and CAFF-GTCN models, due to their more powerful dynamic feature learning capabilities, the overall prediction error is significantly lower than other methods. In addition, Figure 8 A box plot comparison of the absolute prediction errors of the six methods on the test set is provided. It can be seen from the figure that the box plot gradually becomes narrower and closer to 0 from SVR, SAE, LSTM, TCN, GTCN to CAFF-GTCN, indicating that the prediction errors are more concentrated and smaller. The box plot widths of TCN and LSTM are similar, but TCN has a lower and more concentrated absolute prediction error value, indicating that the overall prediction error of TCN is lower. The box plot width of CAFF-GTCN is slightly lower than that of GTCN, which verifies the effectiveness of the CAFF module. Therefore, CAFF-GTCN is more suitable for mining the nonlinearity and time correlation between process variables and establishing a dynamic soft sensor model with higher prediction accuracy.

[0127] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A dynamic soft-sensing method based on cascaded attention feature fusion and gated TCN, characterized in that: The method comprises: Step S1, obtaining historical data of process variables and quality variables in a complex industrial process, where the quality variables are process parameters that need to be predicted by the soft sensor model, and the process variables are other process parameters measured by hardware sensors; Step S2, constructing a gated TCN network model CAFF-GTCN with cascaded attention feature fusion, wherein the CAFF-GTCN includes a cascaded attention fusion module CAFF and a gated TCN network structure; Step S3, using the historical data of the complex industrial process acquired in step S1 as training data to train the CAFF-GTCN model; Step S4, obtaining real-time data of process variables in a complex industrial process, inputting the CAFF-GTCN prediction model trained in step S3, and obtaining dynamic prediction values ​​of quality variables; The step S2 comprises: Step S2.1, constructing a TCN network model, wherein the TCN network model is stacked by multiple residual blocks; each residual block contains two layers of dilated causal convolutional layers, each layer adopts weight normalization, RELU activation function and Dropout strategy; at the same time, the input and extracted features are fused through residual connection, and finally the output features of the module are obtained; Step S2.2, designing a cascaded attention fusion module CAFF, using the cascaded attention fusion module CAFF to improve the residual connection of the TCN network model constructed in step S2.1, wherein the cascaded attention fusion module CAFF includes two self-attention modules SAM and a multi-scale channel attention module MS-CAM, so as to pay attention to important information in the features from the perspectives of time and variables, respectively, and further extract dynamic nonlinear features; Step S2.3, design a gated TCN network structure, replace the dilated causal convolutional layer in TCN with a gated dilated causal convolutional layer, and use the gating signal to enhance the network's ability to filter and remember information; at the same time, replace the ReLU activation function in TCN with the SELU function to suppress the gradient vanishing problem during network training; finally, the gated TCN network model CAFF-GTCN with cascaded attention feature fusion is obtained; The cascade attention fusion module CAFF is used to fuse the features extracted at two different time scales; the two features X and Y are weighted by a self-attention module SAM respectively, and the historical information at different times in each feature is weighted. After obtaining the weighted features X' and Y', the two features are added to obtain the intermediate feature H; Transpose the intermediate feature H obtained by SAM to obtain the input feature of MS-CAM In the MS-CAM module, each dimension of the intermediate feature H is processed as a channel, d represents the number of channels of the input feature, and w×1 represents the size of the input feature. MS-CAM extracts the global feature G(F C ) and local features L(F C ), and use the Sigmoid function to calculate the channel attention weight matrix M C , and get the weight matrix Through the weight matrix The features X and Y are fused to obtain the final fused feature Z of CAFF.

2. The method according to claim 1, characterized in that The self-attention module SAM is used to assign weights to samples at different positions in the sequence, and samples at different positions correspond to historical information at different times; For a given input feature matrix w represents the length of the sequence, and d represents the dimension of the feature; first, a linear transformation is performed to obtain the query matrix Q, the key matrix K, and the value matrix V; The self-attention weight matrix M is calculated based on the query matrix Q, key matrix K, and value matrix V S ; The attention weight matrix M S Multiplying with the value matrix V obtains the SAM-weighted features.

3. The method according to claim 2, characterized in that The gated dilated causal convolution layer selects two layers of dilated causal convolution layers whose convolution kernel parameter weights are not shared, one layer is used for feature extraction, and the other layer combines the Sigmoid function to obtain a gating signal; the corresponding elements of the outputs of the two layers are multiplied to obtain the final output of the gated dilated causal convolution layer.

4. The method according to claim 3, characterized in that: The expression of the SELU activation function is: Among them, λ and α are two adjustable parameters of the SELU activation function.

5. A method for predicting the SO2 concentration in tail gas of a sulfur recovery process, characterized in that: The method is implemented by using the soft measurement method based on cascaded attention feature fusion and gated TCN as described in any one of claims 1-4.

6. The method according to claim 5, characterized in that For the prediction of SO2 concentration in tail gas of sulfur recovery process, the process variables include MEA region GAS gas flow, MEA region air flow, MEA region secondary air flow, SWS region GAS gas flow and SWS region air flow; the quality variable is SO2 concentration in tail gas.

Images