A soft measurement modeling method based on residual time attention time convolution network

Abstract

The application discloses a soft measurement modeling method based on a residual time attention time convolution network and belongs to the field of soft measurement modeling. First, the time attention module is combined with the time convolution network, and the mutual influence between previous time steps and the current time step is integrated into the current time. Then, the residual connection is added on the basis of the time attention. Finally, the quality variable is predicted. The application can better extract the nonlinearity between data to improve the prediction performance and more accurately extract the relevant features between data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of soft-sensing modeling, and particularly relates to a soft-sensing modeling method based on a residual time attention time convolutional network. BACKGROUND

[0002] In modern complex industrial processes, it is often impossible to monitor and optimize key quality variables in real time due to harsh environments, technical limitations, severe delays, and other reasons, but quality variables are crucial to industrial processes. With the rapid development of economy and technology, industrialization is also rapidly advancing, and process industries such as chemical and pharmaceutical industries have more precise requirements for production modes. In actual industrial production processes, some important variable indicators are difficult to obtain online, which cannot help realize real-time monitoring of process industries in time, which has a great impact on the production quality and production output of industrial automation. Soft-sensing technology provides a solution for complex nonlinear process industries. In soft-sensing modeling, process variables collected in industrial processes are used to predict quality variables that are difficult to obtain, and the mathematical model established can accurately reflect the mapping relationship between variables. Soft sensors not only have good prediction performance, but also can improve the safety of industrial processes. The superior performance makes soft-sensing technology widely used in control systems to realize the monitoring, control and optimization of process industries. Deep learning is widely used in soft sensor modeling and has achieved good prediction performance. However, how to establish a more accurate model to better extract the correlation between data and improve the prediction performance of the model is a research hotspot in the field of soft sensing.

[0003] Recurrent neural networks have always been considered as a synonym for time series models, and have shown superior performance in industrial time series problems. This is because the long-term memory function inside RNN and LSTM determines that it can well handle the strong nonlinearity and dynamics between data and extract more effective hidden features. CNN is usually considered more suitable for processing image data and not suitable for processing time series problems. However, some recent studies have shown that, compared with recurrent neural networks, time convolutional networks perform better in handling time series problems. In soft-sensing modeling problems, the influence of some process variables on the quality variables to be predicted sometimes lasts for many time steps. Although RNN and LSTM are good at capturing long-term dependencies, the effective information transmitted will still weaken as the time series grows. In addition, RNN and LSTM must wait for the calculation results of the previous time to predict the current time, which requires longer running time when processing large-scale data. The time convolutional network (TCN) structure can well solve the above problems.

[0004] The time convolution network is not a single convolution layer, but is constructed by stacking a temporal block and flexibly using a residual connection, the temporal block is a series of stacked layers, each temporal block sequentially includes two dilated causal convolution layers and a corrected linear unit (ReLU) activation function, and in order to improve the stability of training and faster convergence speed, weight normalization is performed on each convolution kernel. In addition, a dropout regularization operation is added after the ReLU activation layer to prevent overfitting, and finally a residual connection is added to add the input of the current temporal block to the output of the second dropout layer to realize the cross-layer transmission of information, and the addition result is used as the input of the next temporal block, if the channel number of the input of the first convolution layer and the output of the second convolution layer is inconsistent, a 1*1 convolution is added in the residual connection.

[0005] The causal convolution, dilated convolution and residual connection in the TCN have greater advantages in processing the nonlinear and long-time dynamic dependence of the process variable, the size of the receptive field can be flexibly set according to the task characteristics, and the TCN inherits the basic weight sharing characteristics of the traditional convolution neural network, to a certain extent, the problems of gradient disappearance and gradient explosion existing in the RNN are relieved, and the TCN can be operated in large-scale parallel, without waiting for the operation result of the last time like the RNN, and the operation speed is also improved. However, the existing soft measurement modeling method based on TCN does not consider that there is a strong correlation between the process variable at the previous time step and the process variable at the current time. SUMMARY

[0006] The present application is directed to the deficiencies of the prior art, and proposes a soft measurement modeling method based on a residual time attention time convolution network. First, the time attention module is combined with the time convolution network, and the mutual influence between the previous time step and the current time step is integrated into the current time. However, in this process, the original information may be lost, so in order to preserve the original information to the greatest extent, a residual connection is added on the basis of the time attention, and finally the quality variable is predicted.

[0007] The specific steps of the present application are as follows:

[0008] Step S1: Obtain an industrial data set and perform preprocessing.

[0009] Step S2: Design a residual time attention time convolution network model, and perform supervised training on the training set.

[0010] The network structure consists of three stacked temporal blocks. Each temporal block is processed in sequence by the residual temporal attention module, causal expansion convolution module, weight normalization layer, ReLU layer, dropout layer, causal expansion convolution module, weight normalization layer, ReLU layer, and dropout layer. At the same time, considering that the input and output of the residual block may have inconsistent dimensions, a 1×1 convolution is added to the residual connection, and finally a linear transformation is used to obtain the predicted output.

[0011] Step S3: Based on step S2, the test set is tested using the residual temporal attention temporal convolutional network model.

[0012] As a further improved technical solution, step S1 includes the following steps:

[0013] Step S11: Collect the data set D = {X, Y} = {(x t ,y t )}, X is the set of collected process variables, Y is the set of collected quality variables, where t∈1,2,...,T, T is the number of collected samples, x t ∈R d represents the d-dimensional process variable collected at time t, y t is the quality variable at time t.

[0014] Step S12: Window processing is performed on the collected data set D, and the window size is k. The new processed data set is D′={X′,Y′}={(x (t-k)：t ,y t )}, X′ is the new process variable set after windowing X in step S11, and Y′ is the new quality variable set after windowing Y in step S11, where x (t-k)：t =[x (t-k) ,...,x t ].

[0015] Step S13: Divide the new data set obtained in step S12 into a training set and a test set in a ratio of 7:3.

[0016] As a further improved technical solution, step S2 includes the following steps:

[0017] Step S21: The training set process variables obtained in step S1 are used as the input of the residual time attention time convolutional network and the predicted value of the corresponding quality variable is output. For the prediction at time t, the input process variable of the network is x (t-k)：t ={x(t-k) ,..., x t}, network computes the quality variable y t corresponding output The calculation is as follows:

[0018] H1 = temporal block (1) {x (t-k)：t}

[0019] H2 = temporal block (2) (H1)

[0020] H3 = temporal block (3) (H2)

[0021]

[0022] As a further improved technical solution, step S21 further comprises:

[0023] Step S211: the input of the first layer temporal block (1) is the preprocessed data x (t-k)：t of step S1, which is first subjected to a residual temporal attention module to obtain where the residual attention module first integrates the potential relevant features between the current time variable and the variables before this time into the current time to obtain However, in this process, the original information may be partially lost, so it is necessary to add a residual connection to add the original input data x (t-k)：t to the subsequent operation to maximize the preservation of the original information, thereby obtaining Then sequentially perform causal dilated convolution layer, weight normalization layer, ReLU layer, dropout layer, causal dilated convolution layer, weight normalization layer, ReLU layer, dropout layer, and finally add a 1x1 convolution residual connection to add the input of the temporal block (1) to the output of the second dropout layer to obtain the hidden features H1 of the first layer temporal block, and the specific calculation process of the temporal block (1) is as follows:

[0024] First, pass through the residual temporal attention module, and use three different linear transformations f, g, h to map the input process variable x (t-k)：t to vectors keys (K), query (Q), and values (V) respectively:

[0025] K = f(x (t-k)：t )

[0026] Q = g(x (t-k)：t )

[0027] V = h(x (t-k)：t )

[0028] Secondly, the weight matrix W is calculated:

[0029]

[0030] where W i，j represents the i-th row and j-th column of the weight matrix W, i, j = t-k,..., t, d k is the dimension of keys, query and values. The lower triangular part of W is extracted to get W s :

[0031]

[0032] Then, the softmax is performed on the first dimension of W s , and then multiplied by V to get

[0033]

[0034] So far is the result of the time attention module. Then, in order to maximize the preservation of original information, a residual connection is added:

[0035]

[0036] The result of the residual time attention module is Finally , as with a general temporal convolution network, the input of the temporal block (1) is the output of the first causal dilated convolution layer, which is sequentially passed through a weight normalization layer, a ReLU layer, and a dropout layer, and then enters two convolution layer modules, which are sequentially passed through a weight normalization layer, a ReLU layer, and a dropout layer. Assuming z is the actual calculation output of the above series of stacked layers, the input of the temporal block (1) is added to the output of the second dropout layer through a 1x1 convolution residual connection. The 1x1 convolution is only to prevent the channel number of the input of the temporal block (1) from being inconsistent with the output of the second dropout layer. If the channel numbers are consistent, then a 1x1 convolution is not needed. Assuming the channel numbers are consistent, then:

[0037]

[0038] Finally, H1 is the output of the first layer temporal block.

[0039] Step S212: The input of the second layer temporal block is the output H1 of the first layer temporal block, and then the same as the first layer, it goes through a series of calculation processes such as residual temporal attention module, causal dilated convolution module, weight normalization layer, ReLU layer, dropout layer and residual connection module, to obtain the output H2 of the second layer temporal block, and the temporal block (2) The calculation process is as follows:

[0040] H2 = z(H1) + H1

[0041] Wherein, w(H1) is the actual calculation output of a series of stacked layers containing the residual temporal attention module.

[0042] Step S213: The input of the third layer temporal block is the output H2 of the second layer temporal block, and the same as the first layer and the second layer temporal block, it goes through the same network structure and calculation process to obtain H3, and the temporal block (3) The calculation process is as follows:

[0043] H3 = z(H2) + H2

[0044] Step S214: The result H3 obtained by the third layer temporal block is transformed by a linear transformation to obtain the network prediction quality variable, and the calculation method is as follows:

[0045]

[0046] Step S22: The hyperparameters of the residual temporal attention time convolutional neural network model are changed multiple times, including the convolution kernel size f and the number of convolution kernels k n , and the objective function loss under different hyperparameters is calculated, so that the parameters with the minimum objective function are the optimal hyperparameters of the residual temporal attention time convolutional network.

[0047] As a further improved technical solution, step S22 includes the following steps:

[0048] Step S221: In the case of a certain number of network structures temporalblock layers, in order to ensure that the range of the receptive field can completely cover the input information, according to the formula satisfied by the receptive field:

[0049] r = 1 + 2 * (f-1) * 2 n -1

[0050] Where r is the receptive field size, f is the convolution kernel size, n is the number of network temporal block layers, and the size of the convolution kernel can be determined according to the required receptive field size of the specific task.

[0051] Step S222: After determining the size of the convolution kernel, the most appropriate number of convolution kernels k needs to be set n In order to determine the optimal number of convolution kernels, the hyperparameters of the network are first randomly set, and the objective function loss under different parameter conditions is calculated to obtain the hyperparameters with the minimum objective function as the optimal hyperparameters of the residual temporal attention temporal convolution network.

[0052] Compared with the prior art, the beneficial effects of the present application are:

[0053] The temporal convolution network structure has certain advantages in processing long-time complex data, and the present application considers that there is more or less correlation between variables at each time and variables at previous times on the basis of the temporal convolution network. The present application can more accurately extract the relevant features between data, and compared with other methods, the prediction performance is improved to a certain extent. The technical scheme of the present application is applied to the soft sensor of the de-butanizer to realize accurate prediction of butane concentration. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 It is an example of dilated causal convolution for the dilation factor d = 1, 2, 4 and the convolution kernel size f = 3;

[0055] Figure 2 It is an example of residual connection for the dilation factor d = 1 and the convolution kernel size f = 3;

[0056] Figure 3 It is a network structure diagram of the soft measurement modeling method of the temporal convolution network;

[0057] Figure 4 It is a structure diagram of the temporal attention;

[0058] Figure 5 It is a network structure diagram of the soft measurement modeling method of the residual temporal attention temporal convolution network based on the present application;

[0059] Figure 6 It is a residual temporal attention structure diagram;

[0060] Figure 7Specific prediction result figures of the four models. DETAILED DESCRIPTION

[0061] The technical solutions of the present application will be further described below with reference to the drawings.

[0062] Example 1

[0063] The present application provides a soft measurement modeling method based on a residual time attention time convolutional network, which is applied to a debutanizer soft sensor to realize accurate prediction of butane concentration. The debutanizer is an important unit for removing butane (C4) from naphtha. However, the concentration of C4 is not directly measured at the bottom, but obtained at the top by a gas chromatograph. The position of the analyzer causes a certain time delay in the measurement of the C4 concentration. Therefore, the soft sensor design can realize online estimation of the C4 concentration, and real-time monitoring of the C4 concentration is realized in order to reduce the C4 to a minimum and maximize profits. Therefore, the soft sensor is used for prediction.

[0064] The causal convolution, dilated convolution and residual connection in the time convolutional network (TCN) have greater advantages in processing the nonlinear and long-time dynamic dependence of process variables. The size of the receptive field can be flexibly set according to the characteristics of the task, and the problems of gradient disappearance and gradient explosion are alleviated. At the same time, TCN can be run in large-scale parallel, and the operation speed is also improved. The causal dilated convolution and residual connection are shown in Figure 1 and Figure 2 , where d is the dilated coefficient, f is the kernel size, and zero-padding is used to ensure that the length of the input and output tensors is the same. Figure 1 When the dilated coefficient is 1, it is a normal convolution, and increasing d and f can increase the receptive field. Zero-padding can ensure that the input and output tensor lengths are the same. Figure 2 When the number of input channels and the number of output channels are not equal, a 1x1 convolution can be selected.

[0065] With the complexity of industrial processes, the correlation between data is becoming more and more complex. TCN can flexibly change the kernel size and receptive field size according to the specific task, and also consider the variables a long time ago, better handle the nonlinearity and dynamics between input variables, so TCN is more suitable for industrial soft measurement modeling than recurrent neural networks, such as Figure 3 is a soft measurement time convolutional model (TCN) with three temporal blocks. However, although the original TCN considers that there is a strong correlation between input variables, in fact, there is a certain correlation between variables at each time and variables before that time. Therefore, the present application provides a soft measurement modeling method based on a residual time attention time convolutional network, and the network structure diagram is as followsFigure 5 , first introduce time attention in TCN, like Figure 4 Integrate the input information at the current time and the input information before this time together as the new input of each layer temporal block, so that the potential related features between them can be extracted, but in the integration process, the original information may be lost, so in order to solve this problem, the attention mechanism is introduced and the residual connection is added, as shown in Figure 6 , the original information is retained to the maximum extent. Finally, the linear transformation is used to obtain the prediction output.

[0066] More specifically, the industrial soft sensor modeling method based on time attention time convolution network includes the following steps:

[0067] Step S1: collect the data set for the de-butane column butane concentration soft measurement task, and pretreat;

[0068] Step S1 includes the following steps:

[0069] Step S11: collect the data set D={X,Y}={(x t ,y t )} for industrial soft measurement modeling, X is the collected process variable set, Y is the collected quality variable set, wherein t∈1,2,...,T, T is the number of collected samples, which is 2393, x t indicates the process variable collected at t time, wherein the process variable has 7, y t is the quality variable at t time, note that in order to better learn the nonlinear dynamic relationship between variables, the quality variable at t-1 time is also taken as the quality variable at t time, so that the quality variable at t time has 8, that is, x1(t), x2(t), x3(t), x4(t), x5(t), x6(t), x7(t), y(t-1).

[0070] Step S12: window processing is performed on the collected data set D, the window size is k=39, and the new data set after processing is D'={X',Y'}={(x (t-k)：t ,y t )}, X' is the new process variable set after window processing of X in step S11, Y' is the new quality variable set after window processing of Y in step S11, wherein x (t-k)：t =[x (t-k) ,...,x t ], the number of samples at this time is 2354.

[0071] Step S13: The new data set obtained in step S12 is divided into a training set and a test set in a ratio of 7:3, so there are about 1648 groups of training set and 706 groups of test set.

[0072] Step S2: A residual time attention time convolution network model is designed, and supervised training is performed on the training set. The residual time attention time convolution network model is as shown in Figure 5 .

[0073] The network structure includes three layers of temporal block. Each layer of temporal block is sequentially processed through a residual time attention module, a causal dilated convolution module, a weight normalization layer, a ReLU layer, a dropout layer, a causal dilated convolution module, a weight normalization layer, a ReLU layer, and a dropout layer. Each convolution kernel is subjected to weight normalization, which is to improve the stability of training and faster convergence speed, which can offset problems such as gradient explosion. In addition, a dropout regularization operation is added after the ReLU activation layer to prevent overfitting. At the same time, considering that the input and output of the residual block may not be consistent in dimension, a 1x1 convolution is added in the residual connection, and finally a linear transformation is used to obtain the predicted output.

[0074] Step S2 includes the following steps:

[0075] Step S21: The training set process variable obtained in step S1 is taken as the input of the residual time attention time convolution network, and the predicted value of the corresponding quality variable is output. For the prediction at time t, the input process variable of the network is x (t-k)：t ={x (t-k) ,..., x t}, and the predicted value of the corresponding output quality variable yt after network calculation is The calculation method is as follows:

[0076] H1 = temporal block (1) {x (t-k)：t}

[0077] H2 = temporal block (2) (H1)

[0078] H3 = temporal block (3) (H2)

[0079]

[0080] Step S21 further comprises:

[0081] Step S211: temporal block (1) The input of step S21 is the data x (t-k)：t preprocessed in step S1, which is first processed by a residual temporal attention module to obtain wherein the residual attention module first integrates the potential relevant features between the current time variable and the variables before the time into the current time to obtain However, in this process, the original information may be partially lost, so it is necessary to add a residual connection to add the original input data x (t-k)：t to the subsequent operation to maximize the preservation of the original information, thereby obtaining Then sequentially perform causal dilated convolution layer, weight normalization layer, ReLU layer, dropout layer, causal dilated convolution layer, weight normalization layer, ReLU layer, dropout layer, and finally add a 1x1 convolution residual connection to add the input of the temporal block (1) to the output of the second dropout layer to obtain the hidden features H1 of the first layer temporal block, and the specific calculation process of the temporal block (1) is as follows:

[0082] As shown in Figure 4 , first pass through the time attention module, use three different linear transformations f, g, h to map the input process variable X (t-k)：t to vectors keys (K), query (Q) and values (V) respectively:

[0083] K = f(x (t-k)：t )

[0084] Q = g(x (t-k)：t )

[0085] V = h(x (t-k)：t )

[0086] Then calculate the weight matrix W:

[0087]

[0088] wherein W i，j represents the i-th row and j-th column of the weight matrix W, i, j = t-k,..., t, d kis the dimension of keys, query and values. At the same time, because of the characteristics of time series task, future information cannot be disclosed, so the lower triangular part of W is extracted to get W s ( Figure 4

[0089]

[0090] Then do softmax on the first dimension of W s , and do dot product with V to get

[0091]

[0092] So far is the result of time attention module, then add residual connection to retain the original information as much as possible, as shown in Figure 6 :

[0093]

[0094] The result of residual time attention module is Finally , as a temporal block (1) , the input of the first causal dilated convolution layer, through weight normalization layer, ReLU layer and dropout layer in turn, and then enter two convolution layer modules, through weight normalization layer, ReLU layer and dropout layer in turn, assuming that z is the actual calculation output of the above series of stacked layers, finally 1x1 convolution residual connection adds the input of temporal block (1) to the output of the second dropout layer, and 1x1 convolution is only to prevent the channel number of temporal block (1) input and the output of the second dropout layer from being inconsistent, if the channel number is consistent, then 1x1 convolution is not needed, assuming that the channel number is consistent, then:

[0095]

[0096] Finally, H1 is the output of the first layer of temporal block.

[0097] ​Step S212: The input of the second layer of the network temporal block is the output H1 of the first layer of the temporal block, and then the same as the first layer, it passes through a series of calculation processes such as a residual temporal attention module, a causal dilated convolution module, a weight normalization layer, a ReLU layer, a dropout layer, and a residual connection module, to obtain the output H2 of the second layer of the temporal block, and the temporal block (2) The calculation process is as follows:

[0098] H2 = z(H1) + H1

[0099] where w(H1) is the actual calculation output of a series of stacked layers containing a residual temporal attention module.

[0100] Step S213: The input of the third layer of the network temporal block is the output H2 of the second layer of the temporal block, and the same as the first layer and the second layer of the temporal block, it passes through the same network structure and calculation process to obtain H3, and the temporal block (3) The calculation process is as follows:

[0101] H3 = z(H2) + H2

[0102] Step S214: The result H3 obtained by the third layer of the temporal block is transformed by a linear transformation to obtain the network prediction quality variable, and the calculation method is as follows:

[0103]

[0104] Step S22: The hyperparameters of the residual temporal attention temporal convolutional neural network model are changed multiple times, including the convolution kernel size f and the number of convolution kernels k n , and the objective function loss under different hyperparameters is calculated, so that the parameters that minimize the objective function are the optimal hyperparameters of the residual temporal attention temporal convolutional network, and the objective function is calculated as follows:

[0105]

[0106] where j represents the calculation of all samples in the training set, and m is the number of training set samples, and at this time m = 1648.

[0107] Step S22 includes the following steps:

[0108] Step S221: In the case of a certain number of network structure temporal block layers, in order to ensure that the range of the receptive field can completely cover the input information, according to the formula that the receptive field satisfies:

[0109] r = 1 + 2 x (f - 1) x (2 n -1)

[0110] Wherein r is the receptive field size, f is the convolution kernel size, n is the number of network temporal block, according to the specific task required receptive field size can determine the size of the convolution kernel, at this time known n = 3, the input length is 40, the receptive field must be greater than or equal to the input length, so the size of the convolution kernel f = 4 can be obtained.

[0111] Step S222: after determining the size of the convolution kernel, the most appropriate number of convolution kernels k needs to be set n In order to determine the optimal number of convolution kernels, first set the network hyperparameters learning rate, batch size, epoch to 0.001, 64 and 240 respectively, then set the number of convolution kernels to 20, 30 and 40 respectively, and calculate the objective function loss under different parameters, get the objective function minimum parameter is the optimal parameter of residual temporal attention temporal convolution network, experimental verification, when the number of convolution kernels is 30, it is the optimal parameter.

[0112] Step S3: on the basis of step S2, using residual temporal attention temporal convolution network model to test the test set.

[0113] Step S3 includes the following steps:

[0114] Step S31: input the test set into the residual temporal attention temporal convolution network model, and the model outputs the test quality variable prediction value according to the mode described in step S21.

[0115] Step S32: calculate the root mean square error (RMSE) measurement index of the test set, the calculation method is as follows:

[0116]

[0117] Wherein T is the sample number of the test set, y t and Respectively represent the label value and the prediction value at time t.

[0118] In order to verify the effectiveness of the residual temporal attention temporal convolution network, AR-TCN model, LSTM model and SAE model are established to predict butane concentration as comparison. In order to make the results fair and reasonable, the parameters of the model AR-TCN in the comparison experiment and the proposed RTA-TCN are consistent, wherein the number of temporal block (b n) are all set to 3 and kernel size (f) are all set to 4, both AR-TCN and RTA-TCN are trained with Adam optimizer, learning rate is 0.001, batch size is set to 64, epoch is 240; window size k of LSTM model is set to 40, containing a hidden layer with 60 neurons; and SAE, a network model with one-dimensional vector as input, stacks the first 39 time-lag samples in the current sample, so the network structure of SAE is set to [320, 250, 140, 80, 60, 5], and the experimental results are shown in Table 1 as follows:

[0119] Table 1

[0120]

[0121] As can be seen from the table, the prediction performance of the AR-TCN model has been significantly improved compared with LSTM and SAE, and the proposed residual time attention time convolution network (RTA-TCN) soft measurement modeling method shows better prediction performance, with the minimum RMSE of 0.00652, the maximum R 2 and the maximum of 0.99870.

[0122] The specific prediction results of the four models are shown in Table 2 as follows: Figure 7 LSTM shows the worst prediction results, which may be because LSTM is poor at capturing the nonlinear and dynamic relationship between data; SAE performs a little better than the LSTM model, SAE uses unsupervised pre-training, which can extract more effective features, but the input of SAE is a one-dimensional vector stacked by multiple lag data, which may break the autocorrelation structure of some variables, so the model performance is biased among the four models; AR-TCN can already show good performance, AR-TCN model can describe the nonlinearity and dynamics of process variables better and extract more representative high-level features. As can be seen from the results, under the same input conditions, the time series convolution network shows better prediction performance than the convolution neural network in time series tasks. Figure 7The middle (c) shows that the AR-TCN performs very well on the first half of the data, but the performance on the last few data is not very good, while the RTA-TCN model performs best on the whole data set and can well track the true value of the test set data, because compared with the AR-TCN, the RTA-TCN also considers the potential correlation between the previous time variable and the current time variable, integrates the internal influence of the previous time variable on the current time variable into the current time, and adds a residual connection to prevent the loss of original information that may occur during integration, so that more representative hidden features can be extracted.

Claims

1. A soft-sensing modeling method based on a residual temporal attention temporal convolution network, characterized in that, Comprise the following specific steps: Step S1: obtain the data set for the depropanizer propane concentration soft measurement task, and pretreat; Step S2: design a residual time attention time convolution network, and perform supervised training on the training set; The network comprises three layers of stacked blocks, and each layer of the stacked blocks is sequentially subjected to a residual time attention module, a causal dilated convolution module, a weight normalization layer, a ReLU layer, a dropout layer, a causal dilated convolution module, a weight normalization layer, a ReLU layer, and a dropout layer; Meanwhile, a 1×1 convolution is added in the residual connection of the residual time attention module, and finally a linear transformation is used to obtain the prediction output; Step S3: based on step S2, the residual time attention time convolution network model is used to test the test set; Step S1 comprises the following specific steps: Step S11: collect dataset D = {X, Y} = {(x t ,y t )}, X is the collected process variable set, Y is the collected quality variable set, where t ∈ 1, 2,..., T, T is the number of collected samples, x t ∈ R d represents the d-dimensional process variable collected at time t, y t is the quality variable at time t; Step S12: windowing the collected dataset D with window size k, the new dataset after processing is D' = {X', Y'} = {(x (t-k):t , y t )}, X' is the new process variable set after windowing X in step S11, Y' is the new quality variable set after windowing Y in step S11, where x (t-k):t = [x (t-k) ,..., x t ] ; Step S13: divide the data set D' into a training set and a test set in proportion; Step S2 comprises the following specific steps: Step S21: taking the process variables in the training set as the input of the residual time attention time convolution network, and outputting the predicted value of the corresponding quality variable, wherein for the prediction at time t, the input process variables of the network at this time are x (t-k):t ={x (t-k) ,...,x t} , and the predicted value of the corresponding output quality variable y t of the network after calculation Step S22: multiple times change the hyperparameters of the residual temporal attention temporal convolutional neural network model, including the convolution kernel size f and the number of convolution kernels k n And calculate the objective function loss under different parameter conditions, so that the hyperparameters that minimize the objective function are the optimal hyperparameters of the residual temporal attention temporal convolutional network.

2. The soft-sensing modeling method based on the residual temporal attention time convolution network according to claim 1, characterized in that, Step S21 specifically comprises: Step S211: for the first layer temporal block, the input of the first layer temporal block is the preprocessed data x (t-k):t , first through the residual temporal attention module to obtain Wherein the residual attention module first passes through the temporal attention to integrate the potential correlation features between the current time variable and the variable before the time into the current time to obtain At the same time, a residual connection is added to the original input data x (t-k):t After the subsequent operation, thereby obtaining Then sequentially perform the causal dilated convolution layer, the weight normalization layer, the ReLU layer, the dropout layer, the causal dilated convolution layer, the weight normalization layer, the ReLU layer, the dropout layer, and finally add the 1×1 convolution residual connection to add the input of the first layer temporal block to the output of the second dropout layer to obtain the hidden features H1 of the first layer temporal block output; The specific calculation process of the residual time attention module is as follows: Firstly, the residual time attention module is used to map the input process variables x (t-k):t to vectors K, Q and V using three different linear transformations f, g, h, respectively, and then to calculate the weight matrix W: where W i,j represents the weight matrix W, i, j = t-k,..., t, d k is the dimension of the vectors K, Q and V, while the lower triangular part of W is extracted to obtain W s : Then in the first dimension of W s ​ Up to this point The result from the temporal attention module is then added with a residual connection in order to preserve the original information to the maximum extent: The result obtained by the residual temporal attention module is Step S212: the input of the second layer of the temporal block is the output H1 of the first layer of the temporal block, and then the same calculation process as the first layer of the temporal block is performed to obtain the output H2 of the second layer of the temporal block; Step S213: the input of the third layer of the temporal block is the output H2 of the second layer of the temporal block, and the same calculation process as the first layer and the second layer of the temporal block is performed to obtain H3, Step S214: the result H3 obtained by the third layer of the temporal block is subjected to a linear transformation to obtain the network prediction quality variable, and the calculation method is as follows: Wherein, Linear is a linear transformation.

3. The soft-sensing modeling method based on the residual temporal attention temporal convolution network according to claim 2, characterized in that, In step S21, the last 1×1 convolution residual connection of each temporal block layer adds the input of the first layer of the temporal block to the output of the second dropout layer, but when the number of channels is consistent, the 1×1 convolution is not performed.

4. The soft-sensing modeling method based on the residual temporal attention time convolution network according to claim 3, characterized in that, In step S22, the convolution kernel size f and the number of convolution kernels k are determined n Specifically includes the following steps: Step S221: under the condition that the number of network structure temporal block layers is certain, according to the formula that the receptive field satisfies: r＝1+2×(f-1)×2 n -1 Wherein, r is the receptive field size, f is the convolution kernel size, and n is the number of network temporal block layers; Step S222: after the convolution kernel size is determined, the number of convolution kernels k is set n First, the hyperparameters of the network are initialized and set, and the objective function loss under different parameter conditions is calculated to obtain the optimal hyperparameters of the residual temporal attention temporal convolution network as the hyperparameters of the minimum objective function.

Images