TTPA-LSTM soft measurement method based on multiple sampling rates

Abstract

The invention discloses a TTPA-LSTM soft measurement method based on multiple sampling rates, and belongs to the field of soft measurement modeling and application in the modern industrial process. The method comprises the step of accurately predicting the concentration of butane at the bottom of the debutanizer in the industrial process by constructing a TTPA-LSTM model. According to the invention, through a non-incremental heuristic function related to a time interval in the time sensing module, short-term memory is weighted, more information is reserved while a down-sampling rate method is simulated, and the problem of difficult feature extraction caused by high and low sampling rates is solved; on the other hand, a time mode attention module is introduced, cross-time-step integration is achieved on obtained features through a two-dimensional convolution filter, the problem of multivariate time sequence time delay is solved, meanwhile, the prediction capacity of the model is improved, and the problem existing in actual industrial production is solved.

Description

Technical Field

[0001] The invention relates to a TTPA-LSTM soft measurement method based on multiple sampling rates, and belongs to the field of soft measurement modeling and application of modern industrial processes. Background Art

[0002] In actual industrial processes, effectively obtaining quality variables directly related to production is the core of process control. However, due to the limitations of measurement technology and complex industrial environments, it is difficult to directly obtain these quality variables. In order to solve the problem of quality variable collection in complex environments, soft measurement technology that indirectly obtains quality variables through easily measured process variables has become the key to improving the quality and efficiency of current manufacturing production. Soft measurement modeling methods are divided into two categories: mechanism modeling and data-driven modeling. Mechanism modeling strictly relies on the basic principles of the system such as physics and chemistry to build models. It needs to consider multiple parameters, resulting in a complex structure of the model. It is also difficult to adapt to the nonlinearity and time-varying nature of the industrial process, and the generalization ability of the model is limited. Data-driven modeling methods have the advantages of reducing dependence on mechanism process knowledge, simplifying the modeling process, and being able to cope with the uncertainty and randomness in actual industrial processes. They have developed rapidly in the field of industrial process modeling. Machine learning methods in data-driven modeling methods mainly include artificial neural networks, principal component regression, Gaussian process regression, and extreme learning machines. Among them, artificial neural network models are widely used due to their powerful nonlinear learning ability.

[0003] However, with the increasing number of variables collected in industrial processes, it is difficult for each detection device to unify the sampling period due to factors such as process characteristics, distributed control, signal transmission, and economy, forming a multi-sampling rate data, which puts higher requirements on the learning ability of the deep model. Soft sensing methods for multi-sampling rate data are mainly divided into two categories. One is to use data preprocessing technology to organize multi-sampling rate data outside the model based on up and down sampling and other methods before modeling; the other is to directly model through a specific model to reduce the impact of information loss caused by multiple sampling rates. Data preprocessing technology usually uses methods such as numerical interpolation (NI) to fill in the missing data, or uses downsampling methods to reduce the overall data to a low sampling rate. However, due to the problems of excessive data volume, poor data quality, limited computing resources, and aliasing, data preprocessing methods are difficult to effectively target multi-sampling rate characteristics, and direct modeling methods usually use specific methods combined with neural networks to build relevant models to solve the problem of multi-sampling rate data modeling. On the other hand, the time series data sets of industrial processes have sampling rate differences among the variables and also contain time lag characteristics. This time lag characteristic in multivariate time series (hereinafter referred to as multivariate time series lag) makes it difficult for time-aware models that do not have the ability to cross time steps to parse this special time series characteristic in the data, which in turn affects the modeling results.

[0004] Aiming at the problem of multi-sampling rate data modeling in industrial processes, the present invention proposes a time-aware temporal pattern attention (TTPA) mechanism by considering different time lags in multivariate time series, and builds a new soft measurement model in combination with a long short-term memory (LSTM) network. Summary of the invention

[0005] Aiming at the problem that the existing soft measurement technology does not fully and comprehensively consider the modeling of multi-sampling rate data and the different time lags in the multivariate time series, which leads to the reduction of modeling quality and the inapplicability of the model, the present invention proposes a TTPA mechanism and provides a TTPA-LSTM soft measurement method based on multi-sampling rate. The method is based on the TTPA mechanism and the LSTM network, and is applied to the soft measurement modeling of the actual industrial process, thereby realizing the accurate prediction of its key quality variables.

[0006] The technical solution of the present invention is:

[0007] Step 1: Collect process variables and quality variables in actual industrial processes as sample data:

[0008] Analyze the process characteristics and select process variables that have a greater impact on the prediction of quality variables from the industrial process as the input variables of the model. The sampling rates of different process variables are inconsistent due to various factors, and the sampling frequency of quality variables is much lower than that of other variables. The sampled data set is recorded as [X, Y].

[0009] Step 2: Preprocess the sample data:

[0010] Analyze the sampling rate of each variable of the obtained data and record it, remove outliers, divide batches and time steps, etc. Specifically, it includes: first observe the missing values ​​of the obtained data, calculate the sampling rate of each variable according to the basic sampling interval; use the 3σ criterion to remove data points that are obviously not within a reasonable range. Secondly, according to the model training requirements, divide the data into appropriate batches and input them into the model to improve the modeling accuracy. Finally, analyze the time series relationship of the variables and expand the data dimension with a reasonable time step to obtain better prediction results.

[0011] Step 3: Divide the preprocessed dataset and build a time-aware model:

[0012] The preprocessed data is divided into training set and test set in a ratio of 4 to 1. The non-increasing heuristic decay function is calculated according to the sampling time interval of the data, and the time-aware model is constructed to reconstruct the data corresponding to the high and low sampling rates into short-term and long-term information respectively, and complete the feature extraction of multi-sampling rate data.

[0013] Step 4: Build the TTPA-LSTM model in combination with the time-aware model and set the parameters:

[0014] The row vector is used to extract the hidden features of the previous moment obtained by the time perception model, the number of convolution filters and the dimension of the output variable are designed, and the obtained features are transmitted to the convolution filter. The features of each moment are integrated according to the set time step and then passed to the scoring function to calculate the attention score of the hidden features at each moment. After the TTPA mechanism analyzes the interdependence between multiple variables in the previous time series, the results are finally input into the fully connected layer to obtain the predicted value and calculate the performance evaluation index.

[0015] Step 5: Optimize the parameters of the TTPA-LSTM model:

[0016] The Adam optimizer is used to update the model parameters by calculating the mean absolute error between the predicted value and the true value, and the result is back-propagated to calculate again, and the parameters are continued to be updated until the training loss no longer decreases, and the trained model is saved.

[0017] Step 6: Input the process variables corresponding to the quality variables to be predicted into the trained TTPA-LSTM model to obtain the predicted values:

[0018] Finally, the effectiveness and superiority of the model were verified by the soft measurement application of butane concentration at the bottom of the debutanizer industrial process. The experimental results show that the proposed model has a smaller error fluctuation and lower deviation value for the prediction of butane concentration at the bottom of the debutanizer dataset with multi-rate sampling and multivariate time series lag characteristics than other multi-rate sampling methods. The overall prediction effect is better than other methods and can better predict the dynamic trend of quality variables.

[0019] Furthermore, step 3 includes: constructing a time-aware module for extracting multi-sampling rate data features:

[0020] In the time perception module of the TTPA mechanism, LSTM is introduced to fully extract multi-sampling rate features and decompose and weight long-term and short-term information. When modeling multi-sampling rate data, the time between sequence elements in the LSTM network is evenly distributed. This way of reading long and short-term information is difficult to judge the importance of information based on the variable time interval, resulting in a small proportion of quality-related information in the extracted features. Especially when the sampling rate of the target data is unbalanced by multiple times, this shortcoming will be seriously magnified, which will have an adverse effect on the prediction results. Therefore, the concept of time perception is combined with the LSTM network, and the subspace decomposition method is used to achieve the effect of integrating time interval information into the LSTM. This interval is manifested in multi-sampling rate data as a sampling interval difference caused by different sampling rates.

[0021] In the calculation process of this module, the cell state C at the previous moment is used t-1 Get short-term memory And adjust it according to the weight to get the weighted short-term memory At the same time, C t-1 Get removed The long-term subspace component The final combination obtains the adjusted previous memory The detailed mathematical expression is as follows:

[0022]

[0023]

[0024]

[0025]

[0026] Among them, W d and b d are the weight and threshold of subspace decomposition, ⊙ is the multiplication operation of the corresponding elements of the matrix, g(Δ t ) is a heuristic decay function used to convert the elapsed time into a weight. The mathematical expression of the function is t Since the time interval of multi-sampling rate data in industrial processes is fixed, the introduction of short-term memory and Δ t Combined with the trainable parameters of the model, the heuristic decay function g(Δ t ) is defined as:

[0027]

[0028] Where e is the natural logarithm and ε is less than 10 -5 Random variables, in the time perception module, for some information with lower importance in the previous moment, it is attenuated and weighted; the larger the sampling interval, the smaller the proportion of variables with lower sampling rates in the overall information. If some short-term information has little impact on the current result due to sampling rate and other reasons, it is proportionally adjusted according to the time span to reduce this impact. Therefore, the problem of information difference caused by different sampling rates is solved by a non-increasing heuristic decay function. After decomposing the input information, the short-term component is converted into corresponding weights according to the elapsed time and weighted, and the cell memory is used to approximate the learning time interval. Finally, the obtained short-term component is combined and adjusted with another component obtained by the previous decomposition, reducing the impact of short-term effects without losing long-term related information. It can simulate the effect of the downsampling method and extract multi-sampling rate data features without loss.

[0029] After such processing, the obtained information not only contains the original long and short-term memories, but also constructs a non-increasing heuristic decay function according to the multi-sampling rate time intervals to weight the short-term information. This can effectively deal with the data imbalance problems between process variables and between process variables and quality variables under multi-sampling rate conditions, and improve the multi-sampling rate data modeling effect of the model.

[0030] Further, step 4 includes: constructing a feature optimization module based on temporal pattern attention;

[0031] After processing the input information, the time perception module has completed the extraction of multi-sampling rate time interval information. When the model processes the time lag in the multivariate time series, it passes the hidden information obtained by the time perception module to the feature optimization module to construct the TTPA mechanism, thereby realizing the weighted summation operation of the multivariate vector across time steps to capture more sufficient time information. This module first integrates the hidden information h from the previous moment t-n , extract the row vector and input it to K convolution filters C j Operation, let the maximum time length be T, divide the historical information into m row vectors to enhance the feature extraction ability of the model, when the prediction contains w time lag data, the convolution result of the i-th row vector corresponding to the j-th filter is obtained The expression is:

[0032]

[0033] Among them, H i,(t-w-1+l) is the result of the convolution operation on the i-th row and the tw-1+l-th column, C j,T-w+l is the corresponding convolution filter;

[0034] To obtain the weighted sum of row vectors V t , the score function f is defined to evaluate the convolution result corresponding to each row vector Relevance:

[0035]

[0036] in, is the convolution result of the i-th row vector, W a Calculate the attention weight matrix for function f, h t Hide information for the current moment;

[0037] Then calculate the attention score α i The value of , and the attention score is obtained through the sigmoid activation function:

[0038]

[0039] On this basis, the weighted sum V tThe expression is:

[0040]

[0041] And hide the information h at the current moment t Combined to get the final output The expression is:

[0042]

[0043] Among them, W h , W v They are respectively t and V t The weights of each calculation.

[0044] Compared with the typical attention mechanism that only focuses on the processing of a single time step, the TTPA mechanism can select time series information that has a greater impact on the model's prediction results. Through a two-dimensional convolution filter, each time step in the sequence is bidirectionally filtered, hidden information at each moment is extracted across time steps, and the resulting vectors are weighted and summed to resolve the impact of interdependence in the time series, effectively obtaining the time lag information contained in the previous time steps, and selecting variables that have a greater impact on the prediction results. At the same time, by selecting information across time steps through convolution filters and attention mechanisms, it is possible to accurately determine that some hidden information in a previous moment has more predictive value. The features obtained in this way are more effective than traditional methods.

[0045] In summary, the present invention aims at the problem of multi-sampling rate data modeling in industrial processes, and considers different time lags in multivariate time series, proposes a TTPA mechanism, and combines LSTM to build a new soft measurement model. The model uses a time perception module to complete the subspace decomposition of the input information, constructs a non-increasing heuristic function to convert the elapsed time information into weights, and realizes feature extraction of multi-sampling rate data through the operation of long and short-term memory, while simulating the effect of downsampling, it acquires information losslessly, and improves the modeling ability of multi-sampling rate data. The obtained time series features are further processed by the feature optimization module, the time lag information between variables in each time step is selected in a targeted manner, and the hidden information that has a greater impact on the prediction result at the current moment is extracted across time steps. At the same time, new global features are obtained after optimization by the scoring mechanism to improve the prediction accuracy of the model.

[0046] The beneficial effects of the present invention are:

[0047] Aiming at the problem of multi-sampling rate containing multivariate time series time lag in industrial process data, the present invention proposes a TTPA-LSTM soft sensing method based on multi-sampling rate; by constructing a TTPA-LSTM model, the problem of multi-sampling rate data modeling in industrial processes and the problem of reduced modeling quality and inapplicability of the model caused by different time lags in multivariate time series is solved; in terms of multi-sampling rate, the proposed model uses a non-increasing heuristic function related to the time interval in the time perception module to weight the short-term memory, simulates the downsampling rate method while retaining more information, and solves the problem of feature extraction difficulties caused by high and low sampling rates; on the other hand, the temporal pattern attention module is introduced to integrate the obtained features across time steps through a two-dimensional convolution filter, solves the problem of multivariate time series time lag, and improves the prediction ability of the model. Modeling is implemented for the debutanizer industrial process simulation case to solve such problems existing in actual industrial production. Compared with the representative methods of existing multi-sampling rate modeling, the present invention has higher prediction accuracy for industrial processes with different sampling rates and multivariate time series time lags, verifying the superiority and effectiveness of the model prediction ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 It is a diagram illustrating the multi-sampling rate data of the industrial process according to the present invention;

[0050] Figure 2 It is a multi-sampling rate feature processing flow chart of the time perception module of the present invention;

[0051] Figure 3 The TTPA-LSTM model structure diagram provided by the present invention;

[0052] Figure 4 This is the process flow diagram of debutanizer;

[0053] Figure 5 Box plots of the prediction errors of butane concentration for each multi-sampling rate model;

[0054] Figure 6 This is the prediction result curve of butane concentration at the bottom of the tower by TTPA-LSTM model;

[0055] Figure 7 This is the RMSE line chart of butane concentration prediction of the TTPA-LSTM model under multiple parameters. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention more clear, the following will further describe the implementation mode of the present invention in detail with reference to the accompanying drawings. First, the basic theoretical knowledge involved in the present invention is introduced:

[0057] Taking three sampling rates as an example, Figure 1 The illustration of multi-sampling rate data is given in . The basic sampling interval is T, and there are k auxiliary variables x, of which the sampling intervals from the 1st to the nth are T, the sampling intervals from the n+1th to the kth are 2T, and the sampling interval of the quality variable y is 4T. During the online monitoring of multi-sampling rate industrial processes, the sampling loss of quality variables results in a low proportion of quality-related information in the data, which makes it difficult for the model to learn process characteristics and the modeling difficulty is high. Therefore, the task of soft measurement modeling of multi-sampling rate data is to overcome the impact of the low sampling rate of quality variables, so that the model can unify variables at different time scales according to the quality correlation, capture the detailed information related to the overall system in the low-frequency sampling data, and then accurately learn the characteristics of variables and processes.

[0058] In data-driven soft-sensing modeling, the LSTM network introduces a gate structure to optimize the temporal feature extraction process. The input gate that determines the amount of information to be absorbed, the forget gate that determines the amount of information to be discarded, and the output gate that selects the most important information for the current time step jointly control the cell state C from the previous moment. t-1 , hidden state h t-1 With the current external input x t The combined data flow operations integrate and provide output. The output expressions of each gate are as follows:

[0059] f t =σ(W f ·[h t-1 ,x t ]+b f ) (1)

[0060] i t =σ(W i ·[h t-1 ,x t ]+b i ) (2)

[0061] o t =σ(W o ·[h t-1 ,x t ]+b o ) (3)

[0062] Among them, f t 、i t and tare the outputs of the forget gate, input gate, and output gate respectively, σ is the sigmoid function, W f , W i , W o and b f 、b i 、b o They are the weights and bias values ​​corresponding to different gates respectively.

[0063] The calculation formula for memory cells is:

[0064]

[0065] Among them, W c is the weight of long-term information, b c is the bias value of the long-term information, and the new long-term information C is finally obtained. t and the current hidden information h t The calculation method is:

[0066]

[0067] h t =o t ⊙tanh(C t ) (6)

[0068] Embodiment 1

[0069] This embodiment provides a TTPA-LSTM soft measurement method based on multiple sampling rates, which solves the problem of soft measurement modeling of multiple sampling rate data with multivariate time series lag characteristics in actual industrial processes by constructing a TTPA-LSTM model; Figure 3 As shown in the figure, the model consists of two modules, namely the time perception module for extracting multi-sampling rate data features and the multivariate time series time lag feature optimization module. First, the time perception module is constructed, and the LSTM unit with time perception ability is used in combination with the sampling interval to integrate the data to improve the information loss caused by the sampling rate difference and obtain better hidden features; then the obtained information is input into the designed feature optimization module to capture the time lag information in the multivariate time series, and through the convolution filter and weighted summation calculation, the hidden output containing complete information is obtained.

[0070] The technical solution of the present invention is:

[0071] Step 1: Collect process variables and quality variables in actual industrial processes as sample data:

[0072] Analyze the process characteristics and select process variables that have a greater impact on the prediction of quality variables from the industrial process as the input variables of the model. The sampling rates of different process variables are inconsistent due to various factors, and the sampling frequency of quality variables is much lower than that of other variables. The sampled data set is recorded as [X, Y].

[0073] Step 2: Preprocess the sample data:

[0074] Analyze the sampling rate of each variable of the obtained data and record it, remove outliers, divide batches and time steps, etc. Specifically, it includes: first observe the missing values ​​of the obtained data, calculate the sampling rate of each variable according to the basic sampling interval; use the 3σ criterion to remove data points that are obviously not within a reasonable range. Secondly, according to the model training requirements, divide the data into appropriate batches and input them into the model to improve the modeling accuracy. Finally, analyze the time series relationship of the variables and expand the data dimension with a reasonable time step to obtain better prediction results.

[0075] Step 3: Divide the preprocessed dataset and build a time-aware model:

[0076] The preprocessed data is divided into training set and test set in a ratio of 4 to 1. The non-increasing heuristic decay function is calculated according to the sampling time interval of the data, and the time-aware model is constructed to reconstruct the data corresponding to the high and low sampling rates into short-term and long-term information respectively, and complete the feature extraction of multi-sampling rate data.

[0077] Step 4: Build the TTPA-LSTM model in combination with the time-aware model and set the parameters:

[0078] The row vector is used to extract the hidden features of the previous moment obtained by the time perception model, the number of convolution filters and the dimension of the output variable are designed, and the obtained features are transmitted to the convolution filter. The features of each moment are integrated according to the set time step and then passed to the scoring function to calculate the attention score of the hidden features at each moment. After the TTPA mechanism analyzes the interdependence between multiple variables in the previous time series, the results are finally input into the fully connected layer to obtain the predicted value and calculate the performance evaluation index.

[0079] Step 5: Optimize the parameters of the TTPA-LSTM model:

[0080] The Adam optimizer is used to update the model parameters by calculating the mean absolute error between the predicted value and the true value, and the result is back-propagated for recalculation. After an appropriate number of iterations, cross-validation is performed and the parameters are continued to be updated until the training loss no longer decreases, and the trained model is saved.

[0081] Step 6: Input the process variables corresponding to the quality variables to be predicted into the trained TTPA-LSTM model to obtain the predicted values:

[0082] Finally, the effectiveness and superiority of the model were verified by the soft measurement application of butane concentration at the bottom of the debutanizer industrial process. The experimental results show that the proposed model has a smaller error fluctuation and lower deviation value for the prediction of butane concentration at the bottom of the debutanizer dataset with multi-rate sampling and multivariate time series lag characteristics than other multi-rate sampling methods. The overall prediction effect is better than other methods and can better predict the dynamic trend of quality variables.

[0083] Furthermore, step 3 includes: constructing a time-aware module for extracting multi-sampling rate data features:

[0084] In the time perception module of the TTPA mechanism, LSTM is introduced to fully extract multi-sampling rate features and decompose and weight long-term and short-term information. When modeling multi-sampling rate data, the time between sequence elements in the LSTM network is evenly distributed. This way of reading long and short-term information is difficult to judge the importance of information based on the variable time interval, resulting in a small proportion of quality-related information in the extracted features. Especially when the sampling rate of the target data is unbalanced by multiple times, this shortcoming will be seriously magnified, which will have an adverse effect on the prediction results. Therefore, the concept of time perception is combined with the LSTM network, and the subspace decomposition method is used to achieve the effect of integrating time interval information into the LSTM. This interval is manifested in multi-sampling rate data as a sampling interval difference caused by different sampling rates.

[0085] like Figure 2 As shown, through the cell state C at the previous moment t-1 Get short-term memory And adjust it according to the weight to get the weighted short-term memory At the same time, C t-1 Get removed The long-term subspace component The final combination obtains the adjusted previous memory The detailed mathematical expression is as follows:

[0086]

[0087]

[0088]

[0089]

[0090] Among them, W d and b d are the weight and threshold of subspace decomposition, ⊙ is the multiplication operation of the corresponding elements of the matrix, g(Δ t) is a heuristic decay function used to convert the elapsed time into a weight, and the function is related to the time interval Δ t Since the time interval of multi-sampling rate data in industrial processes is fixed, the introduction of short-term memory and Δ t Combined with the trainable parameters of the model, the heuristic decay function g(Δ t ) is defined as:

[0091]

[0092] Where e is the natural logarithm and ε is less than 10 -5 Random variables, in the time perception module, for some information with lower importance in the previous moment, it is attenuated and weighted; the larger the sampling interval, the smaller the proportion of variables with lower sampling rates in the overall information. If some short-term information has little impact on the current result due to sampling rate and other reasons, it is proportionally adjusted according to the time span to reduce this impact. Therefore, the problem of information difference caused by different sampling rates is solved by a non-increasing heuristic decay function. After decomposing the input information, the short-term component is converted into corresponding weights according to the elapsed time and weighted, and the cell memory is used to approximate the learning time interval. Finally, the obtained short-term component is combined and adjusted with another component obtained by the previous decomposition, reducing the impact of short-term effects without losing long-term related information. It can simulate the effect of the downsampling method and extract multi-sampling rate data features without loss.

[0093] After such processing, the obtained information not only contains the original long and short-term memories, but also constructs a non-increasing heuristic decay function according to the multi-sampling rate time intervals to weight the short-term information. This can effectively deal with the data imbalance problems between process variables and between process variables and quality variables under multi-sampling rate conditions, and improve the multi-sampling rate data modeling effect of the model.

[0094] Further, step 4 includes: constructing a feature optimization module based on temporal pattern attention;

[0095] After processing the input information, the time perception module has completed the extraction of multi-sampling rate time interval information. When the model processes the time lag in the multivariate time series, it passes the hidden information obtained by the time perception module to the feature optimization module to construct the TTPA mechanism, thereby realizing the weighted summation operation of the multivariate vector across time steps to capture more sufficient time information. This module first integrates the hidden information h from the previous moment t-n , extract the row vector and input it to K convolution filters C j Operation, let the maximum time length be T, divide the historical information into m row vectors to enhance the feature extraction ability of the model, when the prediction contains w time lag data, the convolution result of the i-th row vector corresponding to the j-th filter is obtained The expression is:

[0096]

[0097] Among them, H i,(t-w-1+l) is the result of the convolution operation on the i-th row and the tw-1+l-th column, C j,T-w+l is the corresponding convolution filter;

[0098] To obtain the weighted sum V of the row vectors t , a score function f is defined to evaluate the convolution result H corresponding to each row vector i C Relevance:

[0099]

[0100] in, is the convolution result of the i-th row vector, W a Calculate the attention weight matrix for function f, h t Hide the information at the current moment and then calculate the attention score α i The value of , and the attention score is obtained through the sigmoid activation function:

[0101]

[0102] On this basis, the weighted sum V t The expression is:

[0103]

[0104] And hide the information h at the current moment t Combined to get the final output The expression is:

[0105]

[0106] Among them, W h , W v They are respectively t and V t The weights of each calculation.

[0107] Compared with the typical attention mechanism that only focuses on the processing of a single time step, the TTPA mechanism can select time series information that has a greater impact on the model's prediction results. Through a two-dimensional convolution filter, each time step in the sequence is bidirectionally filtered, hidden information at each moment is extracted across time steps, and the resulting vectors are weighted and summed to resolve the impact of interdependence in the time series, effectively obtaining the time lag information contained in the previous time steps, and selecting variables that have a greater impact on the prediction results. At the same time, by selecting information across time steps through convolution filters and attention mechanisms, it is possible to accurately determine that some hidden information in a previous moment has more predictive value. The features obtained in this way are more effective than traditional methods.

[0108] Aiming at the multi-sampling rate problem of multivariate time series with time lag in industrial process data, the present invention proposes a long short-term memory network soft-sensing model based on time-aware pattern attention. In terms of multi-sampling rate, the proposed model uses a non-increasing heuristic function related to the time interval in the time-aware module to weight the short-term memory, simulates the downsampling rate method while retaining more information, and solves the problem of difficult feature extraction caused by high and low sampling rates; on the other hand, the time pattern attention module is introduced to integrate the obtained features across time steps through a two-dimensional convolution filter, which solves the multivariate time series time lag problem, and at the same time improves the prediction ability of the model to solve such problems existing in actual industrial production.

[0109] Embodiment 2

[0110] This embodiment provides a method for predicting the butane concentration at the bottom of the debutanizer process. The method predicts the butane concentration at the bottom of the debutanizer process based on the soft measurement method given in Example 1; the experimental data comes from real-time sampling from the naphtha cracking process. The debutanizer is an important part of industrial petroleum refining and is mainly used in desulfurization and naphtha separation devices. The process flow is as follows: Figure 4 shown. Figure 4The seven process variables x1, x2, x3, x4, x5, x6 and x7 in the figure are the tower top temperature, tower top pressure, tower top reflux, tower top discharge, tray 6 temperature, first tower bottom temperature and second tower bottom temperature respectively. In this process, the condenser at the top of the tower affects the liquefaction of C5 (pentane) compounds and promotes their downward movement. On the contrary, the reboiler at the bottom promotes the volatilization of C4 (butane) compounds in gasoline and promotes their upward recycling. The process ultimately produces high-purity C3 (propane) compounds and C4 compounds from the top. The main process indicators that the process needs to achieve are to ensure the maximum separation of C3 and C4 compounds and to minimize the C4 concentration in the naphtha separator at the bottom of the tower. This concentration value cannot be measured directly and needs to be obtained through instrument online analysis. Therefore, accurate prediction of butane concentration at the bottom of the tower has important practical value for improving the control performance of the process. However, the industrial data set collected by the process has multiple sampling rates and complex multivariate time lag problems, which provides a suitable case for verifying the proposed model.

[0111] The seven process variables collected during the process are listed in Table 1. Due to technical and cost reasons, the sampling rates of each variable cannot be kept consistent. There are three different sampling rates in the data set used in the present invention, where the temperature and pressure variables are sampled at a rate of 10 minutes, the two flow variables are sampled at a rate of 20 minutes, and the mass variable bottom butane concentration is collected once at intervals of 40 minutes. Therefore, there are differences in the sampling rates between process variables and between process variables and quality variables, and it has significant multi-rate sampling characteristics. Due to the influence of the instrument measurement cycle, there is a lag of about 45 minutes to 90 minutes between each process variable and the quality variable, showing obvious time lag characteristics in multivariate time series, specifically manifested as a lag of 10-16 moments.

[0112] Table 1 Debutanizer variable names and their sampling rate descriptions

[0113]

[0114] The model prediction performance evaluation indicators are root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and correlation index (R2).

[0115] In order to verify the effectiveness of the model of the present invention on the debutanizer industrial process data set, the prediction effect of the TTPA-LSTM model is compared with that of various multi-sampling rate models on the validation set. The compared models include the multi-sampling rate data interpolation filling method (NI), the multi-rate sampling data fusion method (MSDF) based on CNN-LSTM, and the multi-layer perceptron model based on slow feature analysis (SFA-MLP); among them, the multi-sampling rate data interpolation filling method (NI) can be referred to the reference " D N.Numerical Hilbert transform algorithm for causal interpolation of functions represented by cubic and exponential splines[J].IEEE Access,2021,9:136702-136709.”; for the multi-rate sampling data fusion method based on CNN-LSTM, please refer to the relevant introduction in the literature “Zhang SM,Zhao C H.Slow-feature-analysis-based batch process monitoring with comprehensive interpretation of operation condition deviationand dynamic anomaly[J].IEEE Transactions on Industrial Electronics,2019,66(5):3773-3783.”; for the multi-layer perceptron model based on slow feature analysis, please refer to the introduction in the literature “Zheng JQ,Ma LW,Wu Y,et al.Nonlinear dynamic soft sensor development with a supervised hybrid CNN-LSTM network for industrial processes[J].ACS Omega,2022,7(19):16653-16664.”.

[0116] Table 2 Prediction evaluation indicators of multi-sampling rate model in debutanizer data set

[0117]

[0118] The comparison results of the model of the present invention with other multi-sampling rate methods on the debutanizer industrial data set are shown in Table 2. Compared with other models, the TTPA-LSTM model introduces a time perception mechanism, simulates the effect of the downsampling rate method and extracts more complete features, while capturing the multivariate time lag information in the data sample. The results show that the prediction results of the method of the present invention on the debutanizer data set with multi-rate sampling and multivariate time series lag characteristics are better than other multi-rate sampling methods for the bottom butane concentration, and the modeling ability is stronger.

[0119] In order to more intuitively observe the prediction effect of each model on the debutanizer industrial data set, Figure 5 The box plots of the prediction errors of each model for butane concentration at the bottom of the tower are given in Figure 6 The prediction curves of each model for the butane concentration at the bottom of the tower in the test set are given in Figure 6 a in the figure is the NI prediction curve; Figure 6 b in the figure is the MSDF prediction curve; Figure 6 c in the figure is the SFA-MLP prediction curve; Figure 6 d in is the prediction curve of TTPA-LSTM (the present invention). The more concentrated the columnar distribution in the error distribution diagram, the smaller the deviation between the predicted value and the true value, and the closer the prediction curve is to the true value, the more accurate the prediction result. The data interpolation method makes up for the information loss caused by multiple sampling rates on the overall level, but due to the limitations of the modeling method, the overall effect is poor; the deep data fusion method and the slow feature analysis method can only alleviate the difficulty of resolving the relationship between variables caused by multiple sampling rates to a certain extent due to the loss of feature extraction. Although the prediction accuracy is improved, there are obvious deviations in some intervals; the TTPA-LSTM model can comprehensively select multi-time hidden information that has a greater impact on the prediction results across time steps, and its error fluctuation is small and the deviation value is low. The overall prediction effect is better than other methods, and the dynamic trend of quality variables can be better predicted.

[0120] In order to further verify the effectiveness of each module of the model of the present invention, an ablation experiment was carried out to compare the LSTM model, TLSTM model, TPA-LSTM model, and TTPA-LSTM model respectively; the results obtained on the debutanizer data set are shown in Table 3.

[0121] Table 3 Ablation experiment prediction evaluation indicators

[0122]

[0123] The typical LSTM model itself has a certain time series modeling capability. After adding the time perception module, the TLSTM model has a significant improvement in the prediction effect of the debutanizer data set with three sampling rates. At the same time, the introduction of the TPA mechanism constitutes a TPA-LSTM model, which has the ability to handle multivariate time series cross-time step lags, and also improves the prediction performance to a certain extent. The prediction performance of the model of the present invention is the best. The above results verify the perception ability of the time perception module for multi-sampling rate information and the ability of the feature optimization module to extract information across time steps to deal with multivariate time series lags, prove the effectiveness of each module, and fully verify the superiority of the proposed model.

[0124] The model parameters have a great influence on the prediction performance. In order to further explore the influence of parameter selection on the RMSE of the butane concentration prediction results, each group only changed the value of one parameter, and the other parameters were cross-validated with the best parameters. The parameter sensitivity tests were performed on the learning rate, the number of hidden layer neurons, the number of convolution filters, and the number of iterations. The optimal values ​​of each parameter were 0.004, 32, 150, and 4, respectively. The results are as follows Figure 7 shown.

[0125] In summary, the TTPA-LSTM soft sensor model overcomes the problems of time lag and sampling rate difference between process variables, constructs a non-increasing heuristic decay function to weight short-term information, obtains the integrated characteristics of long-term and short-term information, reduces the impact of data missing caused by multiple sampling rates, introduces a feature optimization module to realize feature two-dimensional filtering, analyzes the time lag information in the multivariate time series across time steps, obtains more effective quality-related features, improves the accuracy and reliability of tower bottom butane concentration monitoring, and provides strong technical support for the optimal design of the debutanizer process control system.

[0126] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0127] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A TTPA-LSTM soft sensing method based on multiple sampling rates is used to predict quality variables in a specific industrial process, characterized in that: The method comprises: Step 1: Collect historical data in an actual industrial process as sample data, wherein the historical data includes process variables and corresponding quality variables; Step 2: Preprocess the sample data; Step 3: Divide the preprocessed dataset and build a time-aware model; Step 4: Build the TTPA-LSTM model in combination with the time-aware model and set parameters; Step 5: Optimize the parameters of the TTPA-LSTM model; Step 6: Input the process variables corresponding to the quality variables to be predicted into the trained TTPA-LSTM model to obtain the corresponding predicted values.

2. The method according to claim 1, characterized in that The construction of the time perception model in step 3 includes: introducing LSTM in the time perception module of the TTPA mechanism to fully extract multi-sampling rate features and decompose and weight long-term and short-term information; Through the cell state C at the previous moment t-1 Get short-term memory And adjust it according to the weight to get the weighted short-term memory At the same time, C t-1 Get removed The long-term subspace component The final combination obtains the adjusted previous memory The detailed mathematical expression is as follows: Among them, W d and b d are the weight and threshold of subspace decomposition, ⊙ is the multiplication operation of the corresponding elements of the matrix, g(Δ t ) is a heuristic decay function used to convert the elapsed time into a weight, and its expression is: Where e is the natural logarithm and ε is less than 10 -5 A random variable.

3. The method according to claim 2, characterized in that The step 4 comprises: Integrate hidden information h from previous moments t-n , extract the row vector and input it to K convolution filters C j Operation, let the maximum time length be T, divide the historical information into m row vectors to enhance the feature extraction ability of the model, when the prediction contains w time lag data, the convolution result of the i-th row vector corresponding to the j-th filter is obtained The expression is: Among them, H i,(t-w-1+l) is the result of the convolution operation on the i-th row and the (tw-1+l)-th column, C j,T-w+l is the corresponding convolution filter; Define the score function f to evaluate the convolution result corresponding to each row vector Relevance: in, is the convolution result of the i-th row vector, W a Calculate the attention weight matrix for function f, h t Hide information for the current moment; Then calculate the attention score α i The value of , and the attention score is obtained through the sigmoid activation function: On this basis, the weighted sum V t The expression is: And hide the information h at the current moment t Combined to get the final output The expression is: Among them, W h and W v They are respectively t and V t The weights of each calculation.

4. The method according to claim 3, characterized in that: The step 5 comprises: The Adam optimizer is used to update the model parameters by calculating the mean absolute error between the predicted value and the true value, and the result is back-propagated to calculate again, and the parameters are continued to be updated until the training loss no longer decreases, and the trained model is saved.

5. The method according to claim 4, characterized in that The preprocessing described in step 2 includes: normalization, feature selection, error processing and data transformation.

6. The method according to claim 5, characterized in that The ratio of the data set to be divided in step 3 is 4:1 for training set and testing set.

7. The method according to claim 6, characterized in that The industrial processes include petrochemicals, blast furnace ironmaking and fermentation processes.

8. A method for predicting butane concentration at the bottom of a debutanizer process, characterized in that: The method adopts the soft measurement method described in any one of claims 1 to 6 to predict the bottom butane concentration data.

9. The method according to claim 8, characterized in that The process variables are tower top temperature x1, tower top pressure x2, tower top reflux rate x3, tower top discharge rate x4, tower plate 6 temperature x5, first tower bottom temperature x6 and second tower bottom temperature x7; the quality variable is butane concentration at the bottom of the debutanizer tower.

Images