A multi-sampling rate-based TTPA-LSTM soft measurement method

Abstract

The application discloses a kind of TTPA-LSTM soft measurement methods based on multiple sampling rate, belong to modern industrial process soft measurement modeling and application field.The method includes: by constructing TTPA-LSTM model realizes accurate prediction to debutanizer industrial process tower bottom butane concentration.The application is through the non-decreasing heuristic function related to time interval in time perception module, short-term memory weighted processing, simulates the while keeping more information of the method of reducing sampling rate, solves the feature extraction difficult problem caused by high and low sampling rate;On the other hand, introduce time mode attention module, realize cross time step integration to the obtained feature by two-dimensional convolution filter, solve the multivariate time series time lag problem, improve the prediction ability of model, solve the problems existing in actual industrial production.

Description

TECHNICAL FIELD

[0001] The application relates to a multi-sampling rate-based TTPA-LSTM soft measurement method and belongs to the field of modern industrial process soft measurement modeling and application. BACKGROUND

[0002] In actual industrial processes, effectively obtaining quality variables directly related to production is the core of process control. However, due to the limitations of measurement technology and complex industrial environments, it is difficult to directly obtain these quality variables. To solve the problem of quality variable collection in complex environments, soft measurement technology, which indirectly obtains quality variables through easily measured process variables, has become a key to improving quality and efficiency in current manufacturing production. Soft measurement modeling methods are divided into two categories: mechanism modeling and data-driven modeling. Mechanism modeling strictly depends on the basic principles of physics, chemistry and other principles to build a model, requires considering multiple parameters, resulting in a complex model structure, and is difficult to adapt to the nonlinearity and time-varying nature of industrial processes, and the model has limited generalization ability. Data-driven modeling methods have the advantage of reducing the dependence on process knowledge, simplifying the modeling process, and being able to cope with uncertainty and randomness in actual industrial processes, and have developed rapidly in the field of industrial process modeling. Machine learning methods in data-driven modeling methods mainly include artificial neural networks, principal component regression, Gaussian process regression and extreme learning machine, etc. Among them, artificial neural network models are widely used due to their strong nonlinear learning ability.

[0003] However, with the increasing number of collected variables in industrial processes, influenced by factors such as process characteristics, distributed control, signal transmission and economy, it is difficult for each detection device to have a unified sampling period, resulting in a multi-sampling rate data, which poses higher requirements on the learning ability of deep models. Soft measurement methods for multi-sampling rate data are mainly divided into two categories: one is to preprocess the data through data preprocessing techniques, such as up-sampling and down-sampling, to organize the multi-sampling rate data outside the model before modeling; the other is to directly model through a specific model to reduce the impact of information loss caused by multi-sampling rate. Data preprocessing techniques usually use data interpolation (Numerical Interpolation, NI) and other methods to fill in the missing data, or use down-sampling methods to reduce the overall data to a low sampling rate. However, due to problems such as large data volume, poor data quality, limited computing resources and aliasing, data preprocessing methods are difficult to effectively address the characteristics of multi-sampling rate, while direct modeling methods usually use specific methods combined with neural networks to build related models to solve the problem of modeling multi-sampling rate data. On the other hand, the time series data set of industrial processes has different sampling rates for each variable, and also contains time lag characteristics. The time lag characteristics in multi-element time series (hereinafter referred to as multi-element time series time lag) make it difficult for time-aware models without cross-time step capability to analyze this special time sequence characteristic in the data, thereby affecting the modeling results.

[0004] The present application is directed to the problem of multi-sampling rate data modeling in industrial processes, while considering different time lags in multivariate time series, a time-aware temporal pattern attention (TTPA) mechanism is proposed, and a new type of soft measurement model is built combining long short-term memory (LSTM) network. SUMMARY

[0005] The present application is directed to the problem that the existing soft measurement technology does not fully consider the multi-sampling rate data modeling and different time lags in multivariate time series, resulting in reduced modeling quality and unsuitable models, a TTPA mechanism is proposed, and a TTPA-LSTM soft measurement method based on multi-sampling rate is provided, which is based on the TTPA mechanism and the LSTM network, and applied to the soft measurement modeling of actual industrial processes, realizing the accurate prediction of key quality variables.

[0006] The technical scheme of the present application is:

[0007] Step 1: Collecting process variables and quality variables in actual industrial processes as sample data:

[0008] Analyzing process characteristics, selecting process variables that have a greater impact on quality variable prediction from industrial processes as input variables of the model, the sampling rates of different process variables affected by multiple factors are inconsistent, and the sampling frequency of the quality variable is much lower than that of other variables, and the data set obtained by sampling is recorded as [X, Y].

[0009] Step 2: Preprocessing of sample data:

[0010] Analyze the sampling rate of each variable of the obtained data and record it, remove outliers, divide batches and time steps, etc. Specifically, first observe the missing values of the obtained data, calculate the sampling rate of each variable according to the basic sampling interval; remove data points that are obviously not within a reasonable range using the 3σ criterion. Secondly, according to the model training requirements, divide the data into appropriate batches to input into the model to improve the modeling accuracy. Finally, analyze the time sequence relationship of the variables, and expand the data dimension with a reasonable time step to obtain better prediction results.

[0011] Step 3: Dividing the preprocessed data set and constructing a time-aware model:

[0012] The preprocessed data is divided into training set and test set according to the ratio of 4 to 1. According to the sampling time interval of the data, a non-increasing heuristic decay function is calculated, a time-aware model is constructed, the data corresponding to high and low sampling rates are respectively reconstructed into short-term and long-term information, and the feature extraction of multi-sampling rate data is completed.

[0013] Step 4: Constructing TTPA-LSTM model combined with time-aware model and setting parameters:

[0014] The row vector is used to extract the hidden features of the previous time obtained by the time-aware model. The number of convolution filters and the dimension of the output variable are designed, and the obtained features are transmitted to the convolution filter. According to the set time step, the features of each time are integrated and then transmitted to the scoring function to calculate the attention score of the hidden features at each time. After analyzing the mutual dependence between multiple variables in the previous time sequence through the TTPA mechanism, the result is finally input into the fully connected layer to obtain the prediction value and calculate the performance evaluation index.

[0015] Step 5: Optimizing the parameters of TTPA-LSTM model:

[0016] The Adam optimizer is used to update the parameters of the model by calculating the mean absolute error between the predicted value and the true value, and the result is backpropagated to calculate again, continue to update the parameters until the training loss no longer decreases, and the trained model is saved.

[0017] Step 6: Input the process variables corresponding to the quality variables to be predicted into the trained TTPA-LSTM model to obtain the prediction value:

[0018] Finally, the effectiveness and superiority of the model are verified through the application of the depropanizer industrial process bottom butane concentration soft measurement. The experimental results show that compared with other multi-rate sampling methods, the prediction results of the proposed model on the depropanizer data set with multi-rate sampling and multi-element time series time delay characteristics have small error fluctuations and low bias values, and the overall prediction effect is better than other methods, which can better predict the dynamic trend of the quality variable.

[0019] Further, step 3 includes: constructing a time-aware module for extracting multi-sampling rate data features:

[0020] In the time-aware module of the TTPA mechanism, LSTM is introduced to fully extract multi-sampling rate features and decompose and weight long and short-term information. When modeling multi-sampling rate data, the sequence elements in the LSTM network are uniformly distributed in time, which makes it difficult to judge the importance of information according to the variable time interval, resulting in a small proportion of quality-related information in the extracted features, especially when the sampling rate of the target data is multiple and uneven, this shortcoming will be severely magnified, which will adversely affect the prediction results. Therefore, the time-aware concept is combined with the LSTM network, and the subspace decomposition method is used to achieve the effect of integrating time interval information into the LSTM, and this interval is the difference in sampling interval caused by different sampling rates in multi-sampling rate data.

[0021] In the module calculation process, the cell state C t-1 is obtained , and the short-term memory component is obtained by adjusting it according to the weight t-1 . At the same time, the long-term subspace component is obtained by removing the short-term memory component from C d . Finally, the adjusted previous memory is obtained by combining the short-term memory component and the long-term subspace component . The detailed mathematical expressions are as follows:

[0022]

[0023]

[0024]

[0025]

[0026] where W d and b t are the weight and threshold of subspace decomposition respectively, ⊙ is the matrix corresponding element multiplication operation, g(Δ t ) is a heuristic decay function used to convert the elapsed time into a weight, and the mathematical expression of the function is combined with the time interval Δ t . Since the time interval of the multi-sampling rate data of the industrial process is fixed, the short-term memory is combined with Δ t as a trainable parameter of the model, so the heuristic decay function g(Δ -5 ) in the TTPA mechanism is defined as:

[0027]

[0028] where e is the natural logarithm, and ε is a random variable less than 10 -5 . In the time-aware module, the less important part of the information in the previous moment is decayed and weighted; the larger the sampling interval, the smaller the proportion of the variable with lower sampling rate in the overall information. If part of the short-term information has little effect on the current result due to the sampling rate, etc., the proportion is adjusted according to the time span to reduce the effect, so the non-increasing heuristic decay function solves the information difference problem caused by different sampling rates. After decomposing the input information, the short-term component is converted into a corresponding weight according to the elapsed time and weighted, the cell memory approximation learns the time interval, and finally the obtained short-term component is combined and adjusted with another component obtained by previous decomposition, which can simulate the effect of the down-sampling method and extract the features of the multi-sampling rate data without loss.

[0029] After being processed in this way, the obtained information not only contains the original long short-term memory, but also is weighted by a non-increasing heuristic decay function according to the multi-sampling rate time interval, which can effectively deal with the data imbalance problem between process variables and between process variables and quality variables under the condition of multi-sampling rate, and improve the multi-sampling rate data modeling effect of the model.

[0030] Further, step 4 comprises constructing a feature optimization module based on time pattern attention;

[0031] After processing the input information, the time perception module has completed the extraction of multi-sampling rate time interval information. When the model processes the time lag in the multivariate time series, the hidden information obtained by the time perception module is transmitted to the feature optimization module to construct the TTPA mechanism, so as to realize weighted summation operation on the multivariate vector across time steps, so as to capture more sufficient time information. This module first integrates the hidden information h t-n from the previous moment, extracts a row vector and inputs it into K convolution filters C j , let the maximum time length be T, and divide the historical information into m row vectors to enhance the feature extraction capability of the model. When predicting data containing w time lags, the convolution result of the i-th row vector corresponding to the j-th filter is obtained The expression of V is:

[0032]

[0033] Among them, H i,(t-w-1+l) is the result generated by the i-th row and t-w-1+l-th column of convolution operation, C j,T-w+l is the corresponding convolution filter;

[0034] In order to obtain the weighted sum V t of the row vector, a score function f is defined to evaluate the correlation between each row vector and the corresponding convolution result :

[0035]

[0036] Among them, is the convolution result of the i-th row vector, W a is the weight matrix of the function f to calculate attention, h t is the hidden information of the current moment;

[0037] Then the value of attention score a i is calculated, and the attention score is obtained through the sigmoid activation function:

[0038]

[0039] On this basis, the weighted sum V tThe expression of the formula is:

[0040]

[0041] And the current moment hidden information h t The final output is obtained by combination The expression of the formula is:

[0042]

[0043] Wherein, W h , W v Respectively, each h t And V t Respectively calculated weight.

[0044] Compared with the typical attention mechanism, the TTPA mechanism can select the time sequence information that has a greater impact on the model prediction result. By bidirectional filtering of each time step in the sequence through a two-dimensional convolution filter, the hidden information at each time step is extracted across time steps, and the obtained vector is weighted and summed to solve the influence of mutual dependence in the time sequence, effectively obtaining the time lag information contained in the previous time steps, and selecting the variable that has a greater impact on the prediction result. At the same time, by selecting information across time steps through the convolution filter and the attention mechanism, it can accurately judge that part of the hidden information at a certain time in the past has more prediction value, and the features obtained by this method are more effective than traditional methods.

[0045] In summary, the present application aims at the modeling problem of multi-sampling rate data in industrial processes, and considers the different time lags in multivariate time series, and proposes a TTPA mechanism, and builds a new soft measurement model combined with LSTM. The model uses a time perception module to complete the subspace decomposition of the input information, constructs a non-increasing heuristic function to convert the time information into a weight, and realizes the feature extraction of the multi-sampling rate data through the operation of the long short-term memory, which simulates the effect of reducing the sampling rate while losslessly obtaining information, and improves the modeling ability of the multi-sampling rate data. Further, the feature optimization module is used to process the obtained time sequence features, and the time lag information between variables in each time step is selected, the hidden information that has a greater impact on the prediction result of the current moment is extracted across time steps, and the new global features are obtained after the scoring mechanism optimization, thereby improving the prediction accuracy of the model.

[0046] The beneficial effects of the present application are:

[0047] Aiming at the multi-sampling rate problem of multi-element time series time lag in industrial process data, the application provides a TTPA-LSTM soft measurement method based on multi-sampling rate; the TTPA-LSTM model is constructed to solve the problems of multi-sampling rate data modeling in industrial processes and the reduction of modeling quality and the inapplicability of the model caused by different time lags in multi-element time series; in terms of multi-sampling rate, the time interval related non-decreasing heuristic function in the time perception module is used to weight the short-term memory, which simulates the downsampling method while retaining more information, and solves the feature extraction difficulty problem caused by high and low sampling rates; on the other hand, the time pattern attention module is introduced, the two-dimensional convolution filter is used to realize the cross-time step integration of the obtained features, the multi-element time series time lag problem is solved, and the prediction ability of the model is improved. The modeling of the de-butanizing tower industrial process simulation case solves the problems existing in actual industrial production. Compared with the representative method of existing multi-sampling rate modeling, the application has higher prediction accuracy for industrial processes with different sampling rates and multi-element time series time lag, and verifies the superiority and effectiveness of the model prediction ability. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 The application is used for multi-sampling rate data in industrial processes.

[0050] Figure 2 The application is used for the multi-sampling rate feature processing flowchart of the time perception module.

[0051] Figure 3 The application provides a TTPA-LSTM model structure diagram.

[0052] Figure 4 The application is used for the de-butanizing tower process flowchart.

[0053] Figure 5 The application is used for the box plot of the prediction error of butane concentration of each multi-sampling rate model.

[0054] Figure 6 The application is used for the curve diagram of the prediction result of butane concentration at the bottom of the TTPA-LSTM model tower.

[0055] Figure 7 The application is used for the RMSE line graph of butane concentration prediction of the TTPA-LSTM model under multiple parameters. DETAILED DESCRIPTION

[0056] To make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the drawings. First, the basic theoretical knowledge related to the present application is introduced:

[0057] Taking three sampling rates as examples, the illustration of multi-sampling rate data is given in Figure 1 . The basic sampling interval is T, and the auxiliary variable x has k in total, of which the first to n sampling intervals are T, the n+1 to k sampling intervals are 2T, and the sampling interval of the quality variable y is 4T. In the online monitoring of the multi-sampling rate industrial process, the missing sampling of the quality variable makes the proportion of quality-related information in the data low, which leads to the difficulty of the model to learn the process characteristics and the high difficulty of modeling. Therefore, the task of soft-sensing modeling of multi-sampling rate data is how to overcome the influence brought by the low sampling rate of the quality variable, so that the model can unify the variables at different time scales according to the quality correlation, capture the detailed information related to the whole system in the low-frequency sampling data, and then accurately learn the characteristics of the variables and the process.

[0058] In data-driven soft-sensing modeling, the LSTM network introduces a gating structure to optimize the time series feature extraction process. The input gate that determines the amount of information to be absorbed, the forget gate that determines the amount of information to be discarded, and the output gate that selects the most important information at the current time step jointly operate on the data stream combined with the cell state C t-1 from the previous moment and the hidden state h t-1 and the current external input x t to provide the output. The output expressions of the gates are as follows:

[0059] f t =σ(W f ·[h t-1 ,x t ]+b f ) (1)

[0060] i t =σ(W i ·[h t-1 ,x t ]+b i ) (2)

[0061] o t =σ(W o ·[h t-1 ,x t ]+b o ) (3)

[0062] where f t , i t and o tare the outputs of the forget gate, input gate and output gate respectively, σ is a sigmoid function, W f , W i , W o and b f , b i , b o are the weights and bias values corresponding to different gates respectively.

[0063] The calculation formula of the memory cell is:

[0064]

[0065] wherein, W c is the weight of long-term information, b c is the bias value of long-term information, and the calculation mode of the finally obtained completely new long-term information C t and the current time hidden information h t is:

[0066]

[0067] h t = o t ⊙tanh(C t ) (6)

[0068] Embodiment one

[0069] The embodiment provides a kind of TTPA-LSTM soft measurement method based on multiple sampling rate, and the soft measurement modeling problem of multiple sampling rate data with multivariate time series time lag characteristics in actual industrial process is solved by constructing TTPA-LSTM model;As shown in the figure, Figure 3 The model is composed of two modules, which are time perception module for extracting multiple sampling rate data features and multivariate time series time lag feature optimization module respectively. First, build the time perception module, integrate the data by using the LSTM unit with time perception ability combined with sampling interval, to improve the information loss caused by sampling rate difference, and obtain better hidden features;Then the obtained information is input into the designed feature optimization module, the time lag information in multivariate time series is captured, and the hidden output containing complete information is obtained through convolution filter and weighted summation calculation.

[0070] The technical scheme of the application is:

[0071] Step 1: collect process variables and quality variables in actual industrial process as sample data:

[0072] Process characteristics are analyzed, process variables that have a greater impact on the prediction of quality variables are selected from industrial processes as input variables of the model, the sampling rates of different process variables are influenced by various factors and are inconsistent, and the sampling frequency of the quality variable is much lower than that of other variables. The data set obtained by sampling is denoted as [X, Y].

[0073] Step 2: Preprocessing of sample data:

[0074] The sampling rates of the variables of the obtained data are analyzed and recorded, and the work of removing outliers, dividing batches and time steps, etc. is carried out. Specifically, first, observe the missing values of the obtained data, calculate the sampling rate of each variable according to the basic sampling interval; remove the data points that are obviously not within the reasonable range by using the 3σ criterion. Secondly, according to the model training requirements, divide the data into appropriate batches to input into the model to improve the modeling accuracy. Finally, analyze the time sequence relationship of the variables, and expand the data dimension with a reasonable time step to obtain better prediction results.

[0075] Step 3: Divide the preprocessed data set and construct a time-aware model:

[0076] The preprocessed data is divided into training set and test set according to the ratio of 4 to 1. According to the sampling time interval of the data, a non-increasing heuristic decay function is calculated, a time-aware model is constructed, the data corresponding to high and low sampling rates are respectively reconstructed into short-term and long-term information, and the feature extraction of multi-sampling rate data is completed.

[0077] Step 4: Construct TTPA-LSTM model combined with time-aware model and set parameters:

[0078] The previous time hidden features obtained by the time-aware model are extracted by using row vectors, the number of convolution filters and the dimension of output variables are designed, and the obtained features are transmitted to the convolution filters. According to the set time step, the features at each time are integrated and then transmitted to the scoring function to calculate the attention score of the hidden features at each time. After analyzing the mutual dependence relationship between multiple variables in the previous time sequence by the TTPA mechanism, the final result is input into the fully connected layer to obtain the prediction value and calculate the performance evaluation index.

[0079] Step 5: Optimize the parameters of TTPA-LSTM model:

[0080] The Adam optimizer is used to update the parameters of the model by calculating the mean absolute error between the predicted value and the true value, and the result is backpropagated to calculate again. After a suitable number of iterations, cross-validation is performed and the parameters are continuously updated until the training loss no longer decreases, and the trained model is saved.

[0081] Step 6: Input the process variables corresponding to the quality variable to be predicted into the trained TTPA-LSTM model to obtain the prediction value:

[0082] Finally, the effectiveness and superiority of the model were verified by the application of the bottom butane concentration soft measurement of the debutanizer industrial process column. The experimental results show that compared with other multi-rate sampling methods, the prediction results of the proposed model on the debutanizer data set with multi-rate sampling and multi-element time series delay characteristics have small error fluctuations and low bias values, and the overall prediction effect is better than that of other methods, which can better predict the dynamic trend of the quality variable.

[0083] Further, step 3 comprises: constructing a time-aware module for extracting multi-sampling rate data features:

[0084] In the time-aware module of the TTPA mechanism, LSTM is introduced to fully extract multi-sampling rate features and decompose and weight long-term and short-term information. When modeling multi-sampling rate data, the time interval between sequence elements in the LSTM network is uniformly distributed. This way of reading long-term and short-term information is difficult to judge the importance of information according to the time interval of the variable, resulting in a small proportion of quality-related information in the extracted features, especially when the sampling rate of the target data is multiple and uneven, this shortcoming will be seriously magnified, which will adversely affect the prediction results. Therefore, the time-aware concept is combined with the LSTM network, and a subspace decomposition method is used to achieve the effect of fusing time interval information into the LSTM. This interval is the sampling interval difference caused by different sampling rates in multi-sampling rate data.

[0085] As shown in t-1 Figure 2 , the short-term memory component is obtained from the cell state C t-1 at the previous moment , and is adjusted according to the weight to obtain the weighted short-term memory . At the same time, the long-term subspace component is obtained from C t-1 , and the long-term memory is obtained by removing . The detailed mathematical expressions are as follows:

[0086]

[0087]

[0088]

[0089]

[0090] where W d and b d are the weight and threshold of subspace decomposition, respectively, is the matrix multiplication operation of corresponding elements, and g(Δ t) is a heuristic decay function used to convert the elapsed time into a weight, and the function is related to the time interval Δ t Since the time interval of multi-sampling rate data in industrial processes is fixed, the introduction of short-term memory and Δ t Combined with the trainable parameters of the model, the heuristic decay function g(Δ t ) is defined as:

[0091]

[0092] Where e is the natural logarithm and ε is less than 10 -5 In the time perception module, the less important information from previous moments is attenuated and weighted. The larger the sampling interval, the smaller the proportion of variables with lower sampling rates in the overall information. If some short-term information has little impact on the current result due to factors such as the sampling rate, it is proportionally adjusted based on the time span to reduce this impact. Therefore, a non-increasing heuristic decay function is used to address the information discrepancy caused by different sampling rates. After decomposing the input information, the short-term components are converted into corresponding weights based on the elapsed time and weighted. The learning time interval is approximated using cellular memory. Finally, the resulting short-term component is combined and adjusted with another component obtained from the previous decomposition. This reduces the impact of short-term effects while not losing long-term related information. This method can simulate the effects of downsampling methods and extract features of multi-sampling rate data without loss.

[0093] After such processing, the obtained information not only contains the original long-term and short-term memories, but also constructs a non-increasing heuristic attenuation function according to the multi-sampling rate time interval to weight the short-term information. This can effectively deal with the data imbalance problem between process variables and between process variables and quality variables under multi-sampling rate conditions, and improve the multi-sampling rate data modeling effect of the model.

[0094] Furthermore, step 4 includes: constructing a feature optimization module based on temporal pattern attention;

[0095] After processing the input information, the time perception module has completed the extraction of multi-sampling rate time interval information. When the model processes the time lag in the multivariate time series, it passes the hidden information obtained by the time perception module to the feature optimization module to construct the TTPA mechanism, thereby performing weighted summation operations on the multivariate vectors across time steps to capture more comprehensive time information. This module first integrates the hidden information h from the previous moment t-n , extract the row vector and input it to K convolution filters C j Operation, let the maximum time length be T, divide the historical information into m row vectors to enhance the feature extraction ability of the model. When predicting data containing w time lags, the convolution result of the i-th row vector corresponding to the j-th filter is obtained. The expression is:

[0096]

[0097] where H i,(t-w-1+l) is the result of the convolution operation on the i-th row and t-w-1+l-th column, C j,T-w+l is the corresponding convolution filter;

[0098] To obtain the weighted sum V t of the row vectors, a score function f is defined to evaluate the convolution result H i C Correlation:

[0099]

[0100] where, is the convolution result of the i-th row vector, W a is the weight matrix of the attention calculated by the function f, h t is the hidden information at the current time, and then the value of the attention score a i is obtained by a sigmoid activation function to obtain the attention score:

[0101]

[0102] On this basis, the expression of the weighted sum V t is:

[0103]

[0104] and combined with the hidden information h t at the current time, the expression of the final output is:

[0105]

[0106] where W h , W v are the respective weights calculated by h t and V t respectively.

[0107] Compared with the processing method of focusing on a single time step by a typical attention mechanism, the TTPA mechanism can select time sequence information that has a greater impact on the model prediction result. By bidirectional filtering of each time step in the sequence through a two-dimensional convolution filter, hidden information at each time is extracted across time steps, and the resulting vector is weighted and summed to address the influence of mutual dependence in the time sequence, effectively obtaining time lag information contained in previous time steps and selecting variables that have a greater impact on the prediction result. At the same time, by selecting information across time steps through the convolution filter and the attention mechanism, it can accurately determine that part of the hidden information at a certain time in the past has more predictive value. The features obtained in this way are more effective than traditional methods.

[0108] The present application is directed to the problem of multi-sampling rate of multi-element time sequence time lag in industrial process data, and proposes a long short-term memory network soft measurement model based on time-aware mode attention. In terms of multi-sampling rate, the model simulates the downsampling method while retaining more information by weighting the short-term memory through the non-decreasing heuristic function related to the time interval in the time-aware module, and solves the problem of feature extraction difficulty caused by high and low sampling rates; on the other hand, a time pattern attention module is introduced, and a two-dimensional convolution filter is used to integrate the obtained features across time steps, solving the problem of multi-element time sequence time lag and improving the prediction ability of the model, solving such problems existing in actual industrial production.

[0109] Embodiment Two

[0110] The present embodiment provides a method for predicting the concentration of butane at the bottom of a debutanizing tower, which is based on the soft measurement method given in Embodiment One to predict the concentration of butane at the bottom of a debutanizing tower in the process; the experimental data comes from real-time sampling from a naphtha cracking process. The debutanizing tower is an important part of industrial petroleum refining, mainly used in desulfurization and naphtha separation devices, and the process flow is as shown in Figure 4 Figure 4 ​The seven process variables x1, x2, x3, x4, x5, x6 and x7 in the table 1 are the top temperature, the top pressure, the top reflux flow, the top outflow, the 6th tray temperature, the first bottom temperature and the second bottom temperature, respectively. In this process, the condenser at the top of the column affects the liquefaction of C5 (Pentane) compounds, promoting their downward movement. Conversely, the reboiler at the bottom of the column catalyzes the volatilization of C4 (Butane) compounds in gasoline, promoting their upward recirculation. The process ultimately produces high-purity C3 (Propane) compounds and C4 compounds from the top, and the main process indicators that the process needs to achieve are to ensure the maximum separation of C3 compounds and C4 compounds, and to minimize the C4 concentration in the naphtha separator at the bottom of the column. This concentration value cannot be directly measured and needs to be obtained through online instrument analysis. Therefore, accurate prediction of the bottom butane concentration has important practical value for improving the control performance of the process. However, the collected industrial data set of the process has the problems of multiple sampling rates and complex multivariate time delays, which provides a suitable case for verifying the proposed model.

[0111] The seven process variables collected in the process are listed in table 1. Due to technical and cost reasons, the sampling rates of the variables cannot be kept consistent. In the data set used in the present application, there are three different sampling rates. The temperature and pressure variables are sampled at a rate of 10 minutes, the two flow variables are sampled at a rate of 20 minutes, and the mass variable of the bottom butane concentration is collected once every 40 minutes. Therefore, there are differences in sampling rates between the process variables and between the process variables and the mass variables, which have significant multi-rate sampling characteristics. Due to the influence of instrument measurement period, there is a lag of about 45-90 minutes between each process variable and the mass variable, which shows obvious time delay characteristics in the multivariate time series, and there are 10-16 time lags.

[0112] Table 1 Variable name and sampling rate description of debutanizer

[0113]

[0114] The evaluation indexes of the model prediction performance are root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and correlation index (R2).

[0115] In order to verify the effectiveness of the proposed model on the debutanizer industrial process data set, the prediction effects of the TTPA-LSTM model and various multi-sampling rate models were compared on the validation set. The compared models include the multi-sampling rate data interpolation filling method (NI), the multi-rate sampling data fusion method (MSDF) based on CNN-LSTM, and the multi-layer perceptron model based on slow feature analysis (SFA-MLP). The multi-sampling rate data interpolation filling method (NI) can be found in the reference " D N.Numerical Hilbert transform algorithm for causal interpolation of functions represented by cubic and exponential splines[J].IEEE Access,2021,9:136702-136709.”; for the multi-rate sampling data fusion method based on CNN-LSTM, please refer to the relevant introduction in the literature “Zhang SM,Zhao C H.Slow-feature-analysis-based batch process monitoring with comprehensive interpretation of operation condition deviationand dynamic anomaly[J].IEEE Transactions on Industrial Electronics,2019,66(5):3773-3783.”; for the multi-layer perceptron model based on slow feature analysis, please refer to the introduction in the literature “Zheng JQ,Ma LW,Wu Y,et al.Nonlinear dynamic soft sensor development with a supervised hybrid CNN-LSTM network for industrial processes[J].ACS Omega,2022,7(19):16653-16664.”.

[0116] Table 2 Prediction evaluation indicators of the multi-sampling rate model on the debutanizer data set

[0117]

[0118] The comparison results of the model of the present application with other multi-sampling rate methods on the debutanizer industrial data set are shown in Table 2. Compared with other models, the TTPA-LSTM model introduces a time perception mechanism, simulates the effect of the down-sampling rate method, extracts more complete features, and simultaneously captures the multi-element time lag information in the data sample. The results show that the present application method is better than other multi-rate sampling methods in predicting the bottom butane concentration of the debutanizer data set with multi-rate sampling and multi-element time series time lag characteristics, and has stronger modeling capability.

[0119] In order to more intuitively observe the prediction effect of each model on the debutanizer industrial data set, Figure 5 the box plot of the prediction error of the bottom butane concentration of each model is given in Table 2, Figure 6 the prediction result curve of the bottom butane concentration of each model in the test set is given in Table 3, Figure 6 a in Table 3 is the NI prediction curve graph; Figure 6 b in Table 3 is the MSDF prediction curve graph; Figure 6 c in Table 3 is the SFA-MLP prediction curve graph; Figure 6 d in Table 3 is the TTPA-LSTM (the present application) prediction curve graph. The more concentrated the column distribution in the error distribution graph, the smaller the deviation of the predicted value from the true value, and the more accurate the prediction result. The data interpolation method compensates for the information loss caused by multi-sampling rate at the overall level, but due to the limitations of the modeling method, the comprehensive effect is poor; the deep data fusion method and the slow feature analysis method can only alleviate the difficulty of analyzing the relationship between variables caused by multi-sampling rate to a certain extent due to the loss of feature extraction method, although the prediction accuracy is improved, there is obvious deviation in some intervals; the TTPA-LSTM model can comprehensively select the hidden information at multiple time points that have greater impact on the prediction result, and has small error fluctuation and low deviation value, so the overall prediction effect is better than that of other methods, and the dynamic trend of the quality variable can be better predicted.

[0120] In order to further verify the effectiveness of each module of the model of the present application, an ablation experiment is performed, and the LSTM model, the TLSTM model, the TPA-LSTM model and the TTPA-LSTM model are compared; the results obtained on the debutanizer data set are shown in Table 3.

[0121] Table 3 Prediction evaluation index of ablation experiment

[0122]

[0123] The typical LSTM model has certain time sequence modeling capability, and after the time perception module is added, the prediction effect of the TLSTM model on the debutanizer data set with three sampling rates is significantly improved, the TPA mechanism is introduced to form a TPA-LSTM model, the TPA-LSTM model has a multi-time sequence cross-time step time lag processing capability, and the prediction performance is also improved to a certain extent, and the model of the present application has the best prediction performance. The above results verify the perception ability of the time perception module to multi-sampling rate information and the ability of the feature optimization module to extract information across time steps to cope with multi-time sequence lag, prove the effectiveness of each module, and fully verify the superiority of the proposed model.

[0124] The model parameters have a great influence on the prediction performance, in order to further explore the influence of parameter selection on the RMSE of the butane concentration prediction result, only one parameter value is changed in each group, and the remaining parameters are taken as the best parameters for cross-validation, and parameter sensitivity tests are performed on the learning rate, the number of hidden layer neurons, the number of convolution filters and the number of iterations, and the optimal values of the parameters are 0.004, 32, 150 and 4 respectively, and the results are as shown in Figure 7

[0125] In summary, the TTPA-LSTM soft measurement model overcomes the time lag and sampling rate difference problems between process variables, constructs a non-increasing heuristic decay function to weight short-term information, obtains long and short-term information integration features, reduces the influence of data missing caused by multi-sampling rate, introduces a feature optimization module to realize two-dimensional filtering of features, analyzes time lag information in multi-time sequence across time steps, obtains more effective quality-related features, improves the accuracy and reliability of the bottom butane concentration monitoring, and provides strong technical support for the optimization design of the debutanizer process control system.

[0126] Some steps in the embodiments of the present application can be implemented by software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0127] The above only describes the preferred embodiments of the present application, and is not intended to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.​

Claims

1. A multi-sampling rate based TTPA-LSTM soft-sensing method for realizing the prediction of quality variables in specific industrial processes, characterized in that, The method comprises: Step 1: collecting historical data in an actual industrial process as sample data, wherein the historical data comprises process variables and corresponding quality variables; Step 2: pre-processing the sample data; Step 3: dividing the pre-processed data set and constructing a time-aware model; Step 4: combining the time-aware model to construct a TTPA-LSTM model and setting parameters; Step 5: optimizing the parameters of the TTPA-LSTM model; Step 6: inputting process variables corresponding to a quality variable to be predicted into the trained TTPA-LSTM model to obtain a corresponding predicted value; The construction of the time-aware model in step 3 comprises: in the time-aware module of the TTPA mechanism, introducing an LSTM to sufficiently extract multi-sampling rate features and decompose and weight long-term and short-term information; By the cell state of the last time Get short-term memory component And adjust it according to the weight to get the weighted short-term memory At the same time, through Get the long-term subspace component after Remove Finally, the adjusted previous memory is obtained by combining The detailed mathematical expression is as follows: wherein, and are a weight and a threshold for the subspace decomposition, respectively, is a matrix multiplication operation of corresponding elements of matrices, is a heuristic decay function used to convert elapsed time into a weight, expressed as: wherein, is the natural logarithm, is a random variable less than 10 -5 . The step 4 comprises: Integrate hidden information from previous moments , extract the row vector and input it into K convolution filters Operation, let the maximum time length be T, divide the historical information into m row vectors to enhance the feature extraction ability of the model. w When the data is lagged by one moment, we get the i The row vector corresponds to the j The convolution result of the filters The expression is: wherein, is a convolution operation i is a result of a convolution operation is a result of a convolution operation is a corresponding convolution filter; Defining a score function to evaluate the convolution result corresponding to each row vector Correlation: wherein, is the convolution result of the i th row vector, is a function calculates the weight matrix of attention, is the hidden information at the current moment; The attention score is then calculated The value of the attention score is obtained by a sigmoid activation function: On this basis, the weighted sum is calculated The expression for this is: and the current time hidden information combining to get the final output The expression is: wherein, and are respective and calculated weights.

2. The method of claim 1, wherein, The step 5 comprises: An Adam optimizer is adopted to realize parameter updating of the model by calculating the mean absolute error between the predicted value and the true value, and the result is back propagated to be calculated again and the parameters are continuously updated until the training loss no longer decreases, and the model obtained by training is saved.

3. The method of claim 2, wherein, The pre-processing in step 2 comprises normalization processing, feature selection, error processing and data transformation.

4. The method of claim 3, wherein, The proportion of the data set division in step 3 is that the training set: test set is 4:

1.

5. The method of claim 4, wherein, The industrial process comprises petroleum chemical industry, blast furnace ironmaking and fermentation process.

6. A depropanizer process column bottom butane concentration prediction method characterized by, The method adopts the soft measurement method in any one of claims 1-4 to predict the tower bottom butane concentration data.

7. The method of claim 6, wherein, The process variable is overhead temperature x 1. Overhead pressure x 2. Overhead reflux rate x 3. Overhead draw rate x 4. Tray 6 temperature x 5. First bottoms temperature x 6. Second bottoms temperature x 7. The mass variable is debutanizer bottoms butane concentration.

Images