A soft sensing method based on weighted gated stacked quality supervised VAE
By adding quality-related information and gated linear units to the VAE network and integrating different levels of features, the feature extraction and prediction accuracy problems of VAE network in soft measurement are solved, and higher prediction accuracy and feature correlation are achieved.
Patent Information
- Application Number
- CN202410531675.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-29
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-04-29
AI Technical Summary
The existing soft measurement technology based on VAE network failed to fully extract the features related to the quality variables and failed to effectively integrate feature information at different levels, resulting in inaccurate model prediction.
Weighted gated stacking quality supervision VAE (W-GSTVAE) method is used to add quality-related information in the pre-training stage, and integrate feature information at different levels through gated linear units to improve the prediction accuracy of the model.
Effectively extract features more relevant to the target, improve the prediction accuracy of the model, and can reasonably compress features when facing high-dimensional data, enhancing the performance of the model.
Smart Images

Figure CN118444641B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a soft measurement method based on weighted gated stacking quality supervision VAE, belonging to the technical field of industrial process control. Background Art
[0002] In the industrial production process, in order to ensure the control quality, some key quality variables need to be measured in real time. However, due to factors such as extreme environmental weather, expensive instrument costs, sampling and analysis delays, it is very difficult to monitor key quality variables in real time online. Therefore, as a substitute for hardware equipment, soft measurement technology came into being. By establishing a mathematical model between input process variables and output quality variables, soft sensors are constructed to achieve real-time online monitoring of quality variables. Generally speaking, soft measurement technology can be roughly divided into two categories, mechanism models and data-driven models. Among them, the mechanism model requires detailed prior knowledge of the production process. However, modern industrial processes are often complex and strongly nonlinear, and it is difficult to obtain a detailed description of the process. The data-driven model does not rely on the mechanism, but is constructed based on the historical data of the industrial production process, which is more flexible and practical. In addition, with the widespread application of distributed control systems, a large amount of sensor data can be collected, which also makes the application of soft sensors (soft sensors use existing physical sensors to measure process variables, and then use algorithms / computational models to estimate quality variables that are difficult to measure directly) possible. Therefore, the application of soft sensors in industrial processes has received widespread attention. Commonly used data-driven soft sensing methods include principal component regression analysis (PCR), partial least squares (PLS), support vector machine (SVM), artificial neural network (ANN), etc.
[0003] In recent years, deep learning (DL) methods developed from artificial neural networks (ANN) have made great progress in application fields such as speech recognition, natural language processing, image recognition, and target detection. Unlike traditional shallow methods, DL can learn high-level feature representations of data through deep network structures. In addition, due to the multi-layer nonlinear mapping, the extracted representations have multiple levels of abstraction. These features can often achieve better results when used for classification or regression tasks, and have powerful data mining and nonlinear modeling capabilities. Due to the widespread existence of complexity and nonlinearity in today's industrial scenarios, DL is very suitable for the field of soft measurement. Nowadays, more and more DL methods are introduced into soft measurement modeling. Among these DL methods, stacked autoencoder (SAE) is one of the most widely used networks. A single-layer AE is actually a three-layer feedforward neural network model that uses an unsupervised learning algorithm, and its purpose is to reconstruct the original input through hidden feature representations. In order to adapt to different industrial scenarios, SAE has many variants. For example, Yuan X et al. proposed a variable weighted stacked autoencoder that can deeply mine output-related features and extend it to a semi-supervised soft measurement model; Zhang X et al. used gated recurrent units and encoder-decoder networks to propose a transferable deep feature extractor and applied it to soft measurement; Tian Y et al. introduced an extension layer in the stacked autoencoder and added the original information and extended information of each layer to the subsequent layer to enhance the utilization of each layer's features. However, when it comes to complex industrial processes, almost all the collected process data will be affected by random noise. Due to the existence of this random disturbance, the above encoder-based methods become difficult to handle effectively. The variational autoencoder (VAE) proposed by Kingma DP and Welling M solves the problem of the impact of random noise. VAE is a deep generative model that combines the concepts of deep learning and Bayesian variational inference and is widely used in image recognition and natural language processing. As a probabilistic framework model based on a deep network, VAE has both the data mining and nonlinear modeling capabilities of deep learning, and can also model process uncertainty and data noise like a probabilistic model. In recent years, VAE has been introduced into industrial processes and gradually applied to process monitoring and soft sensor modeling.
[0004] Like autoencoders, VAE has the ability to reduce data dimension and extract features. When used as feature encoding, it can extract more effective information than ordinary autoencoders. Although a single-layer VAE has a certain feature extraction capability, it cannot effectively reduce the dimension of data or select features when facing strongly nonlinear process data, because the interaction between features in strongly nonlinear data is more complex, so it cannot extract useful high-level feature representations. Based on this problem, Shen et al. proposed a nonlinear probabilistic latent variable regression model, including a pre-training stage and a fine-tuning stage; in the pre-training stage, deep nonlinear features are extracted through multi-layer VAE, and in the fine-tuning stage, the model is further trained according to specific regression tasks. In order to ensure the integrity of information during feature extraction, the number of hidden layer neurons of each VAE in the pre-training stage is set to the same. In this case, although relatively complete information can be retained, it is impossible to extract compressed high-level features to capture different levels of feature representations in the data when facing high-dimensional data. However, if only the hidden layer of the deep VAE is compressed for feature extraction, since unsupervised learning is used in the pre-training stage, the network cannot guarantee the correlation between the extracted features and the quality variables, which may cause the extracted features to contain information that is useless for modeling. How to reasonably integrate quality-related information into the training process of each layer of VAE while compressing high-dimensional data, so as to extract features that are more relevant to the target and improve the prediction accuracy of the model is a key issue. In addition, the fine-tuning stage often only uses the last layer of features for prediction, resulting in the neglect of useful information in shallow features. However, considering that feature information at different levels contributes to the final prediction results to different degrees, how to integrate feature information at different levels according to the degree of contribution to further improve the prediction accuracy of the model is also a key issue. Summary of the invention
[0005] In order to solve the problem that the existing soft measurement technology based on VAE network does not fully consider the extraction of features related to quality variables and fails to fully utilize the feature representation of different levels of the network, resulting in inaccurate model prediction. The present invention provides a soft measurement modeling method of weighted gated stacked quality supervised VAE (W-GSTVAE) with VAE as the baseline network. In the modeling process, by adding quality-related information during the pre-training process, the model is enhanced to extract target-related features, and the gated linear unit is used to reasonably integrate the features of each layer in the deep network. A new modeling method different from the existing soft measurement modeling method is designed, which greatly improves the prediction accuracy of the soft measurement model.
[0006] A method for predicting butane concentration at the bottom of a debutanizer based on weighted gated stacked quality supervised VAE, comprising:
[0007] Step 1: Select the input variables of the model (i.e., the process variables required for prediction) based on theoretical analysis and experience, collect data, including input variables and quality variables, and standardize the data.
[0008] The labeled training dataset is denoted as Where x is the input variable, y is the quality variable, N l is the number of samples. The standardized set of input variables in the training data set is denoted as X s , the sample to be predicted is recorded as X new ;
[0009] Step 2: Build a stacked quality supervised VAE (STVAE) network model, extract high-level features of the data through multiple layers of VAE, and add the prediction error of the quality variable to the reconstruction error of VAE to ensure the correlation between the extracted features and the quality variable. In this way, the features extracted in each layer are not only the feature representation of the input variable, but also an effective explanation of the quality variable.
[0010] Step 3: Add the variable weighting part to the STVAE network to obtain the weighted stacked quality supervised VAE (W-STVAE). When the input variables are reconstructed, the maximum information coefficient (MIC) between each input variable and the quality variable is calculated, and different weights are assigned to each input variable to comprehensively measure the correlation between the input variable and the quality variable, and further extract features related to the target;
[0011] Step 4: Use gated linear units to integrate feature information at different levels to obtain the final weighted gated stacked quality supervised VAE (W-GSTVAE) soft measurement model.
[0012] Step 5: X s As the input of the W-GSTVAE model, the reconstruction value of the input sample and the predicted target value are calculated for pre-training;
[0013] 1) X s As the input of the W-GSTVAE encoder, the sample features in the latent space are calculated to obey the distribution The mean and variance of
[0014] 2) Use the reparameterization technique to calculate the mean and variance Sampling to obtain hidden variable z;
[0015] 3) Use the hidden variable z as the input of the decoder to obtain the reconstructed value of the sample And the predicted value of the sample
[0016] Step 6: Repeat Step 5 to calculate the loss function LOSS of each layer of pre-training p , the Adam optimization method is used to update the model parameters;
[0017] Step 7: After pre-training, use the hidden layer feature set Z to calculate the gate control value and the linear mapping y instead of the original output k , and get the final output form Fine-tune the global network by minimizing the loss function minΓ. Save the network parameters for online prediction.
[0018] Step 8: The sample X to be predicted new After the same standardization, the data is input into the trained model to obtain the final predicted value.
[0019] The STVAE network in Step 2 is composed of multiple layers of TVAE. Unlike traditional AE, VAE has both deep learning data mining and nonlinear modeling capabilities, and can also model process uncertainty and data noise like a probabilistic model, which can greatly improve the model's ability in probabilistic data description and feature extraction. Assume that the input variable x and the quality variable y are generated by random continuous hidden variables z.
[0020]
[0021]
[0022] Assuming that x and y are conditionally independent of each other, the latent variable z is sampled from the latent space and is passed through two parameters δ x and δ y The neural network is used to obtain x and y. According to the above generation process, the joint probability distribution of the generation model is:
[0023] p δ (x,y,z)=p δx (x|z)p δy (y|z)p(z)
[0024] The log-likelihood function of the marginal probability distribution p(x,y) of the data sample point is:
[0025]
[0026] in To infer the model, an additional variational posterior is used to approximate the true complex posterior probability p(z|x).
[0027] According to Jensen's inequality:
[0028]
[0029] is the lower bound of the marginal probability likelihood function. In order to calculate and δ, maximizing the evidence lower bound:
[0030]
[0031] The lower bound of evidence can be understood in two parts. The first two items are related to the generative model p of x and y. δx (x|z), p δy (y|z), where z follows the approximate variational posterior Assumptions Obeys normal distribution. And because the generative models all obey normal distribution, logp δx (x|z) and logp δy (y|z) can be expressed as:
[0032]
[0033]
[0034] For real-valued samples, these two terms represent the mean squared error between x and y. is the approximate variational posterior and the KL divergence between the prior p(z). Because is a normal distribution, and p(z) is a standard normal distribution, so the term can be written as:
[0035]
[0036] The mean and variance in the formula are obtained by two encoders respectively, n z is the dimension of the latent variable z.
[0037] The loss function of quality supervised VAE pre-training is negative For real-valued data, it can be expressed as:
[0038]
[0039] Where N l is the number of training samples, l is the dimension of the input variable; λ is a weight coefficient used to balance the weight of reconstruction loss and prediction loss; x n represents the input variable of the first layer of pre-training, Represents the reconstructed value of the first layer input variable of pre-training; y n represents the true quality variable, represents the predicted value of the quality variable; σ i(n) represents the variance of the latent space, μ i(n)Represents the mean of the latent space. Because the loss function involves information related to the target value, labeled data is required for supervised training.
[0040] After training the parameters in the network by minimizing the loss function, they can be directly stacked into the deep neural network of STVAE, and the low-level features output by the previous TVAE are used as the input of the next TVAE. The loss function of the kth (k>1) TVAE is:
[0041]
[0042] where z k(n) and is the input and reconstructed output of the kth TVAE; y k(n) represents the true quality variable of the kth layer, represents the predicted value of the k-th layer quality variable; n z(k) is the hidden variable dimension of the kth TVAE.
[0043] In the Step 3, MIC can comprehensively analyze the linear or nonlinear relationship between variables. If there is a relationship between two variables, a grid G can be drawn on a scatter plot with these two variables as the X-axis and Y-axis to divide the data to encapsulate this relationship. There is a data set D containing n samples of two variables. According to the above method, the data set is divided into a and b parts along the X-axis and Y-axis using the grid G. The probability distribution corresponding to each point on the grid G is represented by D| G Indicates that mutual information (MI) can be calculated under this grid division. Different grid divisions will produce different probability distributions and mutual information values. The maximum MI value obtained by dividing the grid G is defined as:
[0044] I(D,a,b)=max I(D| G )
[0045] The MI values under different grids are standardized to obtain modified values between 0 and 1 to ensure fair comparison, so the MIC can be defined as:
[0046]
[0047] Where B is a function of the sample size n, usually set to n 0.6 MIC equilibrium covers all functional relationships and can fully reflect the relationship between response variables.
[0048] In order to further enhance the correlation between features and target values, features highly correlated with output prediction are extracted, and the MIC between input variables and quality variables is calculated during reconstruction. More weight is assigned to variables related to output, while less attention is given to variables with low correlation with output. l The labeled training dataset D l , the importance of a variable is determined by its MIC with the target variable. The weight of the d-th dimension input variable is calculated as follows:
[0049]
[0050] Therefore, the loss function of the proposed weighted quality supervised VAE (W-STVAE) pre-training can be defined as:
[0051]
[0052] For the kth layer, the pre-training loss function is:
[0053]
[0054] Among them l k is the dimension of the input variable of the kth layer, λ d(k) is the weight coefficient of the kth layer.
[0055] The structure of the gated linear unit in Step 4 can be described as:
[0056] M g =W g ·z+b g
[0057]
[0058] M=W·z+b
[0059]
[0060] in Represents the Hadamard product, and z is the hidden layer vector. The gated linear unit linearly maps the input vector and applies the sigmoid function to obtain the gating value. These gating values act as switches, determining how much of the linear mapping information of the input vector will be retained for use when integrating information from different layers. These gating values are generated by the input vector and then applied to the input vector. Therefore, the gated linear unit has strong adaptability and can dynamically adjust the transmission of information according to the input data. Assuming that the depth of the network is n, after the pre-training is completed, the feature set Z of each hidden layer can be obtained, which represents the features from the first layer to the last layer. For the kth layer, first use the features to calculate a gating value:
[0061]
[0062] Then use feature z k Compute the linear map y that replaces the original output k :
[0063] y k =W·z k +b k
[0064] Finally, integrate all linear mappings to get the final output form:
[0065]
[0066] In this way, the gated linear unit is used to calculate the gated value to determine the weight of each layer of features on the final output. Finally, the parameters in the network are fine-tuned by the BP back propagation algorithm, which can be defined as minimizing the following unconstrained optimization problem:
[0067]
[0068] in represents the predicted value of the quality variable of the nth training set sample, and y(n) represents the true quality variable of the nth training set sample. It is equivalent to minimizing the mean square error (MSE) between the predicted value and the actual target value.
[0069] Beneficial effects of the present invention:
[0070] (1) The present invention designs a method for predicting butane concentration at the bottom of a debutanizer tower based on weighted gated stacked quality-supervised VAE. Compared with traditional methods, this method can better model process uncertainty and data noise, and extract features that are more relevant to the output target variable, thereby improving the prediction accuracy of the model.
[0071] (2) The present invention uses multi-layer VAE to perform advanced feature extraction, which can compress high-dimensional data and obtain feature representations at different levels.
[0072] (3) In order to ensure the integrity of the extracted compressed high-level features and their correlation with the quality variables, the present invention adds the prediction error of the target value and the maximum information coefficient (MIC) between the input variable and the quality variable to the reconstruction error of VAE to guide the feature learning process, thereby improving the prediction accuracy of the model.
[0073] (4) The present invention uses gated linear units to integrate feature information at different levels. Although deeper hidden layers tend to extract more useful high-level features, each hidden layer is a different feature representation of the input variable. The shallower the layer, the more input information is retained. Conversely, the deeper the layer, the stronger the correlation with the output. In the traditional pre-training modeling process, the impact of the shallow layer on the output is often ignored, resulting in the loss of some effective information. To this end, gated linear units are used to make full use of the feature representations of different layers. At the same time, the contribution of each layer of features to the final prediction results can be quantified, further integrating the information of each layer to improve the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0075] Figure 1 A flow chart of a soft measurement method based on weighted gated stacked quality supervision VAE provided by the present invention;
[0076] Figure 2 The process flow chart of the debutanizer tower involved in the present invention is as follows;
[0077] Figure 3 It is a schematic diagram of the quality supervision VAE part in the W-GSTVAE model involved in the present invention;
[0078] Figure 4 It is a schematic diagram of the gated linear unit part in the W-GSTVAE model involved in the present invention;
[0079] Figure 5 The overall structure diagram of the W-GSTVAE model constructed by the present invention;
[0080] Figure 6 Prediction fitting curve diagram of the four models provided by the present invention on the debutanizer industrial data set;
[0081] Figure 7 Box plots of the predicted absolute errors of the four models provided by the present invention. DETAILED DESCRIPTION
[0082] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0083] Embodiment 1:
[0084] This embodiment provides a soft sensing method based on weighted gated stacked quality supervision VAE, which is used to predict quality variables in an industrial process. The method includes:
[0085] Step 1: Collect historical data of specific industrial processes;
[0086] Soft sensor modeling includes two parts: offline modeling and online prediction. The variables of the model (i.e., process variables in a specific industrial process) are selected based on theoretical analysis and experience. Then data collection is performed to obtain a historical data set, which is divided into a training set and a test set, and the data is standardized.
[0087] The labeled training dataset is Where x is the process variable in the industrial process, which is used as the input variable of the model, y is the quality variable, which is used as the output variable of the model, l is the dimension of the input process variable, and N l is the number of samples. The standardized set of input variables in the training data set is denoted as X s , the sample to be predicted is recorded as X new ;
[0088] Step 2: Build a stacked quality supervised VAE (STVAE) network to extract high-level features of the data through multiple layers of VAE. Add the prediction error of the quality variable to the reconstruction error of VAE to ensure the correlation between the extracted features and the quality variable. In this way, the features extracted at each layer are not only the feature representation of the input variable, but also an effective explanation of the quality variable.
[0089] Step 3: Add the weighted part to the STVAE network to obtain the weighted stacked quality supervised VAE (W-STVAE) network.
[0090] Specifically, when the input variables are reconstructed, the maximum information coefficient (MIC) between the input variables and the quality variables is calculated, and different weights are assigned to each input variable to comprehensively measure the correlation between the input variables and the quality variables, and further extract features related to the target;
[0091] Step 4: Use gated linear units to integrate feature information at different levels to obtain the final weighted gated stacked quality supervised VAE (W-GSTVAE) soft sensor model;
[0092] Step 5: X s As the input of the W-GSTVAE model, the reconstruction value of the input sample and the predicted target value are calculated for pre-training;
[0093] 1) X s As the input of the W-GSTVAE encoder, the sample features in the latent space are calculated to obey the distribution The mean and variance of
[0094] 2) Use the reparameterization technique to calculate the mean and variance Sample the hidden variable z;
[0095] 3) Use the hidden variable z as the input of the decoder to obtain the reconstructed value of the sample And the predicted value of the sample
[0096] Step 6: Repeat Step 5 to calculate the loss function LOSS of each layer of pre-training p , the Adam optimization method is used to update the model parameters;
[0097] Step 7: After pre-training, use the hidden layer feature set Z to calculate the gate control value and the linear mapping y instead of the original output k , and get the final output form Fine-tune the global network by minimizing the loss function minΓ. Save the network parameters for online prediction.
[0098] Step 8: The sample X to be predicted new After the same standardization, the data is input into the trained model to obtain the final predicted value.
[0099] Embodiment 2
[0100] This embodiment provides a method for predicting butane concentration in a debutanizer based on weighted gated stack quality supervision VAE (W-GSTVAE), see Figure 1 , the method comprising:
[0101] Step 1: Collect input and output data to form a historical training sample database.
[0102] This example uses a public data set of a real debutanizer industrial process. Figure 2 The debutanizer unit is mainly composed of a heat exchanger, a condenser, a reboiler, a reflux pump, a separator and an accumulator.
[0103] Table 1 shows the seven variables measured by the sensor, i.e., process variables. Since the main purpose of the debutanizer is to control the butane content to a minimum, it is necessary to measure the butane concentration at the bottom of the tower in real time. However, in practice, the butane content is obtained by sampling and analysis by a gas chromatograph, and there is a large measurement delay in this process. In addition, there are complex nonlinear relationships between process variables. The data set includes a total of 7 input variables and 1 quality variable, the quality variable being the butane concentration at the bottom of the debutanizer, with a total of 2394 sample data.
[0104] Considering the process dynamics, 13-dimensional augmented data variables are obtained based on expert knowledge and physical analysis:
[0105] [u1(k),u2(k),u3(k),u4(k),u5(k),u5(k-1),u5(k-2),u5(k-3),(u6(k)+u7(k)) / 2,y(k-1),y(k-2),y(k-3),y(k-4)] T
[0106] Table 1 Process variables of debutanizer process
[0107]
[0108] Step 3: After augmenting the original sample data set, 2390 sample data are finally obtained. The data is standardized and the standardized data set is divided into a training set and a test set. The first 1000 samples are used as the training data set, and the remaining samples are used as the test data set to build and train the W-GSTVAE network. The standardized set of input variables in the training data set is X s , the sample to be predicted is X new .
[0109] Step 4: Stacked Quality Supervised VAE (STVAE) design.
[0110] Considering that only compressing the number of hidden neurons in the deep layer of stacked VAE can effectively process high-dimensional data, it may lead to loss of effective information in the features. Therefore, quality supervision information is added to guide feature learning during the training process of each layer of VAE to enhance the correlation between features and quality variables and improve the prediction accuracy of the model. The structure of quality supervision VAE (TVAE) can be found in Figure 3 Assuming that the input variable x and the output target value y in each layer of VAE are generated by random continuous hidden variables z, the generation process can be expressed as:
[0111]
[0112]
[0113] Assuming that x and y are conditionally independent of each other, the latent variable z is sampled from the latent space and is passed through two parameters δ x and δ y The neural network is used to obtain x and y. According to the above generation process, the joint probability distribution of the generation model is:
[0114] p δ (x,y,z)=p δx (x|z)p δy (y|z)p(z)
[0115] The log-likelihood function of the marginal probability distribution p(x,y) of the data sample point is:
[0116]
[0117] in To infer the model, an additional variational posterior is used to approximate the true complex posterior probability p(z|x). According to Jensen's inequality:
[0118]
[0119] is the lower bound of the marginal probability likelihood function. In order to calculate and δ, maximizing the evidence lower bound:
[0120]
[0121] The lower bound of evidence can be understood in two parts. The first two items are related to the generative model p of x and y. δx (x|z), p δy (y|z), where z follows the approximate variational posterior Assumptions Obeys normal distribution. And because the generative models all obey normal distribution, logp δx (x|z) and logp δy (y|z) can be expressed as:
[0122]
[0123]
[0124] For real-valued samples, these two terms represent the mean squared error between x and y. is the approximate variational posterior and the KL divergence between the prior p(z). Because is a normal distribution, and p(z) is a standard normal distribution, so the term can be written as:
[0125]
[0126] The mean and variance in the formula are obtained by two encoders respectively, n z is the dimension of the latent variable z.
[0127] The loss function of TVAE pre-training is negative For real-valued data, it can be expressed as:
[0128]
[0129] Where Nl is the number of training samples, l is the dimension of the input variable; λ is a weight coefficient used to balance the weight of reconstruction loss and prediction loss; x n represents the input variable of the first layer of pre-training, Represents the reconstructed value of the first layer input variable of pre-training; y n represents the true quality variable, represents the predicted value of the quality variable; σ i(n) represents the variance of the latent space, μ i(n) Represents the mean of the latent space
[0130] After training the parameters in the network by minimizing the loss function, they can be directly stacked into the deep neural network of STVAE, and the low-level features output by the previous TVAE are used as the input of the next TVAE. The loss function of the kth (k>1) TVAE is:
[0131]
[0132] where z k(n) and is the input and reconstructed output of the kth TVAE; y k(n) represents the true quality variable of the kth layer, represents the predicted value of the k-th layer quality variable; n z(k) is the hidden variable dimension of the kth TVAE.
[0133] Step 5: Weighted Stacked Quality Supervised VAE (W-STAVE) design.
[0134] In order to further enhance the correlation between features and target values, features that are highly correlated with output prediction are extracted. The maximum information coefficient (MIC) between input variables and quality variables is calculated during reconstruction. More weight is assigned to variables that are related to the output, while less attention is given to variables that are less correlated with the output. The weight calculation for the d-th dimension input variable is as follows
[0135]
[0136] Therefore, the loss function of the first layer of weighted quality supervised VAE (W-STVAE) pre-training is:
[0137]
[0138] For the kth layer, the pre-training loss function is:
[0139]
[0140] Among them l k is the dimension of the input variable of the kth layer, λd(k) is the weight coefficient of the kth layer.
[0141] Step 6: Design of weighted gated stacked quality supervised VAE (W-GSTVAE). The gating mechanism is introduced into W-STVAE to integrate the output features of each layer and make full use of the feature information of each layer to improve the effective utilization of shallow features. Figure 4 As shown, its structure can be described as:
[0142] M g =W g ·z+b g
[0143]
[0144] M=W·z+b
[0145]
[0146] in Represents the Hadamard product, z is the hidden layer vector, and the gated linear unit linearly maps the input vector and applies the sigmoid function to obtain the gated value. These gated values act as switches, determining how much of the linear mapping information of the input vector will be retained for use when integrating information from different layers. These gated values are generated by the input vector and then applied to the input vector. Therefore, the gated linear unit has strong adaptability and can dynamically adjust the transmission of information according to the input data. The overall structure of the W-GSTVAE model is shown in Figure 5 shown.
[0147] Step 7: Place the X s As the input of the W-GSTVAE model, the reconstruction value of the input sample and the predicted target value are calculated for pre-training;
[0148] 1) X s As the input of the W-GSTVAE encoder, the sample features in the latent space are calculated to obey the distribution The mean and variance of
[0149] 2) Use the reparameterization technique to calculate the mean and variance Sample the hidden variable z;
[0150] 3) Use the hidden variable z as the input of the decoder to obtain the reconstructed value of the sample And the predicted value of the sample
[0151] Step 8: Repeat step 7 to calculate the loss function LOSS of each layer of pre-training pThe network structure is determined by trial and error. The W-GSTVAE structure is [13 10 7 4], and the Adam optimization method is used to update the model parameters. The number of training rounds in the pre-training stage is set to 300, the batch size is set to 50, and the learning rate is set to 0.01. In the training process of each layer, the input will first perform feature encoding to calculate the weight value, and then assign the weight value to the reconstruction of the input variable in the loss function. Finally, the quality information is added to the pre-training loss function.
[0152] Step 9: After pre-training, use the hidden layer feature set Z to calculate the gate control value and the linear mapping y instead of the original output k , and get the final output form Fine-tune the global by minimizing the loss function minΓ. For the kth layer, first use the feature to calculate a gating value:
[0153]
[0154] Then use feature z k Compute the linear map y that replaces the original output k :
[0155] y k =W·z k +b k
[0156] Finally, integrate all linear mappings to get the final output form:
[0157]
[0158] In this way, the gated linear unit is used to calculate the gated value to determine the weight of each layer of features on the final output. Finally, the parameters in the network are fine-tuned by the BP back propagation algorithm, which can be defined as minimizing the following unconstrained optimization problem:
[0159]
[0160] in represents the predicted value of the quality variable of the nth training set sample, and y(n) represents the true quality variable of the nth training set sample. It is equivalent to minimizing the mean square error (MSE) between the predicted value and the actual target value. The number of training rounds in the fine-tuning phase is set to 100, the batch size is set to 20, and the learning rate is set to 0.01. Save the network parameters for online prediction.
[0161] Step 10: The sample to be predicted X new After the same standardization, the data is input into the trained model to obtain the final predicted value.
[0162] Aiming at the problem of how to extract quality-related features and improve the model prediction performance while reducing the dimension of complex industrial data, the present invention proposes an industrial soft sensor modeling method based on gated stacked quality supervised VAE. First, stacked VAE is used to enhance the feature extraction capability. In the pre-training stage, the prediction error of the quality variable is added to the loss function and the maximum information coefficient (MIC) between the input variable and the quality variable is calculated to guide the feature learning process and improve the correlation between the features extracted by the deep network and the quality variable. Secondly, in the fine-tuning stage, the output features of each layer are integrated using gated linear units to make full use of the feature information of each layer. At the same time, the contribution of each layer feature to the final prediction result can be quantified, the effective utilization rate of shallow features is improved, and the prediction accuracy of the model is improved.
[0163] In order to compare the superiority of the W-GSTVAE proposed in this application, it is compared with the SAE and STVAE methods. Among them, SAE is a commonly used deep learning soft sensor modeling method (for details, please refer to the introduction in "Yuan X, Ou C, Wang Y, et al. A novel semi-supervised pre-training strategy for deep networks and its application for quality variable prediction in industrial processes [J]. Chemical Engineering Science, 2020, 217: 115509."). STVAE is an improved method based on VAE in the above derivation process.
[0164] To ensure the rationality of the comparison, the hidden layer structures of these models are consistent with the W-GSTVAE model. In addition, a comparison is made with the nonlinear probabilistic latent variable regression model (H-NPLVR) based on the VAE method (for details, please refer to the introduction in "Shen, Bingbing, LeYao, and Zhiqiang Ge." Nonlinear probabilistic latent variable regression models for soft sensor application: From shallow to deep structure." Control Engineering Practice 94 (2020): 104198.") to prove the advantages of the model proposed in this application. The deep hidden layer neurons of the H-NPLVR method are the same number. For comparison, the H-NPLVR model structure is set to [13 10 10 10]. At the same time, this application uses the following two parameters as the prediction model performance evaluation indicators in each prediction method, and their calculation formulas are as follows:
[0165] (1) Root mean square error (RMSE):
[0166]
[0167] Where N is the number of test set samples, and y(n) is the true output value of the nth sample in the test set. is the predicted value of the nth sample in the test set. RMSE focuses on the prediction error of the sample. The smaller the RMSE, the better the model performance.
[0168] (2) Coefficient of Determination (R 2 ):
[0169]
[0170] in It is the average value of the true output value of all samples in the test set. 2 It represents the square correlation between the true output value and the predicted value of the test set, reflecting the ability of the model to explain the variance of the output data. 2 The closer it is to 1, the better the model performance.
[0171] All experiments in the embodiments of the present application were performed on the same computer: i5-12400@2.50GHz CPU processor, 16GB memory, Pytorch1.11 under 64-bit Windows11 operating system.
[0172] The prediction results of butane concentration by SAE, H-NPLVR, STVAE and W-GSTVAE are as follows: Figure 6 The detailed experimental comparison results of the test set are shown in Table 2. The evaluation indicators are RMSE and R 2 .
[0173] According to Table 2, it can be found that the proposed W-GSTVAE has achieved the best performance among the four models. Although the traditional SAE can generally capture the changing trend of butane concentration, the prediction effect for peaks and troughs in the data is relatively poor. The prediction effect of H-NPLVR and STVAE at peaks has been greatly improved compared with SAE. This is because VAE can extract more effective information than traditional autoencoders when used as feature encoding. H-NPLVR sets the deep VAE neurons to the same number to ensure the integrity of information in the feature extraction process; STVAE guides the model to extract features that are more relevant to the target value through quality supervision. In terms of prediction performance, H-NPLVR is very close to STVAE, which shows that setting the same hidden layer neurons and quality supervision can improve the performance of modeling. The prediction performance of W-GSTVAE has been significantly improved compared with the first three methods. Both peaks and troughs can be accurately predicted, and the degree of fit with the true value curve is high, which means that the model proposed in this application has stronger fitting ability and higher prediction accuracy. The weighted part further extracts high-level features related to quality variables by analyzing the correlation between different variables, while the gated linear unit can integrate the feature information of each layer to improve the predictive ability of the model.
[0174] Table 2 Evaluation indicators of four methods on DC dataset
[0175]
[0176] exist Figure 7 It can be clearly seen from the error box plot that the proposed W-GSTVAE model has a narrower error range than the other three methods, which has obvious advantages. W-GSTVAE pays more attention to those samples that cause larger prediction errors, rather than just those samples that account for a larger proportion of the distribution. It can more accurately predict the butane concentration, which once again proves the superiority of the proposed method. From the perspective of industrial processes, it is very important to predict the butane content as accurately as possible when the butane content is relatively high, because this helps to detect abnormalities in the process state in a timely manner.
[0177] The present invention discloses a soft sensor modeling method based on weighted gated stacked quality supervision VAE, and uses it in the prediction of butane concentration at the bottom of a debutanizer. The present application scheme first uses stacked VAE to enhance the feature extraction capability of VAE, and considers the prediction error of the target value and the maximum information coefficient (MIC) between the input variable and the quality variable in the pre-training loss function, so as to extract features that are more relevant to the quality variable. In this way, high-dimensional data can be compressed and complete and useful modeling information can be retained. Then, the output features of each layer are integrated using a gated linear unit to enhance the prediction accuracy of the model. Finally, the superiority of the proposed method is verified on an actual industrial data set of a debutanizer. It can provide strong technical support for the butane concentration monitoring of the debutanizer.
[0178] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.
[0179] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A soft sensing method based on weighted gated stacked quality supervision VAE for predicting quality variables in industrial processes, characterized in that: The method comprises: Step 1, collecting historical data in a specific industrial process, wherein the historical data includes process variables and corresponding quality variables; Step 2: construct a stacked quality supervision VAE network STVAE, which consists of multiple layers of TVAE, and add the prediction error of the quality variable to the reconstruction error of VAE; Step 3, adding a weighted part to the STVAE network to obtain a weighted stacked quality supervision VAE network W-STVAE; the weighted part is used to assign different weights to each process variable according to the maximum information coefficient between each process variable and the quality variable; Step 4, using gated linear units to integrate feature information at different levels in the W-STVAE network to obtain the final weighted gated stacked quality supervision VAE soft sensor model W-GSTVAE; Step 5, using the historical data collected in step 1 as training samples, training the weighted gated stacked quality supervision VAE soft measurement model W-GSTVAE to obtain a trained soft measurement model W-GSTVAE; Step 6: Input the process variable corresponding to the quality variable to be predicted into the trained soft sensor model W-GSTVAE to obtain the corresponding predicted value.
2. The method according to claim 1, characterized in that The historical data collected in step 1 is recorded as D l ={(x1,y1),(x2,y2),…,(x Nl ,y Nl )}, where x is the process variable, y is the quality variable, N l is the number of samples, l is the dimension of the process variable; the set of standardized process variables in the historical data is recorded as X s .
3. The method according to claim 2, characterized in that The weight of the d-th dimension process variable is calculated as follows: Among them, MIC d is the maximum information coefficient between the d-th dimension process variable and the quality variable.
4. The method according to claim 3, characterized in that The step 5 comprises: Step 5.1, X s As the input of the W-GSTVAE encoder, the features of each sample in the latent space are calculated to follow the normal distribution The mean and variance of Step 5.2, based on the mean and variance Sampling to obtain hidden variable z; Step 5.3, use the hidden variable z as the input of the decoder to obtain the reconstructed value of each sample And the predicted value of the sample Step 5.4, repeat steps 5.1 to 5.3 to calculate the loss function LOSS of each layer of pre-training p , the Adam optimization method is used to update the model parameters; Step 5.5, use the hidden layer feature set Z to calculate the gate control value and the linear mapping y instead of the original output k , and get the final output form Fine-tune the global model by minimizing the loss function minΓ; save the network parameters as the trained soft sensor model W-GSTVAE.
5. The method according to claim 4, characterized in that In the soft sensor model W-GSTVAE, the pre-trained loss function is: Among them, x n represents the input process variable of the first layer of pre-training, represents the reconstructed value of the input process variable of the first layer of pre-training; y n represents the true quality variable, represents the predicted value of the quality variable; n z represents the dimension of latent variables, σ i(n) represents the variance of the latent space, μ i(n) represents the mean of the latent space; For the kth layer of the network, the pre-trained loss function is: Among them, l k is the dimension of the k-th layer input process variable, λ d(k) is the weight coefficient of the kth layer, z k(n) and is the input and reconstructed output of the kth TVAE, λ is the weight coefficient, y k(n) represents the true quality variable of the kth layer, represents the predicted value of the quality variable at the kth level; n z(k) is the hidden variable dimension of the kth TVAE.
6. The method according to claim 5, characterized in that The minimization loss function minΓ is: Among them, N l represents the number of samples in the training set, represents the predicted value of the quality variable of the nth sample in the training set, and y(n) represents the true quality variable of the nth sample in the training set.
7. The method according to claim 1, characterized in that The industrial processes include petrochemicals, blast furnace ironmaking, and fermentation processes.
8. A method for predicting butane concentration at the bottom of a debutanizer based on weighted gated stack quality supervision VAE, characterized in that: The method is implemented based on the method described in any one of claims 1 to 6, and the method comprises: Step 1: Collect historical data from the debutanizer process, including 7 process variables and 1 quality variable; Step 2: Augment the collected historical data and use the augmented data as training samples to train the W-GSTVAE network; Step 3: collect process variables in the debutanizer process in real time, input them into the W-GSTVAE network trained in step 2, and obtain the predicted values of the quality variables.
9. The method according to claim 8, characterized in that The seven process variables are top temperature u1, top pressure u2, top reflux flow u3, top discharge flow u4, plate 6 temperature u5, first tower bottom temperature u6 and second tower bottom temperature u7; the quality variable is butane concentration at the bottom of the debutanizer tower.
10. The method according to claim 9, characterized in that The data after amplification processing in step 2 includes: [u1(k),u2(k),u3(k),u4(k),u5(k),u5(k-1),u5(k-2),u5(k-3),(u6(k)+u7(k)) / 2,y(k-1),y(k-2),y(k-3),y(k-4)] T , where k represents the moment, y represents the quality variable, and T represents the transpose.
