A deep extended VAE soft-sensing method based on instant learning

Abstract

The application discloses a deep extension VAE soft measurement method based on instant learning, and belongs to the technical field of industrial process control. In the modeling process, the DE-VAE is composed of a multilayer E-VAE stack, wherein the output of the E-VAE includes reconstructed original input data, expected output data and hidden variables of previous E-VAE, the correlation between the extracted features and the original input and output is enhanced by multiplexing prior information, meanwhile, the quality variable is introduced in the pre-training process, and the extraction ability of the model to target related features is enhanced. In the online prediction stage, the model is updated by using instant learning, a new modeling method different from existing soft measurement modeling methods is designed, and the prediction accuracy of the soft measurement model is greatly improved.

Description

Technical Field

[0001] This invention relates to a deep extended VAE soft measurement method based on real-time learning, belonging to the field of industrial process control technology. Background Technology

[0002] Soft measurement methods utilize mathematical modeling and data processing techniques to achieve real-time prediction of difficult-to-measure key variables based on easily measurable process variables in the production process. They can be broadly categorized into mechanistic modeling methods and data-driven modeling methods. Compared to mechanistic modeling methods, data-driven modeling methods rely less on mechanistic knowledge of the process and can be flexibly adjusted and adapted to different production environments and data characteristics, thereby capturing complex nonlinear relationships and dynamic changes. Therefore, they are increasingly widely used in actual industrial processes.

[0003] Autoencoders (AEs), as unsupervised deep learning networks, possess excellent nonlinear feature extraction capabilities and scalability. However, soft sensor modeling methods based on AEs are susceptible to measurement noise, which can affect the model's predictive performance. To address this, Variational Autoencoders (VAEs) were proposed. By introducing the concept of Bayesian variational inference between the encoder and decoder, VAEs can better capture the latent structure of data when generating samples. VAEs can also further enhance the model's feature extraction and generation capabilities by stacking multiple encoding and decoding layers. Shen et al. proposed a nonlinear probabilistic latent variable regression model (NPLVR) based on variational autoencoders in "Nonlinear probabilistic latent variable regression models for softsensor application: From shallow to deep structure. Control Engineering Practice 94(2020):104198." They extended the NPLVR model from shallow to deep structures by stacking layers, thereby extracting deeper nonlinear features for soft sensor modeling. Chinese patent CN118444641A discloses a soft measurement method based on weighted gated stacked quality supervision VAE. By introducing a feature integration mechanism of quality information and gated linear units, the prediction accuracy of the soft measurement model is improved.

[0004] However, the above models do not consider that during multi-layer VAE pre-training, reconstruction errors may accumulate as the number of layers increases, affecting the model's feature extraction capabilities and causing the correlation between input and output variables to fail to be effectively mapped, thus reducing the model's prediction accuracy. Furthermore, the time-varying nature of process data may also lead to a decrease in model prediction accuracy. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a just-in-time learning-based deep extended-VAE soft measurement method. By constructing a deep extended-VAE network model and reusing prior information to enhance the correlation between extracted features and the original input and output, a just-in-time learning-based deep extended-variational autoencoder (JITL-DE-VAE) is proposed for predicting the butane concentration at the bottom of a butane removal tower. In the pre-training stage, the deep extended structure is used to fully extract effective features to complete the training of the basic model. In the fine-tuning stage, based on the just-in-time learning concept, similar data is used to update the model parameters, improving the prediction accuracy of the deep extended-variational autoencoder model. This overcomes the problems of reconstruction error accumulation and insufficient model adaptation to the time-varying nature of industrial processes in existing soft measurement techniques based on multi-layer VAE networks, which lead to a decline in prediction performance.

[0006] A deep extended VAE soft measurement method based on real-time learning includes:

[0007] Step 1: Select the input variables for the model based on theoretical analysis, collect data including input variables and quality variables, and standardize the data;

[0008] The labeled training dataset is Where x is the input variable, y is the quality variable, and N is the mass variable. l The number of samples; the standardized set of input variables in the training dataset is X. s The sample to be predicted is X. new ;

[0009] Step 2: Extract high-level features from the data using a single-layer VAE. To ensure the correlation between the extracted features and the quality variables, the prediction error of the quality variables is added to the reconstruction error of the VAE. In this way, the features extracted at each layer are not only a feature representation of the input variables, but also an effective interpretation of the quality variables.

[0010] Step 3: Construct a deep extended DE-VAE network model, using the hidden layers and input variables of the previous E-VAE as the input of the next E-VAE, making full use of the effective information of the features and preventing excessive loss of effective information;

[0011] Step 4: Place X s As input to the DE-VAE model, the reconstructed value of the input sample and the predicted target value are calculated for pre-training.

[0012] 4.1: Xs As input to the DE-VAE encoder, the features of the sample in the latent space are calculated to follow a distribution. The mean and variance [μ,σ] 2 ];

[0013] 4.2: Using reparameterization techniques, based on the mean and variance [μ, σ] 2 The latent variable z is obtained through sampling;

[0014] 4.3: Using the latent variable z as input to the decoder to obtain the reconstructed value of the sample. and the predicted value of the sample

[0015] Step 5: Repeat step 4 to calculate the loss function LOSS for each pre-trained layer and update the model parameters using the Adam optimization method;

[0016] Step 6: After pre-training is complete, use the latent variable z to calculate the linear mapping y of the original output. k The final output format is obtained. Fine-tuning of the global network is achieved by minimizing the loss function minΓ; network parameters are saved for online prediction.

[0017] Step 7: In the online prediction phase, an instant learning framework is introduced. When a test sample appears, a small batch of samples most similar to the test sample is searched in historical data (including the training set and the predicted test samples). This batch of data is then used to update the DE-VAE model parameters, resulting in the updated JITL-DE-VAE model.

[0018] Step 8: The sample X to be predicted new After undergoing the same standardization, the data is input into the updated JITL-DE-VAE model to obtain the final predicted value.

[0019] The DE-VAE network in step 3 differs from the traditional AE. VAE possesses both the data mining and nonlinear modeling capabilities of deep learning, and can also model process uncertainties and data noise like a probabilistic model, significantly improving the model's ability in probabilistic data description and feature extraction. Assume that the input variable x and the quality variable y are both generated by random, continuous latent variables z.

[0020]

[0021] Suppose x and y are conditionally independent. A latent variable z is sampled from the latent space and passed through two parameters δ. x and δ y The neural network is used to obtain x and y. Based on the above generation process, the joint probability distribution of the generative model is:

[0022] p δ (x,y,z)=p δx (x|z)p δy (y|z)p(z)

[0023] The log-likelihood function of the marginal probability distribution p(x,y) of the data sample points is:

[0024]

[0025] in For the inference model, it serves as an additional variational posterior to approximate the true complex posterior probability p(z|x).

[0026] According to Jensen's inequality:

[0027]

[0028] in, It is the lower bound of the marginal probability likelihood function, in order to calculate And δ, maximizing the lower bound of evidence:

[0029]

[0030] The lower bound of evidence can be understood in two parts: the first two terms and the generative model p of x and y. δx (x|z), p δy (y|z) is related, among which z Follows approximate variational posterior Assumption It follows a normal distribution.

[0031] Since generative models all follow a normal distribution, therefore logp δx (x|z) and logp δy (y|z) can be represented as:

[0032]

[0033] For real-valued samples, these two terms represent the mean squared errors of x and y. The last term... It is an approximate variational posterior. The KL divergence between the prior p(z) and the prior. Because Since z follows a normal distribution and p(z) follows a standard normal distribution, this term can be written as:

[0034]

[0035] The mean μ and variance σ in the formula 2 It is obtained from two encoders respectively.

[0036] The loss function for quality-supervised VAE pre-training is negative. For real-valued data, it can be represented as:

[0037]

[0038] Where N l It is the number of training samples, x n Represents the original input variables. This represents the reconstructed value of the original input variable, y. n Represents the true quality variable. σ represents the predicted value of the quality variable. 2 The variance of the latent variable is represented by μ, and the mean of the latent variable is represented by μ. Because the loss function involves information related to the target value, labeled data is required for supervised training.

[0039] After training the parameters in the network by minimizing the loss function, they can be directly stacked into a deep neural network for DE-VAE, using the latent variables and inputs of the previous E-VAE layer as the inputs of the next E-VAE layer. The loss function of the k-th E-VAE layer is:

[0040]

[0041] Where k > 1, μ k It is a latent variable z k The mean, σ k It is a latent variable z k Standard deviation, x i Indicates input variables, This represents the reconstructed value of the input variable, y. i Represents the true quality variable. This represents the predicted value of a quality variable. and Let represent the i-th latent variable and the reconstructed value output in the j-th layer E-VAE, respectively, and m be the latent variable dimension of the k-th layer E-VAE.

[0042] Fine-tuning the parameters in the network using the backpropagation algorithm (BP) can be defined as minimizing the following unconstrained optimization problem:

[0043]

[0044] Where N l It is the number of training samples. Let y(n) represent the predicted value of the quality variable for the nth sample, and y(n) represent the actual quality variable for the nth sample. This is equivalent to minimizing the mean squared error (MSE) between the predicted value and the actual target value.

[0045] The instant learning framework in step 7 considers the varying impacts of different process variables on quality variables when measuring the similarity between test samples and historical data. Therefore, each process variable needs to be weighted when calculating sample similarity. To this end, this invention introduces the Maximum Information Coefficient (MIC) to compare the correlation between process variables and quality variables. MIC measures the degree of association between variables by comprehensively analyzing linear or nonlinear relationships. Compared to the Pearson correlation coefficient, MIC provides a more comprehensive reflection of the relationships between variables. By traversing all historical data, a small batch of samples most similar to the query sample is selected. Each historical labeled sample and the query sample x... q The weighted Euclidean distance between them is calculated as follows:

[0046]

[0047] Where x h Let Q represent historical data, and let Q represent an n×n diagonal matrix where the diagonal elements are the maximum mutual information coefficients between process variables and quality variables.

[0048] The beneficial effects of this application are:

[0049] 1. To address the problem of reconstruction error accumulation in variational autoencoders (VAEs), a deep extended structure DE-VAE is proposed. This structure uses the original input data, quality variable data, and latent variables from the previous E-VAE as inputs to the next layer of E-VAE. By reusing prior information, it enhances the correlation between extracted features and the original input and output, making full use of feature information to filter out information irrelevant to the output.

[0050] 2. To address the time-varying nature of industrial processes, an adaptive framework was introduced. By utilizing weighted Euclidean distance and cosine distance, a small subset of samples most similar to the query sample were found in the historical database during the online prediction phase. These similar samples were then used to fine-tune the model, update the model parameters, enhance the model's ability to handle time-varying process data, and improve the model's accuracy.

[0051] 3. In the fine-tuning stage, based on the idea of ​​real-time learning, similar data is used to update the model parameters to improve the prediction accuracy of the deep extended variational autoencoder model. This is to overcome the problems of reconstruction error accumulation and prediction performance degradation caused by insufficient model adaptation to the time-varying nature of industrial processes in existing soft measurement technology based on multi-layer VAE networks. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart of a deep extended VAE soft measurement method based on real-time learning in Embodiment 2 of the present invention;

[0054] Figure 2 The process flow diagram of the butane removal tower involved in Embodiment 2 of the present invention;

[0055] Figure 3 A schematic diagram of the quality supervision VAE part in the JITL-DE-VAE model of a deep extended VAE soft measurement method based on real-time learning provided in Embodiment 2 of the present invention;

[0056] Figure 4 The overall structure diagram of the DE-VAE model in a deep extended VAE soft measurement method based on real-time learning provided in Embodiment 2 of the present invention;

[0057] Figure 5A The SQAE model is used to predict the butane concentration in the butane dehydrogenator in Example 2 of this invention.

[0058] Figure 5B The SVAE model is used to predict the butane concentration in the butane dehydrogenator in Example 2 of this invention.

[0059] Figure 5C The H-NPLVR model is used to predict the butane concentration in the butane dehydrogenator in Example 2 of this invention.

[0060] Figure 5D The W-GSTVAE model is used to predict the butane concentration in the butane dehydrogenator in Example 2 of this invention.

[0061] Figure 5E The fitting curve of butane concentration prediction in the butane debutane tower provided in Example 2 of this invention is shown in the figure.

[0062] Figure 5F The fitting curve of butane concentration prediction in the butane debutane tower using the JITL-DE-VAE model proposed in this application in Embodiment 2 of the present invention;

[0063] Figure 6 Box plots of prediction errors for the six models in Embodiment 2 of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0065] Example 1:

[0066] This embodiment provides a deep extended VAE soft measurement method based on real-time learning, the method including:

[0067] Step 1: Soft measurement modeling includes two parts: offline modeling and online prediction. Based on theoretical analysis, model variables are selected, and then data is collected to obtain a historical dataset. This historical dataset is divided into training and test sets, and the data is standardized. The labeled training dataset is... Where x is the input variable, y is the quality variable, and N is the mass variable. l The number of samples. The standardized set of input variables in the training dataset is X. s The sample to be predicted is X. new ;

[0068] Step 2: Extract high-level features from the data using a single-layer VAE. To ensure the correlation between the extracted features and the quality variables, the prediction error of the quality variables is added to the reconstruction error of the VAE. In this way, the features extracted at each layer are not only a feature representation of the input variables, but also an effective interpretation of the quality variables.

[0069] Step 3: Construct a deep extended DE-VAE network model. First, a deep DE-VAE is constructed by stacking multiple E-VAEs. Then, the hidden layers and input variables of the previous E-VAE are used as the input of the next E-VAE, which makes full use of the effective information of the features and prevents excessive loss of effective information.

[0070] Step 4: Place X s As input to the DE-VAE model, the reconstructed value of the input sample and the predicted target value are calculated for pre-training.

[0071] 4.1: X s As input to the DE-VAE encoder, the features of the sample in the latent space are calculated to follow a distribution. mean and variance

[0072] 4.2: Using reparameterization techniques, based on the mean and variance Sampling to obtain latent variables z ;

[0073] 4.3: Using Latent Variables z The reconstructed value of the sample is obtained as input to the decoder. and the predicted value of the sample

[0074] Step 5: Repeat step 4 to calculate the loss function LOSS for each pre-trained layer and update the model parameters using the Adam optimization method;

[0075] Step 6: After pre-training is complete, use the latent variable z to calculate the linear mapping y of the original output. k The final output format is obtained. Fine-tune the global parameters by minimizing the loss function minΓ. Save the network parameters for online prediction.

[0076] Step 7: Introduce an on-demand learning framework during the online prediction phase. When a test sample appears, search for a small batch of samples most similar to the test sample in historical data (including the training set and the predicted test samples). Then, use this batch of data to update the DE-VAE model parameters.

[0077] Step 8: The sample X to be predicted new After standardization, the data is input into the updated model (JITL-DE-VAE) to obtain the final predicted value.

[0078] Example 2:

[0079] This embodiment provides a method for predicting butane concentration in a butane-dehydrogenator. This method employs a deep extended VAE soft-sensing method based on real-time learning, as described in Embodiment 1, to predict the butane concentration in the butane-dehydrogenator. (See [link to previous embodiment]). Figure 1 The method includes:

[0080] Step 1: Collect input and output data to form a historical training sample database.

[0081] This embodiment uses a publicly available dataset of a real industrial process for butanizer removal. See the process flow diagram of the butanizer for details. Figure 2 The system mainly consists of a heat exchanger, condenser, reboiler, reflux pump, separator, and accumulator. Table 1 shows the seven variables measured by the sensor. Since the primary purpose of the butane removal column is to minimize butane content, real-time measurement of butane concentration is necessary. However, in practice, butane content is obtained through gas chromatography sampling and analysis, which introduces a significant measurement delay. Furthermore, complex nonlinear relationships exist between the variables. The dataset includes seven input variables and one mass variable, namely the butane concentration at the bottom of the butane removal column, totaling 2394 sample data points.

[0082] Considering process dynamics, based on expert knowledge and physical analysis, 13-dimensional augmented data variables are obtained:

[0083] [u1(k),u2(k),u3(k),u4(k),u5(k),u5(k-1),u5(k-2),u5(k-3),(u6(k)+u7(k)) / 2,y(k-1),y(k-2),y(k-3),y(k-4)] T Where k represents time and T represents transpose.

[0084] Table 1 Process variables of the butane removal tower

[0085]

[0086] Step 2: After augmenting the original sample dataset, 2390 samples are obtained. The data is then standardized, and the standardized dataset is divided into training and test sets. The first 1600 samples are used as the training dataset, and the remaining samples are used as the test dataset to build and train the JITL-DE-VAE network. The standardized set of input variables in the training dataset is X. s The sample to be predicted is X. new .

[0087] Step 3: Deep expansion of VAE (DE-VAE) design.

[0088] While simply compressing the number of deep hidden neurons in a stacked VAE can effectively handle high-dimensional data, it may lead to the loss of valuable information in the features. Therefore, quality supervision information is incorporated into the training process of each VAE layer to guide feature learning. For a single-layer VAE with quality supervision, see [link to documentation]. Figure 3 To reduce the accumulation of reconstruction errors caused by multiple E-VAEs, the hidden layers and input variables of the previous E-VAE are further used as inputs to the next E-VAE to improve model prediction accuracy. This is achieved by stacking multiple E-VAEs to obtain a deep extended VAE, or DE-VAE. (See the structure below.) Figure 4 .

[0089] For a single-layer VAE model with quality variables, assuming that the input variable x and the output target value y in the VAE are both generated by random continuous latent variables z, the generation process can be described as follows:

[0090]

[0091] Suppose x and y are conditionally independent. A latent variable z is sampled from the latent space and passed through two parameters δ. x and δ y The neural network is used to obtain x and y. Based on the above generation process, the joint probability distribution of the generative model is:

[0092] p δ (x,y,z)=pδx (x|z)p δy (y|z)p(z)

[0093] The log-likelihood function of the marginal probability distribution p(x,y) of the data sample points is:

[0094]

[0095] in For the inference model, it serves as an additional variational posterior to approximate the true complex posterior probability p(z|x).

[0096] According to Jensen's inequality:

[0097]

[0098] It is the lower bound of the marginal probability likelihood function, in order to calculate And δ, maximizing the lower bound of evidence:

[0099]

[0100] The lower bound of evidence can be understood in two parts: the first two terms and the generative model p of x and y. δx (x|z), p δy (y|z) is related, where z follows an approximate variational posterior. Assumption It follows a normal distribution. Since all generative models follow a normal distribution, therefore logp δx (x|z) and logp δy (y|z) can be represented as:

[0101]

[0102] For real-valued samples, these two terms represent the mean squared errors of x and y. The last term... It is an approximate variational posterior. The KL divergence between the prior p(z) and the prior. Because Since z follows a normal distribution and p(z) follows a standard normal distribution, this term can be written as:

[0103]

[0104] The mean and variance in the formula are obtained from two encoders.

[0105] The loss function for single-layer quality-supervised VAE pre-training is negative. For real-valued data, it can be represented as:

[0106]

[0107] in, Representing the approximate variational posterior The KL divergence between the prior p(z) and N l It is the number of training samples, x n Indicates the pre-training input variables. y represents the reconstructed value of the pre-trained input variable. n Represents the true quality variable. σ represents the predicted value of the quality variable. 2 represents the variance of the latent variables in this layer, and μ represents the mean of the latent variables in this layer;

[0108] After training the network parameters by minimizing the loss function, they can be directly fed into the deep neural network of DE-VAE, using the latent variables and input of the previous E-VAE as the input of the next E-VAE. The loss function of the k-th layer E-VAE is:

[0109]

[0110] Where k > 1, μ k It is a latent variable z k The mean, σ k Latent variable z k standard deviation and Let represent the i-th latent variable and the reconstructed value in the j-th E-VAE, respectively, and m be the dimension of the latent variable in the k-th E-VAE.

[0111] Step 4: Place X s As input to the DE-VAE model, the reconstructed value of the input sample and the predicted target value are calculated for pre-training.

[0112] 4.1: X s As input to the DE-VAE encoder, the features of the sample in the latent space are calculated to follow a distribution. The mean and variance [μ,σ] 2 ];

[0113] 4.2: Using reparameterization techniques, based on the mean and variance [μ, σ] 2 The latent variable z was sampled.

[0114] 4.3: Using the latent variable z as input to the decoder to obtain the reconstructed value of the sample. and the predicted value of the sample

[0115] Step 5: Repeat Step 4 to calculate the loss function (LOSS) for each layer during pre-training. Use a trial-and-error approach to determine the network structure. The final DE-VAE model stacks three E-VAE models, with the number of neurons in the input, hidden, and output layers being 13-7-14, 20-7-21, and 27-7-28, respectively. The Adam optimization method is used to update the model parameters. The number of training epochs during the pre-training phase is set to 100, the batch size to 64, and the learning rate to 0.01.

[0116] Step 6: After pre-training is complete, use the latent variable z to calculate the linear mapping y of the original output. k The final output format is obtained. Fine-tuning the parameters in the network using the backpropagation algorithm (BP) can be defined as minimizing the following unconstrained optimization problem:

[0117]

[0118] Where N l It is the number of training samples. Let y(n) represent the predicted value of the quality variable of the nth sample, y(n) represent the true quality variable of the nth sample, and ||·||2 represent the L2 norm.

[0119] This is equivalent to minimizing the mean square error (MSE) between the predicted value and the actual target value.

[0120] The fine-tuning phase uses 100 training epochs, a batch size of 64, and a learning rate of 0.01. The network parameters are then saved for online prediction.

[0121] Step 7: In the online prediction stage, an instant learning framework is introduced. When a test sample appears, a small batch of samples most similar to the test sample is searched in historical data (including the training set and the predicted test samples). This batch of data is then used to update the DE-VAE model parameters. Since the impact of different process variables on the quality variable varies when measuring the similarity between the test sample and historical data, each process variable needs to be weighted when calculating sample similarity. Therefore, this application introduces the Maximum Mutual Information Coefficient (MIC) to compare the correlation between process variables and quality variables. MIC measures the degree of association between variables by comprehensively analyzing the linear or nonlinear relationships between them. Compared to the Pearson correlation coefficient, MIC is more comprehensive in reflecting the relationship between variables. By traversing all historical data, a small batch of samples most similar to the query sample is selected. Each historical labeled sample and the query sample x... q The weighted Euclidean distance between them is calculated as follows:

[0122]

[0123] Where xh Let Q represent historical data, and let Q represent an n×n diagonal matrix where the diagonal elements are the weighting coefficients of the process variables and the quality variables' MICs.

[0124] The calculation process of MIC can be described as follows: If there is a relationship between two variables, a grid G ​​can be plotted on a scatter plot with these two variables as the X and Y axes to divide the data and encapsulate this relationship. Given a dataset D containing two variables and n samples, the dataset is divided into a and b parts along the X and Y axes using the grid G ​​as described above. The probability distribution corresponding to the points in each cell of grid G ​​is represented by D. G This indicates that mutual information (MI) can be calculated under this mesh partitioning. Different mesh partitionings will produce different probability distributions and mutual information values. The maximum MI value obtained by partitioning mesh G is defined as:

[0125] I(D,a,b)=maxI(D G )

[0126] Standardize the MI values ​​under different grids to obtain modified values ​​between 0 and 1 to ensure fair comparison. Therefore, MIC can be defined as:

[0127]

[0128] To further enhance the correlation between features and target values, features highly correlated with output prediction are extracted. When searching for similar samples, the minimum correlation coefficient (MIC) between input and quality variables is calculated. Variables correlated with the output are assigned more weight, while variables with low correlation are given less attention. Given an input variable of dimension n and a sample size of N... l The labeled training dataset E l The importance of a variable is determined by its MIC (Minimum Indicator) with respect to the target variable. The weights of the d-th dimension input variables are calculated as follows:

[0129]

[0130] Therefore, the diagonal elements of Q can be represented as λ1...λ n .

[0131] Step 8: The sample X to be predicted new After undergoing the same standardization, the data is input into the updated model (JITL-DE-VAE) to obtain the final predicted value.

[0132] This application proposes a time-varying process adaptive model update framework based on a deep extended variational autoencoder (DE-VAE). This framework can fully extract effective information from process variables and update model parameters in real time, thereby improving performance. Addressing the issue of traditional deep learning methods failing to update models in real time, leading to performance degradation, this application first extends the input and output layers of each E-VAE to retain as much effective information as possible, thus improving the model's feature extraction capabilities. Then, a real-time fine-tuning strategy is employed to update the DE-VAE model, enabling deep learning to quickly adapt to the process's operational state and further enhancing the model's prediction accuracy.

[0133] To compare the superiority of the JITL-DE-VAE proposed in this application, five models were used for comparison: SQAE, SVAE, H-NPLVR, W-GSTVAE, and DE-VAE. SQAE is an improved method based on SAE, as described above; SVAE is a supervised variational autoencoder model; H-NPLVR is a hierarchical nonlinear probabilistic latent variable regression model; W-GSTVAE is an improved VAE-based method (see Chinese patent publication CN118444641A for details); and DE-VAE is a VAE method with an extended structure introduced in this application. To ensure the reasonableness of the comparison, the model structures of the first three regression models were set to 13-10-7-4-1, where 10, 7, and 4 represent the number of hidden layer neurons in the three-layer network structure, and the model structure of H-NPLVR was set to 13-10-10-10-1. Furthermore, this application uses the following two parameters as performance evaluation indicators for the prediction models in each prediction method, and their calculation formulas are as follows:

[0134] (1) Root mean square error (RMSE):

[0135]

[0136] Where N is the number of samples in the test set, and y(n) is the actual output value of the nth sample in the test set. This is the predicted value of the nth sample in the test set. RMSE focuses on the prediction error of the sample; the smaller the RMSE, the better the model performance.

[0137] (2) Coefficient of Determination (R²) 2 ):

[0138]

[0139] in R is the average of the true output values ​​of all samples in the test set. 2R represents the squared correlation between the actual output values ​​and predicted values ​​on the test set, reflecting the model's ability to explain the variance of the output data. 2 The closer to 1, the better the model performance.

[0140] All experiments in this application embodiment were conducted on the same computer: an Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz processor, 16GB memory, and PyTorch 3.8 running a 64-bit Windows 11 operating system. The prediction results for butane concentration using SQAE, SVAE, H-NPLVR, W-GSTVAE, DE-VAE, and the proposed JITL-DE-VAE scheme are as follows: Figure 5A , Figure 5B , Figure 5C , Figure 5D , Figure 5E , Figure 5F and Figure 6 The detailed experimental results of the test set are shown in Table 2. The evaluation metrics are RMSE and R. 2 As shown in Figure 5, all six methods can reflect the changing trend of butane content well. However, in predicting both peak and trough values, the JITL-DE-VAE model in this application demonstrates superior prediction accuracy compared to the other models. Regarding the prediction of peak values, the SQAE model has a prediction error of approximately 0.2, significantly weaker than the other five models, proving that VAE's denoising capability is superior to SAE. Regarding the prediction of trough values, the JITL-DE-VAE model in this application has a prediction error below 0.02, superior to the other models. This demonstrates the effectiveness of adding an extended layer, introducing output constraints, and adopting an adaptive update strategy. Therefore, compared to the other five models, the input-output correlation features extracted by the JITL-DE-VAE model in this application are more suitable for process modeling.

[0141] Table 2 Evaluation metrics of the six methods on the DC dataset

[0142]

[0143] exist Figure 6The error box plot clearly shows that the proposed JITL-DE-VAE model has a narrower error range compared to the other five methods, demonstrating a significant advantage. The JITL-DE-VAE approach focuses more on samples that cause larger prediction errors, rather than simply those that constitute a large proportion of the distribution, thus predicting butane concentration more accurately, further proving the superiority of the proposed method. From an industrial process perspective, accurately predicting butane content is crucial when it is high, as this helps in the timely detection of process anomalies.

[0144] This application discloses a soft sensor modeling method for deep extended VAEs based on real-time learning. This method can fully extract effective information from process variables and update model parameters in real time, improving performance. Addressing the issue of traditional deep learning methods failing to update models in real time, leading to performance degradation, this application first extends the input and output layers of each E-VAE to retain as much effective information as possible, thereby improving the model's feature extraction capability. Then, a real-time fine-tuning strategy is employed to update the DE-VAE model, enabling deep learning to quickly adapt to the process operating state and further enhancing the model's prediction accuracy. Finally, the superiority of the proposed method is verified on a real industrial dataset from a butanizer tower. This provides strong technical support for butanizer tower butanane concentration monitoring.

[0145] Some steps in the embodiments of this application can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0146] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A deep extended VAE soft-sensing method based on instant learning, applied to an industrial process to realize prediction of a quality variable, characterized in that, The method comprises: Step 1: select input variables of the model, collect data including process variables and quality variables, and standardize the data; the collected data is denoted as ; wherein is a process variable, is a quality variable, is a sample number; a set of process variables standardized in the training data is , and a sample to be predicted is ; Step 2: When the single-layer VAE extracts the high-level features of the data, the prediction error of the quality variable is also added to the reconstruction error of the VAE to obtain an E-VAE; Step 3: A deep-layer extended DE-VAE network model is constructed, the deep-layer DE-VAE is constructed by stacking multiple E-VAEs, and the hidden layer of the previous layer E-VAE and the process variable are taken as the input of the next layer E-VAE; Step 4: Calculate the reconstruction value of the input sample and the predicted target value as the input of the DE-VAE model, and pre-train the DE-VAE model. As the input of the DE-VAE model, the reconstruction value of the input sample and the predicted target value are calculated, and the DE-VAE model is pre-trained. Step 5: Repeat Step 4 to calculate the loss function for each pre-trained layer Update the model parameters using the Adam optimization method; Step 6: After pre-training, use the latent variables Compute linear mapping of original output to get final output form by minimizing loss function Fine-tune global; Step 7: In the online prediction stage, an instant learning framework is introduced, when the test sample appears, a small batch of samples most similar to the test sample in the historical data is searched, and then the DE-VAE model parameters are updated by using the batch of data to obtain an updated model JITL-DE-VAE, and the historical data includes the training set and the predicted test sample; Step 8: The sample to be predicted After standardization, the input is input into the updated model JITL-DE-VAE to obtain the final prediction value.

2. The method of claim 1, wherein, Each historical marker sample and query sample The weighted Euclidean distance between them is calculated as follows: wherein represents historical data, represents a diagonal matrix of the maximum mutual information coefficients of the process variables and the quality variables, T denotes transposition.

3. The method of claim 2, wherein, The step 4 comprises: 4.1: Compute As input to the DE-VAE encoder, compute the mean and variance of the distribution from which the features of the sample in latent space are drawn ; 4.2: Using the reparameterization trick, we sample from the latent variable according to the mean and variance ; 4.3: Using latent variables As input to the decoder, the reconstructed values of the samples and the predicted values of the samples are obtained.

4. The method of claim 3, wherein, In the model JITL-DE-VAE, the loss function of the first layer E-VAE pre-training is: where denotes the approximate variational posterior and the KL divergence between the prior and the approximate variational posterior is the number of training samples, denotes the pre-training process variable, denotes the reconstructed value of the pre-training process variable, denotes the true quality variable, denotes the predicted value of the quality variable, denotes the variance of the first layer of hidden variables, denotes the mean of the first layer of hidden variables; The loss function of layer E-VAE is: layer E-VAE is: where , is the mean of the latent variable , is the standard deviation of the latent variable , is the number of training samples, and denote the th latent variable and the reconstruction value in the th E-VAE, respectively, is the dimension of the latent variable of the th E-VAE.

5. The method of claim 4, wherein, The step 6 comprises: minimizing a loss function is: wherein is the number of training samples, denotes the predicted value of the quality variable of the n th sample, denotes the true quality variable of the n th sample, and || · ||2represents the L2-norm.

6. The method of claim 1, wherein, The industrial process includes petroleum chemical industry, blast furnace oil refining and fermentation process.

7. A deep extended VAE based on instant learning for depropanizer bottom concentration prediction method, characterized in that, The method is realized based on the method in any one of claims 1-5, and the method comprises: Step 1: Collecting historical data in a debutanizer process, including 7 process variables and 1 quality variable; Step 2: Augmenting the collected historical data, taking the augmented data as training samples to train the JITL-DE-VAE model; Step 3: Real-time collecting process variables in the debutanizer process, inputting the JITL-DE-VAE model trained in step 2 to obtain the predicted value of the quality variable.

8. The method of claim 7, wherein, The 7 process variables are top temperature u 1. Top pressure u 2. Top reflux flow u 3. Top draw flow u 4. Tray 6 temperature u 5. First bottoms temperature u 6. Second bottoms temperature u 7. The mass variable is debutanizer bottoms butane concentration.

9. The method of claim 8, wherein, The data after the augmentation process in step two includes: wherein, k denotes the time instant, T denotes the transpose.

Images