Industrial soft measurement method based on semi-supervised double-guide self-encoding model

Abstract

The invention relates to an industrial soft measurement method based on a semi-supervised double-guide self-encoding model. According to the method, the characteristics of industrial process data are fully considered, a stacked double-guide auto-encoder structure is adopted in a pre-training stage, and feature learning related to quality in the pre-training stage is enhanced by minimizing the sum of a prediction error of a quality variable and a reconstruction error relative to an input variable. And utilizing a semi-supervised learning strategy to combine unlabeled data with labeled data, and optimizing model parameters. A real-time learning strategy is introduced to quickly adapt to the dynamic change of the production process. Compared with an existing method, the method can effectively distinguish features which are more critical to a prediction task, and effectively solves the time-varying problem which is difficult to deal with by a traditional method. The method not only can efficiently capture features related to quality variables, but also can continuously learn from new data to quickly adapt to dynamic changes of the production process, and provides powerful support for accurate monitoring and prediction of the industrial process.

Description

Technical Field

[0001] The invention relates to the field of industrial process control and particularly to an industrial soft measurement method based on a semi-supervised dual-guide autoencoder model. Background Art

[0002] With the advancement of industrial intelligence, the demand for precise monitoring and prediction of industrial processes is growing, placing higher demands on optimizing control accuracy and production efficiency. Accurate prediction of key process variables is crucial for industrial production and is directly related to process stability, quality improvement, and cost control. However, in real-world industrial production, the limitations of measurement technology make online monitoring of key quality variables difficult, hindering the development of industrial process control. To address these issues, industrial soft sensing technology has been proposed. However, traditional soft sensing methods such as principal component analysis (PCA) and partial least squares regression (PLSR) often suffer from performance limitations when dealing with complex or high-dimensional nonlinear industrial data, and their generalization capabilities are weak. To overcome these limitations, deep learning technology has been introduced to the soft sensing field in recent years, demonstrating powerful capabilities for extracting nonlinear features and superior generalization performance. Deep stacked autoencoders (SAE), a representative deep learning model, can capture hidden and complex structural information in data through multi-layered nonlinear transformations, more accurately reflecting data structure than traditional models.

[0003] However, traditional SAE soft-sensor modeling approaches learn high-level features of data by stacking multiple autoencoders layer by layer. Each layer aims to reconstruct the input data and minimize errors. This makes it easy for small errors in the initial hidden layer to be amplified in subsequent layers within a multi-layered structure, interfering with network learning and generalization. Furthermore, SAE relies on unsupervised learning to reconstruct the input at each layer, which results in the model being unable to effectively distinguish features that are more critical to the prediction task during feature extraction.

[0004] In response to the above problems, the present invention proposes an industrial soft measurement method based on a semi-supervised dual-guided autoencoder model. By adding a supervised loss in the pre-training stage, the model can learn data representation while optimizing for a specific prediction task, ensuring that the features learned by the model are directly related to the target monitoring or control task, thereby improving the task relevance and prediction accuracy of the model. However, since labeled data is difficult to obtain in large quantities, unsupervised learning is combined with supervised learning to optimize model parameters using limited labeled data and abundant unlabeled data, that is, to enhance the learning of quality-related features in the pre-training stage, thus solving the problem of scarce labeled data. An instant learning strategy is adopted to continuously optimize the model by fine-tuning parameters online, and data similar to the sample to be tested in the historical data set is selected to fine-tune the model. According to different samples, the local features and changes of the model are accurately captured, and finally the key quality variables of the industrial process are accurately and real-timely predicted. Summary of the Invention

[0005] The core objective of this invention is to provide an industrial soft-sensing method based on a semi-supervised dual-guided autoencoder model. This method not only efficiently captures features related to quality variables but also continuously learns from new data, rapidly adapting to dynamic changes in the production process, providing strong support for accurate monitoring and prediction of industrial processes.

[0006] The present invention specifically comprises the following steps: Step S1. Collect industrial process variable data to form a raw dataset. Divide the raw dataset into a training set and a test set. The training set contains labeled and unlabeled samples; the test set contains labeled samples. Labeled samples include process variables and their corresponding key quality variables, while unlabeled samples contain only process variables.

[0007] Step S2. A predictor is introduced on top of the deep stacked SAE to construct a dual-guided autoencoder (SSDGAE) model. The SSDGAE model consists of an encoder, a decoder, and a predictor. The encoder maps the input feature X to a latent space representation H, and the decoder restores the latent space representation H to the input space, achieving reconstruction. The predictor directly outputs the predicted value of the target variable from the latent space representation H.

[0008] Step S3. Pre-train each layer of the SSDGAE model constructed in step 2 layer by layer using the labeled and unlabeled data in the training set; Step S4. After the layer-by-layer pre-training is completed, the SSDGAE model is fine-tuned in a supervised manner using the labeled data in the training set, and the SSDGAE model is trained as a whole.

[0009] Step S5. Input the test set data into the supervised trained SSDGAE model, select data similar to the test sample from the historical database, and integrate the just-in-time learning strategy to perform online fine-tuning on the SSDGAE autoencoder trained in step S4 to obtain the JIT-SSDGAE model: Step S6: Input the sample to be tested into the JIT-SSDGAE model to obtain the corresponding predicted values of the key quality variables, thereby achieving the prediction task of the industrial process. At the same time, the current sample to be tested is added to the historical database, dynamically updating the historical database used for online fine-tuning in step S5.

[0010] Furthermore, the layer-by-layer pre-training in step S3 is specifically as follows: Labeled and unlabeled data are fed into the SSDGAE model in batches, and each autoencoder layer is trained. For unlabeled data, only the encoder and decoder components of the autoencoder are used to calculate the reconstruction error between the input and the reconstructed output, thereby learning an effective feature representation of the data. For labeled data, both the reconstruction error and the predicted value are calculated, and the prediction error between the predicted value and the true label is calculated. The final loss is the sum of the reconstruction error and the prediction error. The optimizer is used to minimize the final loss through backpropagation, thereby simultaneously optimizing the reconstruction and prediction capabilities of the SSDGAE model.

[0011] Furthermore, the supervised fine-tuning in step S4 is specifically as follows: First, preset the optimization criterion or the maximum number of iterations.

[0012] Then, the labeled training samples are input into the model that has completed layer-by-layer pre-training in batches. The input features are passed through the encoder of the SSDGAE model to obtain the hidden representation h, and then the predicted value is output through the predictor. At this stage, the model does not pass through the decoder, that is, the reconstruction loss is not calculated. Backpropagation is used to calculate the gradient of the prediction loss with respect to the model parameters. Based on the calculated gradient, the optimizer is used to update the model parameters.

[0013] Repeat the above steps until the preset standard or the maximum number of iterations is reached to obtain the trained SSDGAE model.

[0014] Preferably, in step S5, the most similar historical data point is selected based on the Euclidean distance.

[0015] Furthermore, in step S5, the fine-tuned JIT-SSDGAE model is verified using the data in the test set to verify the performance of the fine-tuned model.

[0016] The beneficial effects of the present invention are: Compared with the traditional soft measurement modeling method of stacked autoencoders, the learning of quality-related features in the pre-training stage is enhanced by minimizing the sum of the prediction error of the quality variable and the reconstruction error relative to the input variable, thereby reducing the reconstruction error of the autoencoder. It also enables the model to continuously learn from newly received data and quickly adapt to the dynamic changes of the production process. In response to the nonlinear and dynamically changing characteristics of current industrial process variables, the present invention introduces the SSDGAE autoencoder and combines it with an instant learning strategy, which makes up for the inability of existing soft measurement model solutions to accurately extract quality-related features and the difficulty in coping with time-varying problems. It effectively captures features related to quality variables, improves the prediction accuracy of key variables in industrial processes, and improves the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Flowchart of the method of the present invention in the embodiment; Figure 2 It is the structural diagram of the SSDGAE model in the present invention; Figure 3 This is the flow chart of the carbon dioxide absorption process in the application example; Figure 4 This is a comparison chart of the test results of the model of the present invention and the other three comparison models in the application example. DETAILED DESCRIPTION

[0018] The present invention will be described in detail below based on the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become more apparent. The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0019] like Figure 1 As shown in FIG, an industrial soft sensing method based on a semi-supervised dual-guide autoencoder model specifically includes the following steps: Step S1. Collect industrial process variable data and divide the data into labeled training sets and the corresponding critical quality variables , and an unlabeled training set These data sets are divided into training set and test set. In this embodiment, 75% of the collected data is used as training set and 25% is used as test set by holdout method.

[0020] Step S2. Introduce the predictor based on the deep stacked SAE to build Figure 2The dual-guided autoencoder (SSDGAE) model shown in the figure consists of an encoder, a decoder, and a predictor. The encoder maps the input feature X to a latent space representation H, and the decoder restores the latent space representation H to the input space, achieving reconstruction. The predictor directly outputs the predicted value of the target variable (such as regression or classification output) from the latent space representation H. By introducing the predictor and utilizing both reconstruction loss and prediction loss, dual-guided is achieved.

[0021] Step S3. Use the labeled and unlabeled data in the training set to pre-train each layer of the SSDGAE model constructed in step 2 layer by layer; specifically: Labeled and unlabeled data are fed into the SSDGAE model in batches, and each layer of the autoencoder is trained. For unlabeled data, only the encoder and decoder parts of the autoencoder are used to calculate the reconstruction error between the input and the reconstructed output to learn the effective feature representation of the data. For labeled data, both the reconstruction error is calculated and the predicted value is output by the predictor, and the prediction error between the predicted value and the true label is calculated. The final loss is the sum of the reconstruction error and the prediction error. .

[0022] Starting from the first layer, the autoencoder is trained layer by layer. After each layer is trained, its parameters are frozen and used for the training of the next layer. Each layer training includes reconstruction loss and prediction loss, and its loss function is calculated as follows: ; in, is a hyperparameter that balances the reconstruction loss and the prediction loss, is the reconstruction error relative to the input variables, is the prediction error of the quality variable, N is the total number of input samples in the current batch, For the The original eigenvalues of the input samples, No. The eigenvalues of the input samples reconstructed by the autoencoder, No. The true label value of the sample, No. The model predicts the label value of samples. The optimizer is used to minimize the , thereby simultaneously optimizing the reconstruction and prediction capabilities of the SSDGAE autoencoder.

[0023] Step S4. After the layer-by-layer pre-training is completed, the SSDGAE model is fine-tuned in a supervised manner using the labeled data in the training set. First, preset the optimization criterion or the maximum number of iterations.

[0024] Then, the labeled training samples (labeled data and the corresponding quality variables ) is input into the model that completes layer-by-layer pre-training in batches. The input features are passed through the encoder of the SSDGAE model to obtain the hidden representation h, and then the predicted value is output through the predictor At this stage, the model does not pass through the decoder, that is, the reconstruction loss is not calculated. The loss between the calculated prediction result and the label is expressed as: ;in, is the true label, The model predicts the value. Backpropagation is used to calculate the gradient of the prediction loss with respect to the model parameters. Based on the calculated gradient, the optimizer is used to update the model parameters.

[0025] Repeat the above steps until the performance on the validation set reaches the preset standard or the maximum number of iterations is reached, and the trained SSDGAE model is obtained.

[0026] Step S5. Use the variable data in the test set as the test samples, select data similar to the test samples from the historical database, and integrate the Just-In-Time Learning strategy to perform online fine-tuning on the SSDGAE model trained in Step S4 to obtain the JIT-SSDGAE model: In this embodiment: First, the labeled data in the training set is used as the initial historical database: ;in, is the eigenvector, is the corresponding label.

[0027] Then, the most similar historical data point is selected based on the Euclidean distance, which is calculated as follows: in is the potential representation of the query data, is the potential representation of the i-th historical data point, and d is the dimension of the latent space.

[0028] Finally, based on the calculation results, the historical data most similar to the sample to be tested is selected to perform online fine-tuning on the JIT-SSDGAE model.

[0029] The JIT-SSDGAE model obtained by online fine-tuning is verified using the data in the test set. Input the fine-tuned JIT-SSDGAE model and output the predicted value set , the predicted value set Test set with key variables in the test set Perform a comparison to verify the performance of the model after online fine-tuning.

[0030] In order to judge the prediction ability of the soft sensor model, the R 2 (R-Square), minimum root mean square error RMSE (Root Mean Square Error), mean absolute error MAE (Mean Absolute Error) and mean absolute percentage error MAPE (Mean Absolute Percentage Error). 2 , RMSE, MAE, and MAPE are defined as follows: ; ; ; ; Where, represents quality prediction information, represents the output after inputting the test data into the model, express The average value of Indicates the number of samples in the test set.

[0031] In actual operation, the JIT-SSDGAE model after online fine-tuning is stored as a local model.

[0032] Step S6: Input the test sample into the local model to obtain the corresponding predicted values of the key quality variables, thereby achieving industrial process prediction. Simultaneously, the current test sample is added to the historical database, dynamically updating the historical database used for fine-tuning in step S5. The local model parameters are reset, and the next prediction is performed by repeating steps S5 and S6.

[0033] This paper addresses the limited number of labels and non-dynamic nature of actual process samples by innovatively introducing a semi-supervised learning approach and combining it with a real-time learning strategy to effectively improve the performance of predicting quality variables. Compared with other traditional dynamic deep learning soft sensor model prediction methods, this paper's advantages lie in its ability to accurately capture the characteristics and changes of different test samples, the ability to continuously optimize the model through online parameter fine-tuning, and the ability to more deeply explore data features. This reduces prediction errors and improves the model's performance in predicting quality variables.

[0034] Application Examples The following is as follows Figure 3The quality variable prediction for the carbon dioxide absorption tower shown demonstrates the performance of the present invention. The carbon dioxide absorption tower is a crucial component in the ammonia synthesis process. One of its primary tasks is to reduce the carbon dioxide (CO2) content in the process gas from the preceding unit. Direct measurement of the CO2 content in process gas requires subsequent experimental analysis, which results in a time lag, making it difficult to detect problems promptly and hindering real-time industrial process control. The carbon dioxide absorption process has 11 process variables (U1-U11) and one key variable (Y). The specific measured variables are described in Table 1.

[0035] Table 1 The specific prediction steps are as follows: Step 1. For the process of measuring the carbon dioxide content in the process gas of a carbon dioxide absorption tower, 6,000 samples of industrial process variables and key industrial process variables were collected. 1,500 data points were selected as the training set, 500 data points as the test set, and the number of similar samples, N, was set to 50.

[0036] For training and test sets The data were standardized and normalized to make their mean 0 and standard deviation 1.

[0037] Step 2. Set the number of neurons in each layer of the SSDGAE model to [11, 9, 7, 9, 11, 1], where 11 corresponds to the number of input variables in Table 1, and 1 corresponds to the number of output variables. The number of hidden neurons in the second, third, fourth, and fifth layers is 9, 7, 9, and 11, respectively. Use the sigmoid activation function for each neuron.

[0038] Step 3. Set the model training hyperparameters: learning reat=0.03, epoch=500, batchsize=100. Use the labeled and unlabeled data in the training set to pre-train each layer of the SSDGAE model. Step 4. After completing layer-by-layer pre-training, perform supervised fine-tuning on the SSDGAE model using labeled data from the training set, completing overall training of the SSDGAE model. The optimal model is then tested on the test set to verify the model and complete the development of a prediction model for key variables in the CO2 absorption tower.

[0039] Step 5. The process variables of the CO2 absorption process in the CO2 absorption tower are used as test samples. Whenever a new process variable arrives, the N most similar samples are retrieved from the historical database. These similar samples are used to quickly fine-tune the model online to obtain the JIT-SSDGAE model, enabling the model parameters to dynamically adapt to changes in the current operating conditions.

[0040] Step 6. Input the process variables of the current CO2 absorption process into the JIT-SSDGAE model, and output the predicted values of key variables under the current operating conditions, achieving dynamic, real-time prediction of key variables in the CO2 absorption process. Simultaneously, the process variables of the current CO2 absorption process are added to the historical database, which is dynamically updated to further enhance the model's adaptive capabilities.

[0041] In order to verify the prediction effect of the method of the present invention, the same samples are used to train four models: JIT-SSDGAE model, SSSAE, SSDGAE, and JIT-SGDAE. Among them, SSSAE is a semi-supervised stacked autoencoder without a dynamic adaptive mechanism and the modeling process uses both labeled data and unlabeled data; SSDGAE is a semi-supervised stacked dual-guided autoencoder without a dynamic adaptive mechanism, and the modeling process uses both labeled data and unlabeled data; JIT-SDGAE is a supervised stacked dual-guided autoencoder with dynamic adaptation, and the modeling process only uses labeled data. JIT-SSDGAE is a semi-supervised stacked dual-guided autoencoder with a dynamic adaptive mechanism, that is, the modeling process uses both labeled data and unlabeled data. The trained models are tested on the same test set for the four models. The test results are shown in Tables 2 and Figure 4 As shown in the figure, the test results show that the method of the present invention further improves the performance of the prediction model, and its output is closer to the true value. The prediction error is smaller and the effect is better.

[0042] Figure 4 The overall prediction trend of the SSSAE model matches, but the prediction fluctuation of the 200-300 sample points is gentler than the true value; the SSDGAE model predicts systematically lower values in the 300-400 interval, showing obvious deviations; the JIT-SDGAE model performs unstable at the tail, and the error of the 400-500 sample points increases; in comparison, the JIT-SSDGAE model has the best overall fit, especially in the 0-300 interval, which is almost overlapping.

[0043] Table 2 Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art will still be able to modify the technical solutions described in the foregoing examples or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the invention shall be included within the scope of protection of the invention.

Claims

1. An industrial soft sensing method based on a semi-supervised dual-guide autoencoder model, characterized by: The specific steps include: Step S1. Collect industrial process variable data to form a raw dataset; divide the raw dataset into a training set and a test set; the training set contains labeled samples and unlabeled samples; the test set contains labeled samples; the labeled samples contain process variables and their corresponding key quality variables, and the unlabeled samples contain only process variables; Step S2. Based on the deep stacked SAE, a predictor is introduced to construct a dual-guided autoencoder model (SSDGAE). The SSDGAE model consists of an encoder, a decoder, and a predictor. The encoder maps the input feature X to a latent space representation H, and the decoder restores the latent space representation H to the input space to achieve reconstruction. The predictor directly outputs the predicted value of the target variable from the latent space representation H. Step S3. Pre-train each layer of the SSDGAE model constructed in step 2 layer by layer using the labeled and unlabeled data in the training set; Step S4. After completing layer-by-layer pre-training, perform supervised fine-tuning on the SSDGAE model using labeled data from the training set, and train the SSDGAE model as a whole. Step S5. Take any process variable data in the industrial process as the test sample, select data similar to the test sample in the historical database, and integrate the just-in-time learning strategy to perform online fine-tuning on the SSDGAE autoencoder trained in step S4 to obtain the JIT-SSDGAE model: Step S6. Input the test sample into the JIT-SSDGAE model to obtain the corresponding predicted values of the key quality variables, thereby realizing the prediction of the industrial process; at the same time, the current test sample is added to the historical database, and the historical database used for online fine-tuning in step S5 is dynamically updated.

2. The industrial soft sensing method based on the semi-supervised dual-guided autoencoder model according to claim 1, wherein: The layer-by-layer pre-training in step S3 is specifically as follows: Labeled and unlabeled data are fed into the SSDGAE model in batches to train each autoencoder layer. For unlabeled data, only the encoder and decoder parts of the autoencoder are used to calculate the reconstruction error between the input and the reconstructed output to learn an effective feature representation of the data. For labeled data, the reconstruction error is calculated, and the predicted value is output through the predictor, and the prediction error between the predicted value and the true label is calculated; the final loss is the sum of the reconstruction error and the prediction error; The optimizer is used to minimize the final loss through back-propagation, thereby simultaneously optimizing the reconstruction and prediction capabilities of the SSDGAE model.

3. The industrial soft sensing method based on the semi-supervised dual-guided autoencoder model according to claim 1, wherein: The supervised fine-tuning in step S4 is specifically as follows: First, preset the optimization criteria or maximum number of iterations; Then, the labeled training set is input into the model that has completed layer-by-layer pre-training in batches. The input features are passed through the encoder of the SSDGAE model to obtain the hidden representation h, and then the predicted value is output through the predictor. At this stage, the model does not pass through the decoder, that is, the reconstruction loss is not calculated; backpropagation is used to calculate the gradient of the prediction loss with respect to the model parameters, and based on the calculated gradient, the optimizer is used to update the model parameters; Repeat the above steps until the preset standard is reached or the maximum number of iterations is reached to obtain the trained SSDGAE model.

4. The industrial soft sensing method based on the semi-supervised dual-guided autoencoder model according to claim 1, wherein: In step S5, the most similar historical data point is selected based on the Euclidean distance.

5. The industrial soft sensing method based on the semi-supervised dual-guided autoencoder model according to claim 1, wherein: In step S5, the fine-tuned JIT-SSDGAE model is verified using the data in the test set to verify the performance of the fine-tuned model.

Images