A soft sensing method based on local variable-preserving weighted autoencoder

Abstract

The present invention discloses a soft measurement method based on a local variable-retaining weighted autoencoder, which adopts the structure of a stacked autoencoder; by setting a quality-related weighting coefficient for input variable features and introducing local spatial structure information between samples into an AE loss function, local retention of the spatial relationship of neighboring samples is achieved, the feature extraction capability of the autoencoder is improved, and then the features are transferred to a strong regressor for regression prediction. This method utilizes the spatial structure information between samples and improves prediction accuracy.

Description

Technical Field

[0001] The invention belongs to the field of industrial processes and relates to a soft measurement method based on a local reserved variable weighted autoencoder. Background Art

[0002] In modern industrial processes, timely monitoring and control of process performance are of great significance for achieving green production, energy saving and consumption reduction, and improving enterprise benefits. Process key quality variables are important indicators of process performance. Timely and accurate monitoring and analysis of some process key quality variables are important guarantees and necessary prerequisites for realizing the operating status of the monitoring system, achieving stable process control, and producing high-quality products. However, due to the harsh measurement environment, expensive instrument costs, and large measurement delays, key quality variables cannot or are difficult to be measured in real time and accurately. In modern industrial processes, soft measurement sensors estimate key quality variables by establishing mathematical models and using easily measurable auxiliary variables, thereby providing important real-time information for process monitoring, optimization, and control. Compared with traditional hard measurement methods, soft measurement has many advantages such as rapid response, low cost, diverse methods, easy maintenance, and safe operation, which has enabled it to be successfully applied in a large number of industrial fields such as chemical, biochemical, metallurgical, and pharmaceutical industries.

[0003] Soft sensor sensors can generally be divided into two categories: process mechanism modeling-based methods and data-driven modeling methods. The process mechanism modeling-based methods are suitable for processing steady-state operating data and rely on the physical and chemical background knowledge of the process. However, in actual industrial processes, there is a lack of sufficient understanding of the mechanism of process reaction characteristics and a lack of sufficient expert knowledge, which makes it impossible to establish an accurate model. These reasons have limited the application and promotion of mechanism modeling-based methods to a certain extent. With the widespread use of distributed control systems, a large amount of sensor data can be recorded, collected and analyzed, which makes data-driven modeling soft sensor sensors possible and widely used in practice. Data-driven soft sensor sensors do not require prior process mechanism knowledge or operating experience, and the prediction model they establish is completely based on process historical data.

[0004] Commonly used data-driven soft measurement methods include PCR, PLS, KPCR, KPLS, SVM, etc. Neural network methods are currently the most commonly used methods for dealing with nonlinearity. Multi-layer deep neural networks can more effectively express the characteristics of highly complex systems. In the field of soft measurement, commonly used deep learning methods include deep belief networks (DBN), autoencoders (AE), convolutional neural networks (CNN), and recurrent neural networks (RNN). Among them, autoencoders are the most widely used neural networks. It converts input variables into hidden variables through encoders, and then restores hidden variables to input variables through decoders. Both encoders and decoders are implemented through neural networks. Through this encoding and decoding process, the neural network can learn the internal associations between data, which gives the autoencoder a powerful feature extraction capability and is widely used in industrial process soft measurement modeling. It has the characteristics of unsupervised learning and strong scalability. Industrial process data usually contains a large number of auxiliary variables, that is, it has high-dimensional characteristics. Based on the popular hypothesis, the auxiliary variables are on a low-dimensional manifold in the high-dimensional space that maintains the local Euclidean characteristics. Therefore, it is necessary to consider retaining the structural information of the samples in the low-dimensional space. However, the existing autoencoders do not fully consider the spatial structural information between samples, which easily leads to low prediction accuracy. Summary of the invention

[0005] In view of the deficiency that the existing autoencoder does not consider the spatial structure information between samples, the present invention proposes a soft measurement method based on the local variable retention weighted autoencoder, which introduces the local spatial structure information between samples into the AE loss function, improves the feature extraction ability of the autoencoder by weighting the quality of the input variable features and locally retaining the spatial information of neighboring samples. The features are then transferred to the strong regressor for regression prediction.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A soft measurement method based on a local variable-retaining weighted autoencoder, characterized in that it comprises the following steps:

[0008] Step 1: Collect historical industrial process data to establish a training data set, which includes key quality variables y and auxiliary variables x. where d x is the dimension of the input vector;

[0009] Step 2: Perform normalization preprocessing on the training data set so that the result value is mapped between 0 and 1;

[0010] Step 3: Construct a local variable-preserving weighted autoencoder, which includes an encoder and a decoder. Both the encoder and the decoder are single-layer neural networks. The encoder maps x from the input layer to the hidden layer through a nonlinear function, and the decoder reconstructs the input through another nonlinear transformation. The specific formula is as follows:

[0011] h=Encode(x)=f e (W e ·x+b e )

[0012]

[0013] in, is the hidden layer feature, d h is its dimension; W e is the weight coefficient matrix, b e is the corresponding offset; f e (·) represents a nonlinear activation function;

[0014] The parameter set to be optimized for the entire local variable-retaining weighted autoencoder is θ = {W e ,b e ,W d ,b d}, the loss function is designed as follows:

[0015]

[0016]

[0017]

[0018] where ρ (d) is the correlation coefficient between the d-th dimension input variable and the label variable, Cov(x (d) ,y) is x (d) , the covariance of y, Var(x (d) ), Var(y) is x (d) , y’s respective variances; λ is the trade-off coefficient, which is used to adjust the weight of the regularization term; N is the number of samples; v ij To constrain the two hidden layer feature samples h i 、h j The weight of the proximity between them, σ is an adjustable hyperparameter that controls the rate at which the weight decreases with distance; N k (x i ) indicates that x i The k nearest neighbor samples;

[0019] Step 4: Use the preprocessed training data set to train the local variable-retaining weighted autoencoder;

[0020] In the first iteration, the encoder maps x from the input layer to the hidden layer, and then performs autoencoder training; each subsequent iteration uses the hidden layer features obtained in the previous iteration as the input of the encoder of the current iteration, and performs autoencoder training, thereby achieving layer-by-layer stacking; considering that each hidden layer has effective information in AE feature extraction, each hidden layer is concatenated as the feature extracted by the entire local retained variable weighted autoencoder, and the concatenated dimensionality reduction features and key quality variables y are transferred to the strong regressor to train the strong regressor;

[0021] Step 5: After normalizing the new industrial process data, input it into the trained local variable-retaining weighted autoencoder, concatenate its hidden layers, and input it into the trained strong regressor to obtain the predicted output.

[0022] Furthermore, in step 2, the Min-Max normalization method is used to perform normalization preprocessing on the data set.

[0023] Furthermore, the strong regressor is selected from any one of XGBoost, SVR, and Ridge. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The present invention is a flowchart of a soft measurement method based on a local variable-retaining weighted autoencoder.

[0025] Figure 2 Schematic diagram of the single-layer autoencoder structure.

[0026] Figure 3 Schematic diagram of the stacked autoencoder structure.

[0027] Figure 4 This is a flow chart of the debutanizer tower. DETAILED DESCRIPTION

[0028] The present invention will be described in detail below based on the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more clear. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] The soft measurement method based on the local variable-retaining weighted autoencoder proposed in the present invention has a flow chart as shown in FIG. Figure 1 As shown, the following steps are included:

[0030] Step 1: Collect historical industrial process data to establish a training data set, which includes key quality variables y and auxiliary variables x. where d x is the dimension of the input vector.

[0031] Step 2: Use the Min-Max normalization method to normalize the data set so that the result value is mapped between 0 and 1, and then divide it into the training set Dtrn and the test set Dtest

[0032] Step 3: Construct a local variable-preserving weighted autoencoder (LP-VWAE);

[0033] The goal of AE is to learn a function Γ θ (x) makes like Figure 2 As shown in the figure, the local variable-retaining weighted autoencoder includes an encoder and a decoder, both of which are single-layer neural networks; the encoder maps x from the input layer to the hidden layer through a nonlinear function, h = Encode (x) = f e (W e ·x+b e ),in, is the hidden layer feature, d h is its dimension; W e is the weight coefficient matrix, b e is the corresponding offset; f e (·) represents a nonlinear activation function. After encoding the hidden layer feature h, the decoder reconstructs the input through another nonlinear transformation, as shown in the following formula:

[0034] The parameter set to be optimized in the entire autoencoder AE model is θ = {W e ,b e ,W d ,b d}. The loss function is designed to extract input features that are highly correlated with label variables, and to assign different weights to variables of different dimensions in the reconstructed object; and based on the local consistency assumption, a local preservation regularization term is added to form a local preservation variable weighted autoencoder. The loss function of LP-VWAE is designed as follows:

[0035]

[0036] Where λ is the trade-off coefficient, which is used to adjust the weight of the regularization term; N is the number of samples; ρ (d) is the correlation coefficient between the d-th dimension input variable and the label variable, which is calculated as follows:

[0037]

[0038] Where Cov(x (d) ,y) is x (d) , the covariance of y, Var(x (d) ), Var(y) is x (d) , the variance of y;

[0039] v ij To constrain the two hidden layer feature samples h i 、h j The weight of the closeness between them is calculated as follows:

[0040]

[0041] Where σ is an adjustable hyperparameter that controls the rate at which the weight decreases with distance; N k (x i ) indicates that x i The k nearest neighbor samples.

[0042] Step 4: Use the preprocessed training data set to train the local variable-retaining weighted autoencoder;

[0043] In the first iteration, the encoder maps x from the input layer to the hidden layer, and then performs autoencoding training; each subsequent iteration uses the hidden layer features obtained in the previous iteration as the input of the encoder of the current iteration, and performs autoencoding training, thereby achieving layer-by-layer stacking; Figure 3 As shown in the figure, considering that each hidden layer in AE feature extraction has effective information, each hidden layer is concatenated as the extracted features, and the concatenated dimensionality reduction features and key quality variables y are transferred to the strong regressor to train the strong regressor.

[0044] Step 5: After normalizing the new industrial process data, input it into the trained local variable-retaining weighted autoencoder, concatenate its hidden layers, and input it into the trained strong regressor to obtain the prediction output. The strong regressor can be any one of XG-boost, SVR, and Riege.

[0045] The performance of the soft sensing method of the present invention is explained below in conjunction with two specific application examples.

[0046] 1. Debutanizer

[0047] A debutanizer is a fractionation tower that is primarily used to separate butane from natural gas or naphtha streams during the refining process. Figure 4 As shown. The goal of the debutanizer is to minimize the butane content, but the measurement of butane content requires a lot of time and cost. In order to strengthen the quality control of industrial processes, it is necessary to develop and apply quality variable prediction models. Seven variables were collected in this process, which are maximum temperature, maximum pressure, reflux flow, etc. The data set has a total of 2394 samples. This example uses the first 2 / 3 of the data to learn the model parameters, and uses the last 1 / 3 of the data as a test set.

[0048] The prediction results of the strong regressor, basic AE model, VWAE model and LP-VWAE model are compared. The three autoencoders all adopt a two-layer stacked model structure, with 12 neurons in the first hidden layer and 4 neurons in the second hidden layer. The strong regressor selects Ridge regression and SVR support vector regression. The hyperparameters of LP-VWAE are optimized by grid search, and the parameters are selected as: λ=1, σ=1.5, k=6. Table 1 lists the prediction results of the test set under the above four methods. It can be seen that the LP-VWAE model has the smallest mean square error RMSE and R 2 The best results were achieved in both indicators.

[0049] Table 1 Prediction results of soft sensor model

[0050]

[0051] 2. NO in flue gas from SCR inlet of a power plant x concentration

[0052] A 1000MW coal-fired unit in Taizhou, Zhejiang Province was selected as the research object. x The concentration is mainly affected by the operating conditions of the boiler. Under normal combustion temperatures, NO generated by coal combustion is x The content in flue gas is less than 1%, of which NO accounts for NO x Of these, more than 90% is produced by nitrogen dioxide, 5% to 10% is produced by NO2, and only about 1% is produced by N2O.

[0053] By the furnace outlet NO x The analysis of the generation mechanism is combined with the research in the relevant literature. The main auxiliary variables selected are: unit load, total coal volume, total primary air volume, total secondary air volume, flue gas oxygen content and furnace outlet temperature. The key quality variable is the SCR inlet NO on the A side. x concentration.

[0054] The historical operation data of the load in the process of 400MW to 520MW was selected, which lasted for nearly 6 and a half hours. A point was collected every 1 minute, totaling 600 sets of data points, which were basically stable low-load operation conditions, and more comprehensively reflected the characteristics of the research object under low-load steady-state conditions. The data of each variable was taken from the power plant-level monitoring information system (SIS). This embodiment uses the first 2 / 3 of the data to learn the model parameters, and uses the last 1 / 3 of the data as the test set.

[0055] The prediction results of the strong regressor, basic AE model, VWAE model and LP-VWAE model are compared. The three autoencoders all adopt a two-layer stacked model structure, with 9 neurons in the first hidden layer and 6 neurons in the second hidden layer. The strong regressor selects Ridge regression. The hyperparameters of LP-VWAE are optimized by grid search, and the parameters are selected as: λ=20, σ=8, k=2. Table 2 lists the prediction results of the test set under the above four methods. It can be seen that the LP-VWAE model has the smallest mean square error RMSE and R 2 The best results were achieved in both indicators.

[0056] Table 2 Prediction indicators of each model

[0057] Model Ridge AE+Ridge VWAE+Ridge LP-VWAE+Ridge RMSE 0.0724 0.0693 0.0664 0.0592 <![CDATA[R 2 ]]> 0.7556 0.7757 0.7945 0.8300

[0058] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention shall be included in the protection scope of the invention.

Claims

1. A soft measurement method based on a local variable-retaining weighted autoencoder, characterized in that: The steps include: Step 1: Collect historical industrial process data to establish a training data set, which includes key quality variables y and auxiliary variables x. where d x is the dimension of the input vector; Step 2: Perform normalization preprocessing on the training data set so that the result value is mapped between 0 and 1; Step 3: Construct a local variable-preserving weighted autoencoder, which includes an encoder and a decoder. Both the encoder and the decoder are single-layer neural networks. The encoder maps x from the input layer to the hidden layer through a nonlinear function, and the decoder reconstructs the input through another nonlinear transformation. The specific formula is as follows: h=Encode(x)=f e (W e ·x+b e ) in, is the hidden layer feature, d h is its dimension; W e is the weight coefficient matrix, b e is the corresponding offset; f e (·) represents a nonlinear activation function; The parameter set to be optimized for the entire local variable-retaining weighted autoencoder is θ = {W e ,b e ,W d ,b d }, the loss function is designed as follows: where ρ (d) is the correlation coefficient between the d-th dimension input variable and the label variable, Cov(x (d) ,y) is x (d) , the covariance of y, Var(x (d) ), Var(y) is x (d) , y’s respective variances; λ is the trade-off coefficient, which is used to adjust the weight of the regularization term; N is the number of samples; v ij To constrain the two hidden layer feature samples h i 、h j The weight of the proximity between them, σ is an adjustable hyperparameter that controls the rate at which the weight decreases with distance; N k (x i ) indicates that x i The k nearest neighbor samples; Step 4: Use the preprocessed training data set to train the local variable-retaining weighted autoencoder; In the first iteration, the encoder maps x from the input layer to the hidden layer, and then performs autoencoder training; each subsequent iteration uses the hidden layer features obtained in the previous iteration as the input of the encoder of the current iteration, and performs autoencoder training, thereby achieving layer-by-layer stacking; considering that each hidden layer has effective information in AE feature extraction, each hidden layer is concatenated as the feature extracted by the entire local retained variable weighted autoencoder, and the concatenated dimensionality reduction features and key quality variables y are transferred to the strong regressor to train the strong regressor; Step 5: After normalizing the new industrial process data, input it into the trained local variable-retaining weighted autoencoder, concatenate its hidden layers, and input it into the trained strong regressor to obtain the predicted output.

2. The soft measurement method based on the local variable-retaining weighted autoencoder according to claim 1 is characterized in that: In the step 2, the Min-Max normalization method is used to normalize the data set.

3. The soft measurement method based on the local variable-retaining weighted autoencoder according to claim 1 is characterized in that: The strong regressor is selected from any one of XGBoost, SVR, and Ridge.

Images