Soft-sensing modeling method based on maximum relevance minimum redundancy stack auto-encoder

Abstract

The application discloses a kind of based on maximum correlation minimum redundancy's stacked auto-encoder soft measurement modeling method, the method is aimed at the low correlation between traditional deep learning and output, the information redundancy of hidden feature is high, etc., a new kind of based on maximum correlation minimum redundancy's stacked auto-encoder soft measurement modeling method is proposed.Specifically, the weight is assigned to the extracted hidden feature, according to the influence of different weights on output result, the hidden feature with greater influence on output is selected to give higher weight;For between hidden features, the principal component analysis method is used to remove the redundancy between hidden features.The method selects the input feature with strong correlation with the output variable while analyzing and removing redundant information, so as to realize model structure reduction.Compared with the conventional soft measurement method, the method fully considers the correlation between hidden features and output variables, and shows significant advantages in model prediction performance.

Description

TECHNICAL FIELD

[0001] The present application relates to a method in the field of soft measurement of industrial processes, in particular a soft measurement modeling method based on maximum correlation minimum redundancy stacked autoencoder. BACKGROUND

[0002] In recent years, with the rapid development of advanced science and technology, industrial production processes such as metallurgy, petroleum, chemical industry, food manufacturing, biomedicine, etc. have changed significantly, and higher requirements have been put forward for process control stability, detection timeliness, operation reliability and other indicators. In order to effectively monitor the running state of the system, realize the stable control of the process and the rapid tracking of product quality, the detection of key product quality and parameters of the process is particularly important. However, in actual industrial production processes, due to complex physical, chemical, biological reactions and other factors, the system often shows serious complexity, uncertainty and nonlinearity, making it extremely difficult to obtain key variables in time. In addition, due to the complex and harsh production environment, the high cost of analytical instruments and the lagging nature of analysis time, it is impossible to detect the key quality variables on-line, thereby affecting the real-time and effective control of the process. Soft measurement technology is gradually developed to solve the real-time measurement and control of such quality variables. The basic idea is to select a group of easily measured auxiliary variables closely related to the difficult-to-measure dominant variables as input variables according to certain optimization criteria, and to estimate the dominant variables by constructing a mathematical relationship and using computer software.

[0003] At present, most researches mainly focus on the structure and training method of the model, and pay little attention to the correlation between the training data and the output. Generally speaking, the training data highly correlated with the output can improve the prediction accuracy of the model. There have been many achievements on the preprocessing of data sets, such as data dimensionality reduction, data denoising, outlier detection, missing value detection, etc., but the research on correlation analysis is still limited.

[0004] In the field of deep learning soft measurement modeling, the idea of constructing a soft measurement model based on autoencoder and its variants is the most common, but the conventional autoencoder completely relies on sample input in the unsupervised pre-training stage, and the model learning ignores the correlation between input and output, which makes it difficult to effectively represent the internal relationship between input and output. Based on the stacked autoencoder framework, the loss function of the traditional autoencoder is expanded into a variable weight loss function, which is used for soft measurement. The method of the present application can extract input-output related features and eliminate redundant information, giving a complete soft measurement model structure, so that the soft measurement result is more reliable. SUMMARY

[0005] The present application aims to solve the problem that model learning ignores the correlation between input and output in process industry application, resulting in that the extracted hidden features are difficult to effectively represent the inherent relationship between input and output, and because there is redundancy between hidden features, a soft measurement modeling method based on maximum correlation minimum redundancy stacked autoencoder is proposed. The method not only selects input features with strong correlation with output variables, but also uses principal component analysis to remove redundant information, which can effectively improve the above problems and improve the performance of industrial soft measurement model.

[0006] The present application assigns weights to the extracted hidden features, selects hidden features with greater impact on output by assigning higher weights according to the influence of different weights on output results, and removes the redundancy between hidden features by using principal component analysis. The method not only selects input features with strong correlation with output variables, but also analyzes and removes redundant information, thereby realizing model structure reduction. Compared with conventional soft measurement methods, the method fully considers the correlation between hidden features and output variables, and shows significant advantages in model prediction performance.

[0007] To solve the above technical problems in the modeling process, the technical scheme of the present application is as follows: a soft measurement modeling method based on maximum correlation minimum redundancy stacked autoencoder (referred to as P-VW-SAE), the main steps are as follows:

[0008] Step (1) Collect industrial process data through distributed control system or offline detection method, determine key quality variables y and auxiliary variables x, wherein x=[x1, x2, …, x d ]∈R d represents a vector composed of d auxiliary variables;

[0009] Step (2) Preprocess and normalize the collected industrial process data to obtain a data set with minimum value 0 and maximum value 1, the new data matrix set is input data set and target data set Divide the data set into training set D trn and test set D test ;

[0010] Step (3) Construct a soft measurement model of variable weight autoencoder for the normalized soft measurement model input and output data set, which is used to extract output related hidden features;

[0011] Step (4) Use principal component analysis to remove the redundancy between hidden features using the hidden features extracted in step (3);

[0012] Step (5) The redundant hidden features removed by principal component analysis in step (4) are taken as input, and steps (3) and (4) are repeated multiple times to build a maximum correlation minimum redundancy stacked autoencoder soft measurement model.

[0013] Step (6) After pre-training, the output layer is added to the top of the stacked autoencoder (SAE) to fine-tune the weights.

[0014] Step (7) Build a regression model on the training set, and test it on the test set D test to finally realize the offline estimation of the key quality variables.

[0015] Further, the soft measurement model building process of the variable weight de-redundancy autoencoder in step (3) is as follows:

[0016] (1) In order to train the variable weight autoencoder, first, the labeled data should be used to obtain the variable weight; suppose the labeled training data

[0017] Where X l represents the labeled input data set, Y l represents the labeled target data set, N l represents the number of labeled samples, represents the N l th labeled input data, represents the N l th labeled target data.

[0018] (2) Determine the correlation between the input variable and the target variable, and calculate the correlation coefficient of the dth variable through the labeled data:

[0019]

[0020] Where X l(d) is the dth variable of the labeled data set;

[0021] The covariance and variance are calculated as follows:

[0022]

[0023]

[0024]

[0025] Where and respectively, if the absolute value of the correlation coefficient is large, it means that the input variable is more related to the target variable; therefore, the variable should be given a larger weight value, and vice versa; the weight of the dth input variable should be a monotonically increasing function, and the correlation coefficient is the absolute value; the variable weight used is set to

[0026]

[0027] where λd (d) is the weight of the dth variable.

[0028] (3) Using variable weights, the stacked autoencoder is trained in a supervised manner

[0029]

[0030] where W represents the weight matrix of the encoding layer, v represents the bias vector of the encoding layer, represents the weight matrix of the decoding layer, represents the bias vector of the decoding layer.

[0031] Further, the principal component analysis process in step (4) is:

[0032] After extracting the hidden features using the autoencoder (AE), the correlation coefficient between any two hidden features is represented by C = (i,j = 1,2,…,p) according to the covariance matrix C = (i,j = 1,2,…,p) of the hidden features; then the eigenvalue decomposition formula of the covariance matrix is used ij

[0033]

[0034] where λi(i∈1,2,…,d) represents the eigenvalue of the diagonal matrix.

[0035] The λ i is arranged in descending order, the number of hidden layer nodes before dimensionality reduction by principal component analysis is g, and the first s principal components when the cumulative contribution degree is ρ≥95% are selected, g≥s.

[0036] Further, the process of establishing the maximum correlation and minimum redundancy stacked autoencoder soft measurement model in step (5) is:

[0037] 1) Use the labeled data set {X l , Y l} to train the first variable weight autoencoder in a supervised manner, and the pre-trained weight is {W1,b1}, and the hidden feature of the first layer is

[0038] 2) Assuming that the number of hidden features is d, the hidden features in step 1) are arranged in descending order to obtain​​ Select the first d' hidden features, so that the corresponding accumulation contribution rate satisfies p>95%;

[0039] 3) According to the screening result of step 2), the nodes in the hidden layer which are not selected are removed;

[0040] 4) Repeat the process of step 2) and step 3), and pre-process layer by layer until the last variable weight autoencoder is obtained; all the variable weight autoencoder pre-training weights are {W k ,b k} k＝1,2,…,L ;

[0041] 5) After pre-training, add the output layer to the top of the stacked autoencoder to fine-tune the weights; the pre-trained weights {W k ,b k} k＝1,2,…,L are used to initialize the weights of each hidden layer; the improved weights {W' k ,b k} k＝1,2,…,L .

[0042] Compared with the prior art, the beneficial effects of the present application are: the soft measurement modeling method of the stacked autoencoder based on maximum correlation and minimum redundancy proposed in the present application considers the problems of low correlation between hidden features and output, high redundancy of hidden feature information and the like in traditional deep learning, and introduces a soft measurement model combining variable weight stacked autoencoder and PCA (principal component analysis). The model simultaneously learns the output-related hidden features and simultaneously realizes structural reduction of the hidden features. Considering that not all hidden features can improve the prediction performance of the model, different weights are given to the hidden features, the weight of the hidden feature which has greater influence on the output is greater, so as to realize maximum correlation. PCA is used between the hidden features to remove redundant hidden features, effectively ensuring that the redundancy of the model is minimized. The finally constructed soft measurement method presents significant advantages in model prediction performance. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 It is the principle diagram of the P-VW-SAE soft measurement modeling method in the present application;

[0044] Figure 2 It is the P-VW-SAE model structure diagram;

[0045] Figure 3 It is the curve diagram of the real value and the predicted value of the butane concentration of the stacked autoencoder regression soft measurement model;

[0046] Figure 4 It is the curve diagram of the real value and the predicted value of the butane concentration of the variable weight stacked autoencoder soft measurement model;

[0047] Figure 5 A curve of a true value and a predicted value of the soft measurement model of the butane concentration in the application. DETAILED DESCRIPTION

[0048] The embodiments of the application will be described in detail below with reference to the flowchart of the technical scheme, and the protection scope of the application is not limited to the present case.

[0049] Embodiment 1

[0050] The de-butanizer is an important device for separating desulfurization and naphtha in the industrial oil refining process. In order to effectively improve the desulfurization efficiency of the butane concentration (C4), a soft measurement method is used to predict the butane concentration online. According to the analysis, the seven monitoring variables shown in Table 1 are used as auxiliary variables for constructing the soft measurement model, and the output variable is the C4 concentration.

[0051] Table 1 auxiliary variable description

[0052]

[0053] Since the butane concentration is indirectly measured in the overhead distillate of the sequential de-isopentane column, rather than directly measured in the butane concentration at the bottom of the de-butanizer, the measurement result has a time delay. Therefore, the present application introduces a delay variable in the original seven variables to solve this problem, and the new input variable is constructed as follows:

[0054] x'=[x(t) T ,…,x(t-l) T ] T

[0055] Through the above formula, 35 new input variables are obtained. Among them, t represents the tth variable; l represents the maximum delay time, and in the present application, l = 4. A total of 2394 data samples are obtained, wherein the training set (1000 samples) is used for training the model, and the test set (1394 samples) is used for testing the model performance.

[0056] The specific implementation steps are as follows:

[0057] (1) Collect the industrial process data by the distributed control system or offline detection method to determine the key quality variable y and the auxiliary variable x, wherein x = [x1, x2, …, x d ]∈R d represents a vector composed of d auxiliary variables.

[0058] (2) Preprocess and normalize the collected industrial process data to obtain a data set with a minimum value of 0 and a maximum value of 1, and the new data matrix set is the input data set and the target data set Divide the data set into a training set Dtrn and test set D test ;

[0059] (3) To train the variable weight autoencoder, first use the labeled data to obtain the variable weight; suppose the labeled training data Train the variable weight de-redundancy AE, first determine the correlation of the variable and the target variable, and calculate the correlation coefficient of the dth variable through the labeled data

[0060]

[0061] Where X l(d) is the dth variable of the labeled data. The covariance and variance are calculated as follows

[0062]

[0063]

[0064]

[0065] Where and are the average values of the dth input and target variables respectively. If the absolute value of the correlation coefficient is large, it means that the variable is more related to the target variable. Therefore, the variable should be given a larger weight value, and vice versa. The weight of the dth input variable should be a monotonically increasing function of the absolute value of the correlation coefficient. The variable weight used in the present application is set to

[0066]

[0067] Train the stacked autoencoder in a supervised manner using variable weights

[0068]

[0069] The pre-trained weights are {W1, b1}, and the hidden features of the first layer are

[0070] (4) Principal component analysis de-redundancy, after extracting the hidden features using the autoencoder, according to the covariance matrix c ij , (i, j = 1, 2, …, p) represents the correlation coefficient between any two hidden features. Then use the eigenvalue decomposition formula of the covariance matrix

[0071]

[0072] For λ iArrange in descending order. Before PCA dimensionality reduction, the number of hidden layer nodes is g. Select the top s principal components with a cumulative contribution of ρ≥95%, where g≥s.

[0073] (5) Train the stacked autoencoder. For the second layer AE, the latent features h of the first layer AE are... 1 Provided to the input layer, h 1 The weight of each element is determined by the labeled data. and target output Y l To calculate. Then by minimizing The stacked autoencoder is trained using a variable weight reconstruction loss function. After training, the pre-trained weights are {W2, b2}. This process is repeated layer by layer until the final P-VW-AE is obtained.

[0074] (6) After pre-training, the output layer is added to the top of the SAE to fine-tune the weights. The pre-trained weights {W} k ,b k The weights are used to initialize the weights of each hidden layer. Improved weights can be obtained through backpropagation.

[0075] In this invention's embodiments, the root mean square error (RMSE) and coefficient of determination (R²) are selected. 2 The effectiveness of this method is evaluated as follows:

[0076]

[0077]

[0078] In the formula, N test , y test , Rm represents the total number of test samples, the predicted value of the test samples, the actual value of the test samples, and the mean of the actual values ​​of the test samples, respectively. A smaller RMSE value indicates a smaller error and better model prediction performance. 2 A higher value indicates a better fit of the model and better predictive performance. Table 2 below shows the predictive performance of different methods for butane concentration.

[0079] Table 2 Prediction performance of different methods in the butanizer tower

[0080]

[0081] As shown in Table 2, compared with the traditional stacked autoencoder model, the soft measurement modeling method based on variable-weight redundancy-removing stacked autoencoders proposed in this invention has better prediction performance. This method not only extracts the relevant features of input and output but also removes redundant information, providing a complete soft measurement model structure, thus making the soft measurement results more reliable.

[0082] The above cases are used to explain that the present application has certain feasibility and effectiveness. The focus of the present application is to introduce a variable weight stacked autoencoder, so that the extracted hidden features and output variables have stronger correlation, and redundant features are removed through principal component analysis. Any modification or improvement of the present application within the spirit and protection scope of the claims will be protected by the protection scope of the present application.

Claims

1.A method for soft-sensing modeling based on maximum-relevance minimum-redundancy stacked auto-encoder, characterized in that, The method comprises the following steps: Step (1) Collect industrial process data to determine the key quality variable y and auxiliary variables x, where x = [x1, x2,..., x d ] ∈ R d represents a vector composed of d auxiliary variables; Step (2) pre-process and normalize the collected industrial process data to get a data set with minimum value 0 and maximum value 1, and a new data matrix set as the input data set and the target data set Divide the data set into a training set D trn and a test set D test ; Step (3) constructs a soft measurement model of variable weight de-redundancy autoencoder for the normalized soft measurement model input and output data set, and is used for extracting output related hidden features; Step (4) uses principal component analysis to eliminate the redundancy between the hidden features extracted in step (3); Step (5) takes the hidden features after the principal component analysis in step (4) as input, repeatedly steps (3) and (4) for multiple times, and constructs a maximum correlation minimum redundancy stacked autoencoder soft measurement model; Step (6) adds an output layer to the top of the stacked autoencoder after pre-training to fine tune the weight; Step (7) builds a regression model on the training set and tests it on the test set D test The final implementation of the offline estimation of the key quality variables is tested on the above. 2.The maximum-relevance-minimum-redundancy based stacked autoencoder soft-sensing modeling method according to claim 1, wherein, The soft measurement model establishment process of the variable weight de-redundancy autoencoder in step (3) is as follows: (1) To train the variable weight autoencoder, first the labeled data should be used to obtain the variable weight; suppose the labeled training data where X l represents the labeled input dataset, Y l represents the labeled target dataset, N l represents the number of labeled samples, represents the N l th labeled input data, represents the N l th labeled target data; (2) determining the correlation between the input variable and the target variable, calculating the correlation coefficient of the dth variable through the labeled data: wherein, is the collection of the dth variable of the labeled data; The covariance and variance are calculated as follows: where and are the average values of the dth input variable and the target variable, respectively. If the absolute value of the correlation coefficient is large, it means that the input variable is more related to the target variable; therefore, this variable should be given a larger weight value, and vice versa; the weight of the dth input variable should be a monotonically increasing function of the absolute value of the correlation coefficient; the variable weight used is set as where λ (d) is the weight of the dth variable; (3) training the stacked autoencoder in a supervised manner by using the variable weight wherein W represents a weight matrix of the encoding layer, v represents a bias vector of the encoding layer, represents a weight matrix of the decoding layer, represents a bias vector of the decoding layer. 3.The maximum-relevance-minimum-redundancy based stacked autoencoder soft-sensing modeling method according to claim 1, characterized in that, The principal component analysis establishment process in step (4) is as follows: After extracting the hidden features by using the autoencoder, the correlation coefficient between any two hidden features is represented by c ij (i,j = 1,2,…,p) represents the correlation coefficient between any two hidden features; and the eigenvalue decomposition formula of the covariance matrix is used where λ i (i∈1,2,…,d) represent eigenvalues of the diagonal matrix; For λ i The hidden layer node number before dimension reduction of principal component analysis is g, the first s principal components are selected when the cumulative contribution degree is ρ≥95%, and g≥s. 4.The maximum-relevance-minimum-redundancy based stacked autoencoder soft-sensing modeling method according to claim 1, wherein, The maximum correlation minimum redundancy stacked autoencoder soft measurement model establishment process in step (5) is as follows: 1) Using the labeled dataset {X l ,Y l} to train the first variable weight autoencoder in a supervised manner, the pre-trained weights are {W1,b1}, and the hidden features of the first layer are 2) Suppose the number of hidden features is d, and the hidden features in step 1) are arranged in descending order to obtain The first d' hidden features are selected, and the corresponding accumulated contribution rate satisfies ρ≥95%. 3) according to the screening result of step 2), eliminating the nodes in the hidden layer which are not selected; 4) repeat the process of step 2) and step 3) to pre-process layer by layer until the last variable weight autoencoder is obtained; all the variable weight autoencoders pre-train weights are {W k ,b k} k＝1,2,…,L ; 5) after pre-training, adding an output layer to the top of the stacked autoencoder to fine tune the weight; Pre-trained weights {W k ,b k} k＝1,2,…,L Weights used to initialize each hidden layer; improved weights {W' can be obtained by backpropagation algorithm k ,b′ k} k＝1,2,…,L .

Images