Industrial process monitoring method based on lightweight deep principal component analysis-auto-encoder model

Abstract

The invention discloses an industrial process monitoring method based on a lightweight deep principal component analysis-auto-encoder model, and the method comprises the steps: collecting the related process variable data of an industrial process on line, carrying out the preprocessing of the collected data, inputting the preprocessed data into a trained lightweight deep principal component analysis-auto-encoder model, and carrying out the recognition of the related process variable data. The T2 statistical magnitude and the SPE statistical magnitude are obtained and compared with corresponding control limits, and a monitoring result is output; the lightweight deep principal component analysis-auto-encoder model uses a PCA dimension reduction module for dimension reduction to obtain the data score, inputs the score into the auto-encoder, and outputs the reconstruction data. According to the method, the calculation efficiency of the model is improved, compared with a traditional deep network model, the calculation speed is obviously improved, compared with a single multivariate statistical model, the strong feature extraction capacity of the deep hierarchical cascade model structure is incomparable to that of the single multivariate statistical model, and the problem of complex industrial process monitoring is effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of automatic control technology, and specifically relates to an industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model. Background Art

[0002] With the development of modern industry and manufacturing, the scale of production is constantly expanding, and the complexity and integration of production systems are also increasing. This trend means that even a small failure may cause a series of serious problems due to the complexity and interdependence of the system, such as control system failure, equipment damage, production interruption, and even casualties. Therefore, in order to ensure the safety and stability of industrial production systems, process monitoring technology has become crucial.

[0003] The research methods of process monitoring can be mainly divided into three categories, namely, mechanism model-based methods, knowledge-based methods, and data-based methods. The monitoring method based on the mechanism model is a technology that constructs a mathematical model by deeply understanding the physical and chemical principles behind the production process and the mechanism information such as the material and energy conservation relationship. This mathematical model can effectively detect and diagnose faults. Specific methods include equivalent space method, parameter estimation method, and state estimation method. The advantages of this type of method are that they can provide accurate process descriptions and have good explanatory power and applicability. However, this method may encounter challenges when facing complex large-scale systems. Knowledge-based methods use expert experience for process monitoring, such as fault trees and expert systems. These methods do not require complete mechanism knowledge, but make inference decisions by qualitatively describing the relationships in the production process. However, as the scale of production expands, it becomes very difficult to establish a complex and comprehensive expert knowledge base, so this type of method is more suitable for small and knowledge-rich production processes. Data-based monitoring methods do not require accurate and complete process mechanism information, nor do they rely on the experience and knowledge of experts and operators. It is based on the actual data generated by the industrial process, uses various data analysis methods to model the process, mines the useful information contained in the original process data, estimates the process operation status, and finally realizes process monitoring. Representative methods include multivariate statistical process monitoring method (MSPM), signal processing method, machine learning method, etc. Among them, MSPM is the longest-developed, most classic and most commonly used method, and representative techniques include principal component analysis (PCA), factor analysis (FA), linear discriminant analysis (LDA), independent component analysis (ICA), partial least squares (PLS), canonical correlation analysis (CCA), etc.

[0004] In addition, deep learning, an important branch of machine learning technology, has developed rapidly. Process monitoring methods based on deep learning have also continued to emerge in the past decade. Representative models include autoencoders (AE), deep belief networks (DBN), variational autoencoders (VAE), recurrent neural networks (RNN), convolutional neural networks (CNN), graph neural networks (GNN), Transformer, etc. Deep learning technology was initially only able to solve the monitoring problem of simple processes. It has gradually been combined with advanced statistical learning, data mining, and artificial intelligence technologies. At present, it can achieve effective monitoring of many complex industrial scenarios. However, deep learning models are unexplainable and are not very reliable for users' decisions. The cost of deep learning models having good feature expression capabilities is that they require a sufficient number of samples and corresponding computing power. Single multivariate statistics cannot meet the requirements for complex industrial scenarios. Therefore, a hierarchical cascade model structure with deep learning is proposed. A lightweight model with considerable learning ability can solve the problems faced.

[0005] In order to achieve efficient process monitoring in complex industries, it is necessary to provide an industrial process monitoring method that combines a lightweight multivariate statistical model with a deep learning model. Summary of the invention

[0006] The purpose of the present invention is to address the shortcomings of the prior art. The single-layer monitoring model cannot extract the deep features of the data, and the multi-layer neural network model has high computational cost. An industrial process monitoring method based on lightweight deep principal component analysis (PCA)-autoencoder (AE) is provided.

[0007] An industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model, comprising: online collection of relevant process variable data of the industrial process, preprocessing the collected data, inputting the preprocessed data into a trained lightweight deep principal component analysis-autoencoder model, obtaining a data score T and reconstructing the data; obtaining T using the data score 2 Statistics, use the reconstructed data to obtain the SPE statistics, compare with the corresponding control limits, and output the monitoring results; the lightweight deep principal component analysis-autoencoder model consists of a PCA dimensionality reduction module and an autoencoder, and the PCA dimensionality reduction module is used to reduce the dimension of the input data to obtain the data score, which is input into the autoencoder, and the autoencoder is used to output the reconstructed data.

[0008] Furthermore, the lightweight deep principal component analysis-autoencoder model is a multi-layer structure, each layer includes the PCA dimension reduction module and the autoencoder. In the training model stage, the optimal number of layers can be determined by the output results of each layer of the autoencoder.

[0009] Alternatively, as another alternative, the lightweight deep principal component analysis-autoencoder model has a multi-layer structure, the last layer is composed of the PCA dimensionality reduction module and the autoencoder, and the remaining layers are composed of the PCA dimensionality reduction module and the local autoencoder, and the local autoencoder is composed of an input layer and a hidden layer. The local autoencoder here is equivalent to the above-mentioned autoencoder with the output layer removed.

[0010] The T 2 The statistic is obtained based on the score T of the last layer; the SPE statistic comes from the output data of the autoencoder of the last layer. More specifically, the SPE statistic is calculated from the residual of the reconstructed data after decoding the autoencoder of the last layer.

[0011] Furthermore, the number of layers of the multilayer structure is greater than or equal to 2. Furthermore, the number of layers of the multilayer structure is 2 to 10, and more preferably 3 to 5.

[0012] Furthermore, in the multi-layer structure, the hidden layer of the previous autoencoder is input to the next layer as the input of the PCA dimension reduction module of the layer. For example, for the first layer, the input is the preprocessed data x1, the score is T1=x1P, the test set data score is used as the input of the autoencoder, the hidden layer h1=f(W1T1+b1) of the autoencoder, the autoencoder outputs the decoded data y1=g(W2h1+b2), and the first layer reconstructed data is reconstructed by the decoded data y1 Then the second layer input is the hidden layer data h1 of the first layer. The overall model of this model presents a multi-layer cascade structure. The data feature extraction ability of the multi-layer model is better than that of the single layer. The multi-layer model learns the deeper information of the sample data, and the result data is better than that of the single layer, which can be widely used in process monitoring of complex industries.

[0013] Furthermore, the T 2 Control limits of statistics T 2 lim By calculating the pseudo-inverse, the control limit SPE of the SPE statistic is obtained. lim Calculated using the chi-square distribution.

[0014] Furthermore, SVD decomposition is used to perform the data dimension reduction.

[0015] Furthermore, the industrial engineering is a papermaking wastewater process.

[0016] Furthermore, the relevant process variable data includes one or more of temperature, flow, pressure, chemical oxygen demand, suspended solids value, and pH value.

[0017] Furthermore, the preprocessing is a standardization process, that is, for each variable data, the mean corresponding to its training sample is subtracted, and then divided by the standard deviation corresponding to its training sample to obtain a new variable data with a mean of 0 and a standard deviation of 1.

[0018] In the model training phase, the same preprocessing method is used, and the mean and standard deviation corresponding to the corresponding samples are directly used for calculation.

[0019] The lightweight deep principal component analysis-autoencoder of the present invention adopts a similar method as described above in the model training stage, that is, data collection-preprocessing-input lightweight deep principal component analysis-autoencoder training. The training set data set can use either online data under normal working conditions or normal operating data samples in the collected data set.

[0020] The present invention provides an industrial process monitoring method based on lightweight deep principal component analysis-autoencoder. For the principal component analysis method, the redundant information of the data can be effectively removed, and the key latent variables of the sample data can be extracted. The latent variables are the key features of the data samples and the essential factors for exploring the problem. At the same time, the key latent variables that have been removed from the redundancy are transmitted to the autoencoder, which is conducive to improving the calculation speed of the model. The autoencoder has better data reconstruction capabilities for more "pure" data. For small-scale industrial process detection, a single-layer cascade principal component analysis-autoencoder model has a good monitoring effect. For complex factory-level industrial process data, although the deep learning model has a powerful feature expression ability, the selection of training samples, the heavy parameter adjustment and the calculation work bring many problems. The present invention combines PCA with AE to achieve lightweight modeling.

[0021] Therefore, compared with the prior art, the multi-layer cascaded principal component analysis-autoencoder structure proposed in the present invention can process complex industrial process information. The combination of PCA and AE, the two different models play different roles. PCA reduces the dimension of industrial data, extracts key latent variables and sends them to the autoencoder, which greatly improves the computational efficiency of the model. Compared with the traditional deep network model, the calculation speed is significantly improved. Compared with a single multivariate statistical model, the deep hierarchical cascade model structure has a powerful feature extraction capability that is unmatched by a single multivariate statistical model, and effectively solves the process monitoring problem of complex industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a diagram of the structure of lightweight deep principal component analysis-autoencoder modeling. DETAILED DESCRIPTION

[0023] The technical solution of the present invention will be further described below in conjunction with the drawings in the specific embodiments of the present invention. Obviously, the specific embodiments described are only some of the embodiments of the present invention, not all of the embodiments. Any other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work should fall within the scope of protection of the present invention.

[0024] Example 1

[0025] Taking the papermaking wastewater process as an example, the present invention is further described:

[0026] The present invention proposes an industrial process monitoring method based on lightweight deep principal component analysis-autoencoder, such as Figure 1 As shown, the method includes the following steps (the following content is explained while taking into account the model training process):

[0027] Step 1: Collect data sets. Obtain training samples. Under normal working conditions, collect relevant process variable data X through the distributed control system, and divide the variable data X into training set sample data X0 and test set sample data X1:

[0028] X={x(1),x(2),x(3),…,x(M)},X∈R M×J

[0029] X0={x(1),x(2),x(3),…,x(K)},X∈R K×J

[0030] X1={x(K+1),x(K+2),x(K+3),…,x(M)},X∈R M-K×J

[0031] Where M is the sum of the number of training data and test data samples, where the number of training data samples is K, the number of test data samples is MK, and the number of variables in the training data and test data is J. The variables include temperature, flow, pressure, COD (chemical oxygen demand), SS (suspended solids) value, and pH value.

[0032] During actual monitoring, the corresponding relevant process variable data samples are directly collected online in real time.

[0033] Step 2: Data preprocessing: The purpose of preprocessing in this step is to subtract the mean corresponding to each sample variable in the training set matrix X0 from the variable, and then divide it by its standard deviation, so as to obtain a new matrix x0 with a mean of 0 and a standard deviation of 1. This standardization process is very common in machine learning and data analysis, and helps to improve the performance and stability of the algorithm. The standardization of the test set data is after the training set is standardized. The standardization of the test set data is to subtract the mean of the test set and then divide it by its standard deviation. The result is x1. In actual monitoring, the preprocessing process is similar, and the corresponding mean and standard deviation can be the mean and standard deviation of the training sample.

[0034] Step 3: Training model parameters with training set. Perform PCA dimensionality reduction on the standardized training set data x0. Here, SVD decomposition is used.

[0035]

[0036] Where U and V are orthogonal matrices, U is the left singular vector, V is the right singular vector, U∈R k×k , V∈R j×j , the matrix Σ is a diagonal matrix, the diagonal elements of Σ are called singular values, Σ∈R k×j Contains decreasing (σ1≥σ2≥…≥σ min (k,j)≥0), the variance of the projection of the training set along the i-th column of the matrix V is equal to σ i 2 , which can also be equivalent to finding the eigenvalue decomposition of the covariance matrix S:

[0037]

[0038] Diagonal matrix Λ=Σ T Σ∈R j×j Contains non-negative real eigenvalues ​​with decreasing magnitude (λ1≥λ2≥λ3≥λ4≥...λ m ), and λ i =σ i 2 , according to the number of eigenvalues ​​λ values ​​accounting for the total number of λ values ​​a, a is the number of principal components, the load matrix P∈R j×a The score T is calculated from the first a columns of the matrix V and is T=xP.

[0039] The score T is used as the input of the autoencoder (AE), which is an unsupervised learning algorithm used to learn effective encoding of data. The autoencoder encodes the input data into a lower-dimensional representation (encoding process), which is called a hidden layer, and then reconstructs the input data from this representation (decoding process). The goal of the autoencoder is to minimize the difference between the input and the reconstruction. The mathematical expression of the encoding and decoding process of the autoencoder is as follows (h is the hidden layer output data, y is the output decoded data, W1 and W2 represent the weights of the neurons respectively, b1 and b2 represent the biases of the neurons, P' is the transposed matrix of P, is the data after x is reconstructed by the model):

[0040] h=f(W1T+b1)

[0041] y=g(W2h+b2)

[0042]

[0043] The settings of the autoencoder parameters include the size of the hidden layer. Adjusting the size of the hidden layer is conducive to quickly debugging the model effect. The maximum number of iterations of model training is 100. The loss function uses a loss function with a regularization term added. The L2 regularization coefficient is set to 0.001, which is used to control the penalty strength of the weight to prevent overfitting. The sparsity regularization coefficient is 4. The sparsity regularization coefficient is a hyperparameter that needs to be adjusted according to the specific task and data characteristics. Through proper settings, the performance and interpretability of the model can be effectively improved. The target sparsity ratio is set to 0.05, and 5 percent of the neurons are activated. This sparsity can help the model capture more important features in the data and improve the generalization ability of the model. Finally, choose whether to automatically scale the input data before training the autoencoder. Here it is set to no. In the verification phase or the actual monitoring phase, T 2 The statistic is calculated based on the score T. During the training, validation, or actual monitoring phase, the SPE statistic is calculated based on the residual of the reconstructed data after decoding by the autoencoder. The calculation formula is as follows:

[0044] T 2 =t′ PCs Λ a -1 t PCs

[0045]

[0046] where t PCS is the score vector of the sample in the principal component space, which is obtained by projecting the score T in the principal component space, t' PCS t PCSThe transposed matrix of . Λ is the square of the diagonal matrix after SVD decomposition, Λ a is the first a columns of Λ. e is the residual matrix

[0047] Calculate the T of the training set 2 Statistics and SPE statistics, T 2 Control limits of statistics T 2 lim It can be calculated by the following formula, where F a (0.99,a,na) ​​refers to the inverse function value of the F distribution with a numerator and na denominator at a significance level of 0.99. The control limits of the SPE statistic SPE lim The chi-square distribution χ can be expressed as a 2 Calculated, where χ a 2 (0.99,h) refers to the inverse function value of the chi-square distribution with h degrees of freedom at a significance level of 0.99.

[0048]

[0049] SPElim=gχa 2 (0.99,h)

[0050] Among them, n is the number of samples of data, a is the number of principal components, SPE b It represents the mean of SPE, and var(SPE) represents the variance of SPE.

[0051] Step 4: Increase the number of model layers. In order to solve the process monitoring of complex industries and extract deeper information of the data structure, the one-layer PCA-AE model is expanded to n layers. The model structure diagram is shown in the figure Figure 1 As shown. The one-layer PCA-AE model is extended to multiple layers, which is called the deep PCA-AE model. Compared with the multi-layer AE model, the data has not been processed accordingly. The computational complexity of processing complex industrial data is greatly increased, which leads to an increase in computational cost. The computational efficiency of the deep PCA-AE model far exceeds that of the traditional deep network model. The structural particularity of the deep PCA-AE model has a "lightweight" effect, so it is named lightweight deep principal component analysis-autoencoder. When a one-layer PCA-AE model is expanded to a multi-layer PCA-AE model, one point that needs to be considered is the connection parameters between the first-layer model and the second-layer model, such as Figure 1As shown, the hidden layer of the previous model is input to the next model (for specific calculation steps, please refer to the test set data sample calculation process in step 5). The overall model presents a multi-layer cascade structure. The data feature extraction capability of the multi-layer model is better than that of a single layer. The multi-layer model learns deeper information of the sample data. The resulting data is better than that of a single layer and can be widely used in process monitoring of complex industries.

[0052] Step 5: Test the model with the test set. The actual monitoring process is similar to the test set test process, except that the input samples are different. The former uses samples collected online, while the latter uses the sample set that has been collected. The following is a detailed description of the calculation process of the test set data sample. The test set data sample is X1, which is x1 after standardization. The steps of training model parameters with the training set are similar, except that the parameters in the deep PCA-AE model do not need to be calculated again. The model parameters trained with the training set are used, such as the load matrix P and some parameters involved in the autoencoder. Start testing the model. The input of the model is the test set data. After PCA dimensionality reduction, the first-layer load matrix P1 matrix is ​​obtained. After multiplication, the test set data score T1=x1P1 is obtained. The test set data score is used as the input of the first-layer autoencoder. The hidden layer of the autoencoder h1=f(W1T1+b1). The parameters of the autoencoder here are consistent with those of the test set, such as the size of the hidden layer, etc. The autoencoder outputs the decoded data y1=g(W2h1+b2), and the first-layer reconstructed data is reconstructed by the decoded data y1 (P1' is the transposed matrix of P1). The input of the second layer is the hidden layer data h1 of the first layer. The second PCA is performed to obtain the second layer load matrix P2 matrix. The second layer score T2 = h1P2 is calculated. The second layer score is input to AE to obtain the hidden layer h2 and the decoded data y2. The second layer reconstructs the data Repeat n times, n ≥ 2 (usually 3 to 5).

[0053] hn=f(WnTn-1+bn)

[0054] yn=g(Wnht-1+bn)

[0055]

[0056] The SPE statistic is calculated by subtracting the last layer of reconstructed data from the standardized test set data x1 The square of T 2 The statistic is calculated from the test set scores. 2The statistics come from the PCA data in the last layer of the PCA-AE model, and the SPE statistics come from the autoencoder data in the last layer of the PCA-AE model. The statistics calculated without using a base model can reflect the characteristic information of the data and effectively verify the effectiveness of the proposed model.

[0057] Step 6: Data visualization. Draw a picture on the programming software and draw T with a blue line. 2 Line graph of the statistic and the SPE statistic. The red line represents T 2 The control limits of the statistics and SPE statistics mean that the sample data exceeding the red line means the occurrence of a fault, and the data below the control limit represents normal working condition data. By comparing the labeled industrial process data, the location of the fault can be determined. In order to quantify the monitoring effect of the proposed model, two indicators are introduced, the false alarm rate and the missed alarm rate. The false alarm rate refers to the proportion of sample data that is actually in normal working condition and is mistakenly predicted as fault data. The missed alarm rate refers to the proportion of sample data that is actually faulty and is mistakenly predicted as normal sample data. This method can reduce the false alarm rate and missed alarm rate by more than 30% in industrial processes. The lightweight deep principal component analysis-autoencoder model adopts a hierarchical cascade connection multi-layer PCA-AE model, which can not only cope with large and complex industrial process monitoring, but also learn the deep information of industrial process data, deeply mine key latent variables, and improve the model's applicability to data.

[0058] The above content is merely an example of the overall structure of the present invention. The present invention may also have many other embodiments. Without departing from the framework and essence of the present invention, technicians in the relevant field may make various changes or additions to the above specific embodiments or use similar methods to replace them, which should all fall within the scope of protection of the present invention.

Claims

1. An industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model, characterized in that: include: Collect relevant process variable data of industrial processes online, preprocess the collected data, input the preprocessed data into the trained lightweight deep principal component analysis-autoencoder model, obtain data scores and reconstruct data; use data scores to obtain T 2 Statistics, use the reconstructed data to obtain the SPE statistics, compare with the corresponding control limits, and output the monitoring results; the lightweight deep principal component analysis-autoencoder model consists of a PCA dimensionality reduction module and an autoencoder, and the PCA dimensionality reduction module is used to reduce the dimension of the input data to obtain the data score, which is input into the autoencoder, and the autoencoder is used to output the reconstructed data.

2. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 1 is characterized in that: The lightweight deep principal component analysis-autoencoder model is a multi-layer structure, and each layer includes the PCA dimension reduction module and the autoencoder.

3. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 1 is characterized in that: The lightweight deep principal component analysis-autoencoder model is a multi-layer structure, the last layer is composed of the PCA dimensionality reduction module and the autoencoder, and the remaining layers are composed of the PCA dimensionality reduction module and the local autoencoder, and the local autoencoder is composed of an input layer and a hidden layer.

4. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 2 or 3 is characterized in that: In the multi-layer structure, the hidden layer of the previous autoencoder or local autoencoder is input to the next layer as the input of the PCA dimensionality reduction module of this layer.

5. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 4 is characterized in that: T 2 The statistic is obtained based on the score T of the last layer; the SPE statistic is calculated from the residual of the reconstructed data after decoding by the last layer of autoencoder.

6. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 1 is characterized in that: T 2 Control limits of statistics T 2 lim By calculating the pseudo-inverse, the control limit SPE of the SPE statistic is obtained. lim Calculated using the chi-square distribution.

7. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 1 is characterized in that: The data dimension reduction is performed using SVD decomposition.

8. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 1 is characterized in that: The industrial project is a papermaking wastewater process.

9. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 1 is characterized in that: The relevant process variable data includes one or more of temperature, flow, pressure, chemical oxygen demand, suspended solid value, and pH value.

10. The industrial process monitoring method based on lightweight deep principal component analysis-autoencoder model according to claim 1 is characterized in that: The preprocessing is standardization processing, that is, for each variable data, the mean corresponding to its training sample is subtracted, and then divided by the standard deviation corresponding to its training sample to obtain a new variable data with a mean of 0 and a standard deviation of 1.

Images