An industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model

Abstract

The application discloses an industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model, and comprises the following steps: collecting related process variable data of an industrial process online, pre-processing the collected data, inputting the pre-processed data into a trained lightweight deep principal component analysis-autoencoder model, obtaining T 2 statistical quantity and an SPE statistical quantity, comparing the statistical quantity and the SPE statistical quantity with corresponding control limits, and outputting a monitoring result; the lightweight deep principal component analysis-autoencoder model utilizes a PCA dimension reduction module to reduce the dimension of the data score, inputs the score into an autoencoder, and outputs the reconstruction data. The application improves the calculation efficiency of the model, the calculation speed is obviously improved compared with a traditional deep network model, and compared with a single multivariate statistical model, the deep hierarchical cascading model structure has strong feature extraction capability which cannot be compared with the single multivariate statistical model, and effectively solves the complex industrial process monitoring problem.

Description

Technical Field

[0001] This invention belongs to the field of automatic control technology, specifically relating to an industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model. Background Technology

[0002] With the development of modern industry and manufacturing, production scale is constantly expanding, and the complexity and integration of production systems are also increasing. This trend means that even a small failure can trigger a series of serious problems due to the complexity and interdependence of the system, such as control system failure, equipment damage, production interruption, and even personal injury or death. Therefore, process monitoring technology has become crucial to ensure the safety and stability of industrial production systems.

[0003] Process monitoring research methods can be mainly divided into three categories: mechanistic model-based methods, knowledge-based methods, and data-based methods. Mechanism-based monitoring methods are techniques that construct mathematical models by deeply understanding the physical and chemical principles behind the production process, as well as the conservation relationships of materials and energy. These mathematical models can effectively detect and diagnose faults. Specific methods include the equivalent space method, parameter estimation method, and state estimation method. The advantage of these methods is that they can provide accurate process descriptions and have good interpretability and applicability. However, this method may encounter challenges when dealing with complex, large-scale systems. Knowledge-based methods utilize expert experience for process monitoring, such as fault trees and expert systems. These methods do not require complete mechanistic knowledge but rather rely on qualitative descriptions of the relationships within the production process for reasoning and decision-making. However, as production scales up, establishing a complex and comprehensive expert knowledge base becomes extremely difficult; therefore, these methods are more suitable for small-scale, knowledge-rich production processes. Data-based monitoring methods do not require accurate and complete process mechanism information, nor do they rely on the experience and knowledge of experts and operators. Based on actual data generated by industrial processes, it utilizes various data analysis methods to model the processes, extract useful information from the raw process data, estimate the process operating status, and ultimately achieve process monitoring. Representative methods include Multivariate Statistical Process Monitoring (MSPM), signal processing methods, and machine learning methods. Among these, MSPM is the oldest, most classic, and most commonly used method, with representative techniques including Principal Component Analysis (PCA), Factor Analysis (FA), Linear Discriminant Analysis (LDA), Independent Component Analysis (ICA), Partial Least Squares (PLS), and Canonical Correlation Analysis (CCA).

[0004] Furthermore, deep learning, a crucial branch of machine learning, has developed rapidly, and deep learning-based process monitoring methods have emerged continuously over the past decade. Representative models include autoencoders (AEs), deep belief networks (DBNs), variational autoencoders (VAEs), recurrent neural networks (RNNs), convolutional neural networks (CNNs), graph neural networks (GNNs), and Transformers. Deep learning technology has evolved from initially only being able to solve simple process monitoring problems to gradually combining with advanced statistical learning, data mining, and artificial intelligence techniques. Currently, it can effectively monitor many complex industrial scenarios. However, deep learning models are not interpretable and their reliability for user decisions is not entirely trustworthy. The cost of deep learning models' excellent feature representation capabilities is the need for a sufficient number of samples and corresponding computational power. Single multivariate statistical methods cannot meet the requirements for complex industrial scenarios. Therefore, this paper proposes a hierarchical cascaded model structure with deep learning capabilities. This lightweight model with considerable learning power can solve the problems faced.

[0005] To achieve efficient process monitoring in complex industries, a lightweight multivariate statistical model combined with a deep learning model is needed for industrial process monitoring. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies, such as the inability of single-layer monitoring models to extract deep features of data and the high computational cost of multi-layer neural network models, by providing an industrial process monitoring method based on lightweight deep principal component analysis (PCA)-autoencoder (AE).

[0007] A method for monitoring industrial processes based on a lightweight deep principal component analysis-autoencoder model includes: online collection of relevant process variable data of the industrial process; preprocessing the collected data; inputting the preprocessed data into a trained lightweight deep principal component analysis-autoencoder model to obtain a data score T and reconstruct the data; and using the data score to obtain T. 2 The SPE statistic is obtained using the reconstructed data, compared with the corresponding control limits, and the monitoring results are output. The lightweight deep principal component analysis-autoencoder model consists of a PCA dimensionality reduction module and an autoencoder. The PCA dimensionality reduction module reduces the dimensionality of the input data to obtain the data score, which is then input into the autoencoder to output the reconstructed data.

[0008] Furthermore, the lightweight deep principal component analysis-autoencoder model has a multi-layer structure, with each layer including the aforementioned PCA dimensionality reduction module and an autoencoder. During the model training phase, the optimal number of layers can be determined by analyzing the output of each autoencoder layer.

[0009] Alternatively, as an alternative, the lightweight deep principal component analysis-autoencoder model has a multi-layer structure. The last layer consists of the PCA dimensionality reduction module and the autoencoder, while the remaining layers consist of the PCA dimensionality reduction module and the local autoencoder. The local autoencoder consists of an input layer and a hidden layer. Here, the local autoencoder is equivalent to the aforementioned autoencoder without the output layer.

[0010] The T 2 The statistics are obtained from the score T of the last layer; the SPE statistics come from the output data of the autoencoder of the last layer. More specifically, the SPE statistics are calculated from the residuals of the reconstructed data after decoding by the autoencoder of the last layer.

[0011] Furthermore, the number of layers in the multi-layer structure is greater than or equal to 2, and even more preferably, the number of layers in the multi-layer structure is 2 to 10, and more preferably 3 to 5.

[0012] Furthermore, in the multi-layer structure, the hidden layer input of the previous layer's autoencoder is passed to the next layer as the input to the PCA dimensionality reduction module of that layer. For example, for the first layer, the input is the preprocessed data x1, with a score T1 = x1P. The test set data score is used as the input to the autoencoder, and the hidden layer h1 = f(W1T1 + b1) of the autoencoder. The autoencoder outputs decoded data y1 = g(W2h1 + b2), and the first layer's reconstructed data is reconstructed from the decoded data y1. The input to the second layer is the hidden data h1 of the first layer. The overall model presents a multi-layer cascade structure. The multi-layer model has better data feature extraction capabilities than the single-layer model. The multi-layer model learns deeper information from the sample data, resulting in better data than the single-layer model. It can be widely used for process monitoring in complex industries.

[0013] Furthermore, the T 2 Control limits T of the statistic 2 lim The control limit SPE statistic is obtained by calculating the pseudo-inverse. lim It was calculated using the chi-square distribution.

[0014] Furthermore, SVD decomposition is used to reduce the dimensionality of the data.

[0015] Furthermore, the industrial engineering process is a papermaking wastewater process.

[0016] Furthermore, the relevant process variable data include one or more of the following: temperature, flow rate, pressure, chemical oxygen demand, suspended solids value, and pH value.

[0017] Furthermore, the preprocessing is a standardization process, that is, for each variable data, the mean corresponding to its training sample is subtracted, and then the result is divided by the standard deviation corresponding to its training sample to obtain a new variable data with a mean of 0 and a standard deviation of 1.

[0018] During the model training phase, the same preprocessing method is used, directly calculating the mean and standard deviation of the corresponding samples.

[0019] The lightweight deep principal component analysis-autoencoder described in this invention employs a similar method during the model training phase, namely, data acquisition, preprocessing, and input to the lightweight deep principal component analysis-autoencoder for training. The training dataset can be either online data under normal operating conditions or samples of normally operating data from already collected datasets.

[0020] This invention provides an industrial process monitoring method based on lightweight deep principal component analysis (PCA) and autoencoder. PCA effectively removes redundant information from data and extracts key latent variables from sample data. These latent variables are crucial features of the data samples and are essential factors in identifying the problem. Simultaneously, feeding these key latent variables (after removing redundancy) to the autoencoder improves the model's computational speed. The autoencoder also has better data reconstruction capabilities for "cleaner" data. For small-scale industrial process monitoring, a single-layer cascaded PCA-autoencoder model exhibits good monitoring performance. However, for complex plant-level industrial process data, while deep learning models possess powerful feature representation capabilities, the selection of training samples, the heavy workload of parameter tuning, and computation present numerous challenges. This invention combines PCA and autoencoder to achieve lightweight modeling.

[0021] Therefore, compared with existing technologies, the multi-layered cascaded principal component analysis-autoencoder structure proposed in this invention can handle complex industrial process information. The combination of PCA and AE allows two different models to play different roles. PCA reduces the dimensionality of industrial data and extracts key latent variables, which are then fed to the autoencoder, greatly improving the computational efficiency of the model. Compared with traditional deep network models, the computational speed is significantly improved. Compared with a single multivariate statistical model, the powerful feature extraction capability of the deep hierarchical cascaded model structure is unmatched by a single multivariate statistical model, effectively solving the problem of process monitoring in complex industries. Attached Figure Description

[0022] Figure 1 This is a structural diagram of a model based on lightweight deep principal component analysis-autoencoder. Detailed Implementation

[0023] The technical solution of the present invention will be further described below with reference to the accompanying drawings of specific embodiments. Obviously, the described specific embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Any other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort should fall within the protection scope of the present invention.

[0024] Example 1

[0025] Taking the papermaking wastewater process as an example, the present invention will be further explained as follows:

[0026] This invention proposes an industrial process monitoring method based on lightweight deep principal component analysis-autoencoder, such as... Figure 1 As shown, the method includes the following steps (the following content also takes into account the model training process):

[0027] Step 1: Data Collection. Obtain training samples. Under normal operating conditions, collect relevant process variable data X through the distributed control system, and divide the variable data X into training set sample data X0 and test set sample data X1:

[0028] X={x(1),x(2),x(3),…,x(M)},X∈R M×J

[0029] X0={x(1),x(2),x(3),…,x(K)},X∈R K×J

[0030] X1={x(K+1),x(K+2),x(K+3),…,x(M)},X∈R M-K×J

[0031] Where M is the total number of training and test data samples, with K being the number of training data samples and MK being the number of test data samples. The number of variables in both training and test data is J, and the variables include temperature, flow rate, pressure, COD (chemical oxygen demand), SS (suspended solids), and pH value.

[0032] In actual monitoring, relevant process variable data samples are collected directly online in real time.

[0033] Step Two: Data Preprocessing: The purpose of preprocessing in this step is to subtract the mean of each sample variable in the training set matrix X0 from the mean of that variable, and then divide by its standard deviation, resulting in a new matrix x0 with a mean of 0 and a standard deviation of 1. This standardization process is very common in machine learning and data analysis, and helps improve the performance and stability of the algorithm. Standardization of the test set data follows the standardization of the training set data. Test set data standardization involves subtracting the mean of the test set data and then dividing by its standard deviation, resulting in x1. In actual monitoring, the preprocessing process is similar, and the mean and standard deviation can be taken from the training samples.

[0034] Step 3: Train model parameters using the training set. Perform PCA dimensionality reduction on the standardized training set data (x0 data points). Here, SVD decomposition is used.

[0035]

[0036] Where U and V are orthogonal matrices, U is a left singular vector, V is a right singular vector, and U∈R k×k , V∈R j×j Σ is a diagonal matrix, and the diagonal elements of Σ are called singular values. Σ∈R k×j Includes (σ1≥σ2≥…≥σ) decreasing along its main diagonal min (k,j)≥0), the variance of the projection of the training set along the i-th column of matrix V is equal to σ. i 2 This can also be equivalent to finding the eigenvalue decomposition of the covariance matrix S:

[0037]

[0038] Diagonal matrix Λ=Σ T Σ∈R j×j Includes non-negative real eigenvalues ​​with decreasing magnitude (λ1≥λ2≥λ3≥λ4≥...λ) m ), and λ i =σ i 2 Based on the ratio of the magnitude of the eigenvalues ​​λ to the total number of λ values ​​'a', and the number of principal components 'a', the loading matrix P∈R j×a The score T is obtained from the first a columns of matrix V, and the formula for calculating the score T is T = xP.

[0039] The score T is used as input to an autoencoder (AE), an unsupervised learning algorithm used to learn an efficient encoding of data. An autoencoder reconstructs the input data by encoding it into a low-dimensional representation (encoding process), called the hidden layer, and then reconstructing the input data from this representation (decoding process). The goal of an autoencoder is to minimize the difference between the input and the reconstructed data. The mathematical expressions for the encoding and decoding processes of an autoencoder are as follows (h is the hidden layer output data, y is the decoded output data, W1 and W2 represent the weights of the neurons, b1 and b2 represent the biases of the neurons, and P' is the transpose of P). (The data for x after model reconstruction):

[0040] h = f(W1T + b1)

[0041] y = g(W²h + b²)

[0042]

[0043] The autoencoder parameters include the hidden layer size; adjusting the hidden layer size facilitates rapid model tuning. The maximum number of training iterations is 100. A loss function with regularization is used, with the L2 regularization coefficient set to 0.001 to control the penalty strength of the weights and prevent overfitting. The sparsity regularization coefficient is 4; this is a hyperparameter that needs to be adjusted based on the specific task and data characteristics. Appropriate settings can effectively improve model performance and interpretability. The target sparsity ratio is set to 0.05, meaning 5% of neurons are activated. This sparsity helps the model capture more important features in the data and improves its generalization ability. Finally, the option to automatically scale the input data before training the autoencoder is selected; here, it is set to no. During the validation or actual monitoring phase, T... 2 The statistic is calculated based on the score T. During the training, validation, or actual monitoring phases, the SPE statistic is calculated based on the residuals of the reconstructed data after decoding by the autoencoder. The calculation formula is as follows:

[0044] T 2 =t′ PCs Λ a -1 t PCs

[0045]

[0046] Where t PCS Let t' be the score vector of the sample in the principal component space, specifically obtained by projecting the score T onto the principal component space. PCS For t PCSThe transpose of Λ. Λ is the square of the diagonal matrix after SVD decomposition. a It takes the first a columns of Λ. e is the residual matrix.

[0047] Calculate T of the training set 2 Statistic and SPE statistic, T 2 Control limits T of the statistic 2 lim It can be calculated using the following formula, where F a (0.99, a, na) refers to the inverse function value of the F-distribution with numerator degrees of freedom 'a' and denominator degrees of freedom 'na' at a significance level of 0.99. The control limits for the SPE statistic are... lim The χ² distribution can be expressed by the following formula. a 2 The calculation yields χ, where χ is the value of χ. a 2 (0.99,h) refers to the inverse function value of a chi-square distribution with h degrees of freedom at a significance level of 0.99.

[0048]

[0049] SPElim=gχa 2 (0.99,h)

[0050] Where n is the number of data samples, a is the number of principal components, and SPE is the number of samples. b This represents calculating the mean of SPE, and var(SPE) represents calculating the variance of SPE.

[0051] Step 4: Increase the number of model layers. To address the challenges of process monitoring in complex industries and extract deeper information from the data structure, the single-layer PCA-AE model is expanded to n layers. The model structure diagram is as follows: Figure 1 As shown. Extending a single-layer PCA-AE model to a multi-layer model is called a deep PCA-AE model. Compared to multi-layer AE models, no data processing is performed. Handling complex industrial data significantly increases computational complexity and leads to higher computational costs. The computational efficiency of the deep PCA-AE model far exceeds that of traditional deep network models. The unique structure of the deep PCA-AE model achieves a "lightweight" effect, hence the name "Lightweight Deep Principal Component Analysis-Autoencoder." When extending a single-layer PCA-AE model to a multi-layer PCA-AE model, one point to consider is the connection parameters between the first and second layers, such as... Figure 1As shown, the hidden layer of the upper layer model is input to the lower layer model (for specific calculation steps, please refer to step five regarding the calculation process of the test set data samples). The overall model presents a multi-layer cascade structure. The data feature extraction capability of the multi-layer model is better than that of the single layer. The multi-layer model learns deeper information from the sample data, and the resulting data is better than that of the single layer. It can be widely used for process monitoring in complex industries.

[0052] Step 5: Test the model on the test set. The actual monitoring process is similar to the test set testing process, only the input samples are different. The former uses online collected samples, while the latter uses a pre-collected sample set. The following is a detailed explanation of the test set data sample calculation process as an example. The test set data sample is X1, which is standardized to x1. The steps are similar to training the model parameters on the training set, except that the parameters in the deep PCA-AE model do not need to be recalculated; the model parameters trained on the training set are used, such as the load matrix P and some parameters involved in the autoencoder. Start testing the model. The input of the model is the test set data. After PCA dimensionality reduction, the first layer load matrix P1 is obtained. After multiplication, the test set data score T1 = x1P1 is obtained. The test set data score is used as the input of the first layer autoencoder. The hidden layer of the autoencoder is h1 = f(W1T1 + b1). The parameters of the autoencoder here are consistent with the parameters of the test set, such as the size of the hidden layer, etc. The autoencoder outputs decoded data y1 = g(W2h1 + b2). The first layer reconstructed data is reconstructed from the decoded data y1. (P1' is the transpose of P1). The input to the second layer is the hidden layer data h1 from the first layer. A second PCA is performed to obtain the second layer load matrix P2. The second layer score T2 = h1P2 is calculated and input to the AE to obtain the hidden layer h2 and the decoded data y2. The second layer reconstructed data... Repeat n times, where n≥2 (usually 3 to 5).

[0053] hn=f(WnTn-1+bn)

[0054] yn = g(Wnht-1 + bn)

[0055]

[0056] The SPE statistic is calculated by subtracting the last layer of reconstructed data from the standardized test set data x1. The square of T. 2 The statistic is calculated from the test set scores. T 2The statistics are derived from the PCA data in the last layer of the PCA-AE model, and the SPE statistics are derived from the autoencoder data in the last layer of the PCA-AE model. The statistics calculated from different base models can reflect the characteristic information of the data and effectively verify the effectiveness of the proposed model.

[0057] Step Six: Data Visualization. Plot the data using programming software, drawing a T-shape with a blue line. 2 A line graph of the T statistic and the SPE statistic, with the red line representing T. 2 The control limits for the SPE statistic and the SCR statistic indicate that data exceeding the red line signifies a fault, while data below the control limits represents normal operating conditions. By comparing labeled industrial process data, the location of the fault can be determined. To quantify the monitoring effectiveness of the proposed model, two indicators are introduced: false alarm rate and false negative rate. The false alarm rate is the proportion of sample data that is actually under normal operating conditions but is incorrectly predicted as fault data. The false negative rate is the proportion of sample data that is actually under fault conditions but is incorrectly predicted as normal sample data. Using this method, the false alarm rate and false negative rate can be reduced by more than 30% in industrial processes. The lightweight deep principal component analysis-autoencoder model employs a hierarchically cascaded multi-layer PCA-AE model, which can not only handle the monitoring of large and complex industrial processes but also learn deep information from industrial process data, deeply mine key latent variables, and improve the model's applicability to the data.

[0058] The above description is merely an example of the overall structure of the present invention. The present invention may have many other embodiments. Without departing from the framework and essence of the present invention, any changes or additions to the above specific embodiments made by those skilled in the art, or the use of similar methods to replace them, should fall within the protection scope of the present invention.

Claims

1. An industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model, characterized in that, include: Online data collection of relevant process variables from industrial processes is performed. The collected data is preprocessed and then input into a trained lightweight deep principal component analysis-autoencoder model to obtain data scores and reconstruct the data. The data scores are then used to obtain T0. 2 The SPE statistic is obtained by using the reconstructed data, compared with the corresponding control limit, and the monitoring result is output. The lightweight deep principal component analysis-autoencoder model consists of a PCA dimensionality reduction module and an autoencoder. The PCA dimensionality reduction module is used to reduce the dimensionality of the input data to obtain the data score. The score is input into the autoencoder, and the autoencoder outputs the reconstructed data. The data is reduced in dimensionality using SVD decomposition; The lightweight deep principal component analysis-autoencoder model is a multi-layer structure. The last layer consists of the PCA dimensionality reduction module and the autoencoder, and the remaining layers consist of the PCA dimensionality reduction module and the local autoencoder. The local autoencoder consists of an input layer and a hidden layer. In a multi-layer structure, the hidden layer input of the autoencoder or local autoencoder of the previous layer is given to the next layer as the input of the PCA dimensionality reduction module of that layer. The number of layers in the multi-layer structure is 3 to 5; T 2 The statistics are obtained from the score T of the last layer; the SPE statistics are calculated from the residuals of the reconstructed data after decoding by the last layer autoencoder, and the calculation formula is as follows: ； ； in Let T be the score vector of the sample in the principal component space, specifically obtained by projecting the score T onto the principal component space. for The transpose of the matrix, The square of the diagonal matrix after SVD decomposition. Is to take forward Column; e is the residual matrix; The data for x after model reconstruction; The industrial process monitoring method described herein is suitable for process monitoring in complex industries.

2. The industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model according to claim 1, characterized in that, T 2 Control limits T of the statistic 2 lim The control limit SPE statistic is obtained by calculating the pseudo-inverse. lim It was calculated using the chi-square distribution.

3. The industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model according to claim 1, characterized in that, The industrial process in question is a papermaking wastewater process.

4. The industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model according to claim 1, characterized in that, The relevant process variable data include one or more of the following: temperature, flow rate, pressure, chemical oxygen demand, suspended solids value, and pH value.

5. The industrial process monitoring method based on a lightweight deep principal component analysis-autoencoder model according to claim 1, characterized in that, The preprocessing is a standardization process, which means that for each variable data, the mean of its training sample is subtracted, and then the result is divided by the standard deviation of its training sample to obtain a new variable data with a mean of 0 and a standard deviation of 1.

Images