Intermittent process soft measurement modeling method based on CGCN

Abstract

The invention discloses a CGCN-based batch process soft measurement modeling method, which comprises the following steps of: standardizing and reconstructing batch production process historical data by using a Z score to obtain a preprocessed batch production process data set; then constructing an intermittent process variable causal structure learning module, learning global and local causal relationships among variables, obtaining a causal adjacency matrix among the variables, introducing a space-time attention mechanism into GCN, distributing attention weights for different variables and sampling points of the intermittent process, and extracting causal features of process data; and finally, designing a joint loss function, and establishing a CGCN-based batch process soft measurement model by using the preprocessed batch process data to realize online prediction of batch production process quality variables. According to the method, the non-stationary characteristic of the batch production process and the Euclidean property of the process data are fully considered, the causal characteristics of the batch production process data are extracted, the batch process soft measurement model is established, and the modeling precision of batch process soft measurement is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of intermittent process monitoring, and in particular relates to an intermittent process soft measurement modeling method based on CGCN (Causal Graph Convolutional Network). Background Art

[0002] As an important production method in industrial production, intermittent process is widely used in fine chemicals, biopharmaceuticals, microelectronics and other fields. Accurate measurement of key variables in intermittent process can provide a data basis for online monitoring and optimization control of intermittent process. Data-driven soft measurement modeling does not require prior knowledge of the process. It uses process history data to establish a model between auxiliary variables and dominant variables, and has been widely used in intermittent process soft measurement modeling.

[0003] The non-stationary characteristics of intermittent processes directly affect the accuracy of soft sensor modeling based on multivariate statistical analysis such as principal component regression, partial least squares, and Gaussian process regression. Soft sensor modeling based on deep learning improves the accuracy and generalization of soft sensor models with its powerful learning and nonlinear fitting capabilities. The actual intermittent process data is structural data with Euclidean properties. Soft sensor modeling based on graph convolutional network (GCN) describes the topological structure of the graph through the adjacency matrix, which can capture the complex coupling relationship between variables and has high modeling accuracy. It has been used for intermittent process soft sensor modeling. However, since different production cycles of intermittent processes are not strictly repeated, there are differences in the distribution of process data between batches. The correlation adjacency matrix obtained from the training data is difficult to accurately describe the coupling relationship between actual variables, which directly affects the accuracy of intermittent process soft sensor modeling. When the process of the intermittent production process remains unchanged, the causal relationship between variables is stable in each production batch. The adjacency matrix constructed using the causal relationship between variables is close to the first principle of the intermittent production process. Therefore, a CGCN-based soft measurement modeling method for intermittent processes was invented. The causal structure learning module of intermittent production process variables was constructed to obtain the causal adjacency matrix between variables. Spatiotemporal attention was introduced into GCN to extract the causal characteristics of intermittent process data. A CGCN-based soft measurement model for intermittent processes was established, which can effectively improve the accuracy of soft measurement modeling for intermittent processes. Summary of the invention

[0004] The present invention aims to improve the accuracy of soft-sensor modeling for intermittent processes and invents a soft-sensor modeling method for intermittent processes based on CGCN, comprising the following steps:

[0005] Step 1: Collect multiple batches of intermittent process data, standardize the intermittent process data using the Z-score method, and evenly divide the multiple batches of intermittent process data using non-overlapping time windows to obtain the preprocessed process data set;

[0006] Step 2: Construct a causal structure learning module for intermittent process variables, learn the global and local causal relationships between process variables, obtain the causal adjacency matrix between process variables, introduce spatiotemporal attention into GCN, calculate the attention weights of different variables and sampling points in the intermittent process, and extract the causal features of the intermittent process data;

[0007] Step 3: Design a joint loss function and use the preprocessed intermittent process dataset to establish an intermittent process soft sensor model based on CGCN;

[0008] Step 4: Use the established CGCN-based soft-sensing model for intermittent processes to perform online prediction of the dominant variables of the intermittent processes and obtain the predicted values ​​of the dominant variables.

[0009] The step 1 specifically includes:

[0010] Collecting data from multiple batches of intermittent production processes Where I is the number of batch process data, J is the number of process data sampling points, and N is the number of process variables. X is standardized along the sampling time direction according to formula (1).

[0011]

[0012] In the formula, is the standardized intermittent production process data; mean(·) is the mean of the intermittent production process data X along the sampling time direction; std(X) is the standard deviation of the intermittent production process data X along the sampling time direction.

[0013] Then, using non-overlapping time windows, the standardized I batch process data Evenly divide the time into time slices of length T to obtain the preprocessed data set Among them, M=I×J / T is the number of time slices.

[0014] The step 2 specifically includes:

[0015] The constructed global causal relationship learning module between variables is expressed as:

[0016]

[0017] Where A is the adjacency matrix of intermittent production process variables; is the Gaussian noise matrix; f1(·) and f2(·) are nonlinear mapping functions; I is the identity matrix; T is the matrix transpose symbol.

[0018] Then, the variational autoencoder is used to obtain intermittent production process data. The potential representation of the variational autoencoder is to maximize the evidence lower bound ELBO:

[0019]

[0020] In the formula, For a given Gaussian noise Z, the intermittent production process data The conditional probability distribution of ; Given intermittent production process data , the conditional probability distribution of Gaussian noise Z; p(Z) is the probability distribution of Gaussian noise Z; log(·) is the logarithmic operation; Ε q [·] and D KL [·||·] respectively represent the expectation and KL divergence; Max represents the maximum value.

[0021] The two nonlinear mapping functions contained in formula (2) are functionally identical, and the inference model of the variational autoencoder is and generative models They are defined as:

[0022]

[0023] In the formula, f encoder (·) and f decoder (·) denote the encoder and decoder of the variational autoencoder, respectively.

[0024] The causal structure of variables in intermittent production processes is directed acyclic, and the global causal structure learning problem between variables is defined as:

[0025]

[0026] Where θ is the model parameter of the variational autoencoder; represents the Hadamard product; tr[·] is the trace of the matrix; st represents the constraint condition.

[0027] Then, a variable local causal relationship learning module is constructed to capture the short-term fluctuations in the intensity of causal effects between variables. Refactored to Then the process data X * After the tensor output by the three-layer convolution is transposed, the local features of the process variables are obtained. Then, Einstein summation is introduced to calculate the local causal adjacency matrix of variables in the intermittent production process The calculation formula is:

[0028]

[0029] Where i, l and k are the three dimensions of the local characteristics of the process variables; Indicates H * The element with coordinates (i,l,k) in .

[0030] Through the global causal relationship learning module between variables and the local causal relationship learning module between variables, the global causal adjacency matrix A and the local causal adjacency matrix ΔA of the variables are obtained, and then the two adjacency matrices are fused to obtain the final causal adjacency matrix A between variables. * , the fusion process is:

[0031] A * =Norm(RELU(A+ΔA)) (7)

[0032] Where RELU(·) is the RELU activation function; Norm(·) is the normalization operation.

[0033] In GCN, spatiotemporal attention is introduced. The spatiotemporal attention module includes the temporal attention module and the spatial attention module. By introducing temporal attention, the importance of each time slice can be obtained. The temporal attention matrix It is expressed as:

[0034] TA=V t σ((H T Q1)Q2(Q3H)+b t ) (8)

[0035] In the formula, is the input process data; and is a learnable parameter; σ is an activation function.

[0036] By introducing spatial attention, the importance of each process variable can be obtained. The spatial attention matrix It is expressed as:

[0037] SA=V s σ((H T W1)W2(W3H)+b s ) (9)

[0038] In the formula, and are learnable parameters.

[0039] The temporal attention matrix TA and the spatial attention matrix SA are normalized by the Softmax function as follows: and Make the sum of all its elements equal to 1.

[0040] Intermittent production process data After the spatial attention module, the spatial attention matrix is ​​obtained X and After matrix multiplication, input the temporal attention module to obtain the temporal attention matrix Then combine the output of the previous step with Multiply them and then input them into GCN to obtain the causal features of the process data.

[0041] The step three specifically includes:

[0042] The loss function of the intermittent process soft sensor model based on CGCN is composed of the root mean square loss Loss RMSE , Reconstruction loss Loss RC , KL divergence loss Loss KL and directed acyclic loss Loss H The calculation formula of each loss function is:

[0043]

[0044] Loss=Loss RMSE +Loss RC +Loss KL +Loss H (14)

[0045] In the formula, and j is the predicted value and true value of the jth sampling point in the intermittent production process; is the output value of the decoder in the variational self-encoder; is a directed acyclic function.

[0046] Leverage standardized batch process data The model is trained in an end-to-end manner to obtain a CGCN-based soft-sensing model for intermittent processes.

[0047] The step 4 specifically includes:

[0048] For intermittent production processes, online data online First, X online Normalized to Then, the established CGCN-based soft sensor model for intermittent processes is used to Make a prediction and get the predicted value of the leading variable y for:

[0049]

[0050] Where model(·,w) is the obtained soft sensor model of intermittent process based on CGCN, and w is the model parameter.

[0051] Advantages of the present invention: The present invention fully considers the influence of the non-stationary characteristics of the intermittent process on the soft measurement modeling, and the constructed causal structure learning module between variables of the intermittent production process can learn the correct causal relationship, obtain the causal adjacency matrix between variables, introduce spatiotemporal attention in GCN, make the model pay more attention to important variables and time slices, extract the causal characteristics of the intermittent process data, establish an intermittent process soft measurement model based on CGCN, realize the online prediction of the dominant variables of the intermittent process, and improve the prediction accuracy and stability of the intermittent process soft measurement model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a structural diagram of the intermittent process soft sensor model based on CGCN according to the present invention;

[0053] Figure 2 It is the visualization result of the causal structure between variables learned by the model and the real causal structure between variables in the penicillin fermentation process;

[0054] Figure 3 This is a diagram of the prediction results of the soft measurement model established by the method of the present invention on 5 batches of penicillin fermentation process. DETAILED DESCRIPTION

[0055] The present invention is further described below in conjunction with examples and drawings. It should be noted that the embodiments do not limit the scope of protection claimed by the present invention.

[0056] Example

[0057] The penicillin production process is a typical intermittent production process. When the sampling time range and sampling interval were set to 400 hours and 1 hour, 20 batches of process data were generated using the penicillin fermentation process simulation platform PensimV2.0, of which 15 batches of process data were used for model training and 5 batches of process data were used for soft measurement model testing. The variables for soft measurement modeling of penicillin fermentation process are shown in Table 1, where penicillin concentration was selected as the dominant variable and the remaining 11 variables were selected as auxiliary variables.

[0058] Table 1 Soft sensor modeling variables for penicillin fermentation process

[0059]

[0060] The model structure diagram of the present invention is as follows: Figure 1 As shown, the specific steps are as follows:

[0061] Step 1: First, use z-score to standardize the batch production process data set The 400-hour penicillin fermentation process data of 15 batches collected were evenly divided into 4-hour time slices using non-overlapping time windows to obtain the preprocessed data set of the intermittent production process.

[0062] Step 2: Construct a causal structure learning module for intermittent production process variables, obtain the global causal relationship and local causal relationship between intermittent production process variables, obtain the global causal adjacency matrix A and the local causal adjacency matrix ΔA of the intermittent production process variables, and then fuse the two adjacency matrices according to formula (7) to obtain the final causal adjacency matrix A between intermittent production process variables. * ; Then, the spatiotemporal attention module is introduced into GCN to calculate the temporal attention weight and spatial attention weight respectively, and extract the causal characteristics of the intermittent production process data;

[0063] Step 3: Using the preprocessed dataset from the batch production process Training the model to obtain a trained intermittent process soft-sensing model;

[0064] Step 4: Input the test data into the model to obtain the online prediction results of penicillin concentration.

[0065] The Adam optimizer is selected for gradient descent. In order to improve the convergence speed of the soft sensor model, a dynamic learning rate adjustment mechanism is introduced. The initial value of the learning rate is 0.1. After every 50 rounds of training, the learning rate is reduced to the previous 0.7. The batch size is 500 and the number of epochs is 500.

[0066] In order to verify the effectiveness of the present invention, the method of the present invention is compared with the soft sensor modeling methods of LSTM, GRU, GCN, and STGCN. The root mean square error RMSE and the determination coefficient R 2 To evaluate the performance of the soft sensor model, the calculation formula is:

[0067]

[0068] In the formula, and j represents the predicted value and true value of penicillin concentration at the jth sampling point; is the average value of penicillin concentration at all sampling points.

[0069] In addition, in order to evaluate the stability of the soft sensor model, the stability error SE and stability coefficient SF are constructed, and the calculation formula is:

[0070]

[0071] In the formula, RMSE e and AE and AR represent the average RMSE and average R of all test batches. 2 The smaller the values ​​of stability error SE and stability coefficient SF are, the better the prediction stability of the soft sensor model is.

[0072] The soft sensor model established by this method learns to obtain the visualization results of the causal structure between auxiliary variables. Figure 2 As shown, Figure 2 a) is obtained based on the mechanism equation of penicillin fermentation process, and b) is the causal structure diagram obtained by model learning after binarization. Among them, the white square indicates that there is a causal relationship between the two variables, and the black square indicates that there is no causal relationship between the two variables. The red box in the figure indicates the causal relationship obtained by model learning that is consistent with the actual situation, and the direction of the causal relationship points from the vertical axis variable to the horizontal axis variable. It can be seen from the figure that the causal structure learning module can obtain the causal relationship between most process variables only through process data learning, which shows the effectiveness of the causal structure learning module constructed by the present invention.

[0073] The prediction results of penicillin concentration by different soft sensor models on different batches are shown in the figure below. Figure 3 The evaluation indicators of the model are shown in Tables 2 and 3.

[0074] Table 2 RMSE and R of soft sensor models for penicillin fermentation process constructed by different methods 2

[0075]

[0076] Table 3 Stability indexes of soft sensor models of penicillin fermentation process constructed by different methods

[0077]

[0078] The soft measurement modeling of penicillin fermentation process based on GCN takes into account the Euclidean characteristics of process data, and the prediction results are greatly improved compared with LSTM and GRU. The soft measurement modeling of penicillin fermentation process based on ASTGCN introduces spatiotemporal attention on the basis of GCN, which more fully extracts the information of original process data, and the prediction results are better than the soft measurement model of penicillin fermentation process based on GCN. The soft measurement modeling of the present invention makes full use of the causal relationship between auxiliary variables, combines the spatiotemporal attention mechanism, extracts causal features and establishes a soft measurement model of penicillin concentration, which has the smallest RMSE and the highest R in each batch. 2 At the same time, the stability error and stability coefficient between different batches are the smallest, which proves that the soft sensor model built by the method of the present invention has high modeling accuracy.

Claims

1. A CGCN-based soft-sensor modeling method for intermittent processes, characterized by: The method comprises the following steps: Step 1: Collect multiple batches of intermittent process data, standardize the intermittent process data using the Z-score method, and evenly divide the multiple batches of intermittent process data using non-overlapping time windows to obtain the preprocessed process data set; Step 2: Construct a causal structure learning module for intermittent process variables, learn the global and local causal relationships between process variables, obtain the causal adjacency matrix between process variables, introduce spatiotemporal attention into GCN, calculate the attention weights of different variables and sampling points in the intermittent process, and extract the causal features of the intermittent process data; Step 3: Design a joint loss function and use the preprocessed intermittent process dataset to establish an intermittent process soft sensor model based on CGCN; Step 4: Use the established CGCN-based soft-sensing model for intermittent processes to perform online prediction of the dominant variables of the intermittent processes and obtain the predicted values ​​of the dominant variables.

2. The CGCN-based intermittent process soft sensor modeling method according to claim 1, characterized in that: The step 1 specifically includes: Collecting data from multiple batches of intermittent production processes Where I is the number of batch process data, J is the number of process data sampling points, and N is the number of process variables; X is standardized along the sampling time direction according to formula (1); In the formula, is the standardized intermittent production process data; mean(·) is the mean of the intermittent production process data X along the sampling time direction; std(X) is the standard deviation of the intermittent production process data X along the sampling time direction; Then, using non-overlapping time windows, the standardized I batch process data Evenly divide the time into time slices of length T to obtain the preprocessed data set Among them, M=I×J / T is the number of time slices.

3. The CGCN-based intermittent process soft sensor modeling method according to claim 2, characterized in that: The step 2 specifically includes: The constructed global causal relationship learning module between variables is expressed as: Where A is the adjacency matrix of intermittent production process variables; is the Gaussian noise matrix; f1(·) and f2(·) are nonlinear mapping functions; I is the identity matrix; T is the matrix transpose symbol; Then, the variational autoencoder is used to obtain intermittent production process data. The potential representation of the variational autoencoder is to maximize the evidence lower bound ELBO: In the formula, For a given Gaussian noise Z, the intermittent production process data The conditional probability distribution of ; Given intermittent production process data , the conditional probability distribution of Gaussian noise Z; p(Z) is the probability distribution of Gaussian noise Z; log(·) is the logarithmic operation; Ε q [·] and D KL [·||·] respectively represent the expectation and KL divergence; Max represents the maximum value; The two nonlinear mapping functions contained in formula (2) are functionally identical, and the inference model of the variational autoencoder is and generative models They are defined as: In the formula, f encoder (·) and f decoder (·) denotes the encoder and decoder of the variational autoencoder respectively; The causal structure of variables in intermittent production processes is directed acyclic, and the global causal structure learning problem between variables is defined as: Where θ is the model parameter of the variational autoencoder; represents the Hadamard product; tr[·] is the trace of the matrix; st represents the constraint condition; Then, a variable local causal relationship learning module is constructed to capture the short-term fluctuations in the intensity of causal effects between variables; the intermittent production process standardized data Refactored to Then the process data X * After the tensor output by the three-layer convolution is transposed, the local features of the process variables are obtained. Then, Einstein summation is introduced to calculate the local causal adjacency matrix of variables in the intermittent production process The calculation formula is: Where i, l and k are the three dimensions of the local characteristics of the process variables; Indicates H * The element with coordinates (i,l,k) in the Through the global causal relationship learning module between variables and the local causal relationship learning module between variables, the global causal adjacency matrix A and the local causal adjacency matrix ΔA of the variables are obtained, and then the two adjacency matrices are fused to obtain the final causal adjacency matrix A between variables. * , the fusion process is: A * =Norm(RELU(A+ΔA)) (7) Where RELU(·) is the RELU activation function; Norm(·) is the normalization operation; In GCN, spatiotemporal attention is introduced. The spatiotemporal attention module includes the temporal attention module and the spatial attention module. By introducing temporal attention, the importance of each time slice can be obtained. The temporal attention matrix It is expressed as: TA=V t σ((H T Q1)Q2(Q3H)+b t ) (8) In the formula, is the input process data; and is a learnable parameter; σ is an activation function; By introducing spatial attention, the importance of each process variable can be obtained. The spatial attention matrix It is expressed as: SA=V s σ((H T W1)W2(W3H)+b s ) (9) In the formula, and are learnable parameters; The temporal attention matrix TA and the spatial attention matrix SA are normalized by the Softmax function as follows: and Make the sum of all its elements equal to 1; Intermittent production process data After the spatial attention module, the spatial attention matrix is ​​obtained X and After matrix multiplication, input the temporal attention module to obtain the temporal attention matrix Then combine the output of the previous step with Multiply them and then input them into GCN to obtain the causal features of the process data.

4. The CGCN-based intermittent process soft sensor modeling method according to claim 3, characterized in that: The step three specifically includes: The loss function of the intermittent process soft sensor model based on CGCN is composed of the root mean square loss Loss RMSE , Reconstruction loss Loss RC , KL divergence loss Loss KL and directed acyclic loss Loss H The calculation formula of each loss function is: Loss=Loss RMSE +Loss RC +Loss KL +Loss H (14) In the formula, and j is the predicted value and true value of the jth sampling point in the intermittent production process; is the output value of the decoder in the variational self-encoder; is a directed acyclic function; Leverage standardized batch process data The model is trained in an end-to-end manner to obtain a CGCN-based soft-sensing model for intermittent processes.

5. The CGCN-based intermittent process soft sensor modeling method according to claim 4, characterized in that: The step 4 specifically includes: For intermittent production processes, online data online First, X online Normalized to Then, the established CGCN-based soft sensor model for intermittent processes is used to Make a prediction and get the predicted value of the leading variable y for: Where model(·,w) is the obtained soft sensor model of intermittent process based on CGCN, and w is the model parameter.

Images