Multi-modal process soft measurement modeling method based on domain migration space-time diagram network

Abstract

The invention provides a multi-modal process soft measurement modeling method based on a domain migration space-time diagram network, and belongs to the field of process industry multi-modal process soft measurement modeling. The specific process of the modeling method is as follows: firstly, a basic model is established on available source domain data, graph convolution parameters are used for extracting spatial features of variables, and a gating mechanism is used for extracting time sequence features among the variables; a self-supervised learning method based on a disturbance matrix is designed, and generalization and robustness of the model are improved; the conditional distribution difference of the embedded data calibrates model parameters under a fine tuning mechanism, and expands the prediction range of the model.

Description

Technical Field

[0001] The present invention belongs to the field of multimodal process soft sensor modeling in process industry, and in particular relates to a multimodal process soft sensor modeling method based on a domain migration spatiotemporal graph network. Background Art

[0002] As industrial production processes become increasingly complex, it is difficult to directly measure some key variables with physical sensors due to the limitations of measurement costs and physical conditions. Data-driven soft sensing methods estimate dominant variables by establishing a mapping relationship between easily measured auxiliary variables and difficult-to-measure dominant variables. As an effective solution, it has been widely used in process industries.

[0003] Data-driven methods based on deep learning have shown good feature extraction capabilities in historical data mining, such as convolutional neural networks and autoencoders. In fact, due to physical connections such as sensors, there are dependencies between variables in the production process. For example, signal flow and material flow can reflect whether one variable is related to another variable. In general, the manipulated variable and the controlled variable in the same control loop will show strong dependencies. In addition, due to the transmission of materials, upstream variables will affect downstream variables. Capturing the relationship between structural dependencies between variables has been shown to be beneficial for process modeling of multivariate data. Traditional deep learning methods cannot capture this relationship well because they are designed for data in Euclidean space, while the variable dependencies of actual production processes exist in the form of graphs. As a hot research method, graph networks can mine the coupling information between variables by constructing a relationship graph of variables. As a variant of traditional graph networks, graph convolution networks (GCNs) have powerful representation capabilities by performing convolution operations on structural data. However, while capturing the coupling characteristics of variables, another important system characteristic still needs to be considered, namely dynamic characteristics. Due to loops and feedback, production systems are dynamic systems, which means that capturing the temporal dependencies between variables is very important for describing process behavior. Therefore, it is of great significance to process modeling to simultaneously capture the spatial and temporal characteristics of process data.

[0004] However, due to various factors, such as changes in raw materials and composition, different manufacturing strategies, fluctuations in the external environment, and various product specifications, the production process often presents a multimodal process. Data collected under different modalities usually have problems such as covariance drift and probability distribution differences, which leads to the model established for a specific modality in multimodal cases not being applicable to other modalities, ultimately resulting in a decrease in model performance. In addition, there is not always enough labeled data available in each modality, and sometimes it is difficult to collect enough data for a specific modality and retrain the model. Summary of the invention

[0005] In order to solve the problem of building a robust soft sensor model in industrial multimodal scenarios, this paper develops a regression framework based on the domain transfer spatiotemporal graph network (DTSGN) and builds a basic model on the available source domain data, where the GCN parameters are used to extract the spatial features of the variables and the gated recurrent unit (GRU) is used to extract the temporal features between the variables. A self-supervised learning method based on the perturbation matrix is ​​designed to improve the generalization and robustness of the model. Furthermore, the conditional distribution difference constraint of the embedded data captures the data characteristics available in the target domain under the fine-tuning mechanism. Therefore, a robust transfer spatiotemporal graph model is established, which is suitable for process modeling of new modalities. Finally, the performance advantages of the proposed method are demonstrated in the soft sensor modeling of the sewage treatment process.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A multimodal process soft sensor modeling method based on domain migration spatiotemporal graph network uses graph convolution and gating mechanism to capture spatiotemporal characteristics; adds dynamic perturbation matrix to enhance the generalization and robustness of the model; and reduces the prediction bias of the model based on the difference of embedded conditional distribution. The method includes the following steps:

[0008] (1) Obtaining data

[0009] The source domain and target domain data are obtained, and the model input data matrix is ​​obtained through the sliding method of the time window.

[0010] (2) Training the source domain model

[0011] The training is performed in an end-to-end manner to capture relevant soft graph structure information that helps predict the output variables. The network composed of each time step is used to predict the output variable, and the prediction loss and graph structure constraints are used as the training loss function to regulate the source domain training process.

[0012] (3) Training target model

[0013] The conditional distribution difference constraint between the embedded target domain data is used to fine-tune the network parameters with the prediction loss of a small amount of target data.

[0014] (4) Model performance evaluation

[0015] The root mean square error (RMSE), mean absolute percentage error (MAE) and mean absolute percentage error (MAPE) are used to evaluate the performance of the model.

[0016] Further, the process of (1) is as follows:

[0017] The source domain S has relatively sufficient labeled data fields, are the i-th process data and output data in the source domain respectively. The target domain T has a small amount of labeled data fields, are the jth process data and output data of the target domain respectively. The source domain and target domain have N and M sample sizes respectively, where N>>M, and the target domain test sample The number is K, where l represents labeled data and u represents unlabeled data. In order to realize process soft sensor modeling, a sliding window of size w is used to process historical data based on the sliding method of time window, where the data after the source domain sliding window is expressed as represents the source domain process data at time tw, Respectively represent the source domain variable matrix and the predicted key variable at time t. The target domain data after sliding window is expressed as represents the target domain process data at time tw, They represent the target domain process variables and prediction key variables at time t respectively.

[0018] Further, the process of (2) is as follows:

[0019] Step 2.1: First, train the spatiotemporal graph network model based on the available source domain data, and add the perturbation matrix S following the Bernoulli distribution to the graph relationship matrix A captured by the model, so as to train a relationship matrix Z with better generalization performance.

[0020] The perturbation graph information is mainly based on random vector matrices to achieve self-supervision tasks. Assume that we have a predefined perturbation matrix S = {s ij} i、j＝1,...,m , m represents the matrix dimension, follows the Bernoulli distribution, and generates a perturbation matrix S, in which the elements can change randomly with a certain probability. The specific formula is as follows:

[0021] s ij ~Bernoulli(p)(1)

[0022] P(s ij =1) = p (2)

[0023] P(s ij =0) = 1-p (3)

[0024] Where p is the probability value, Bernoulli(p) means that each element is independently sampled from the Bernoulli distribution, and a perturbation matrix S is obtained. Further, perturbations are added to the graph relationship matrix A of the original model to form a new model relationship matrix Z, as shown below:

[0025] Z=A+S (4)

[0026] Each element in the matrix S can be randomly adjusted according to the specified probability to ensure that different perturbation effects can be obtained at different training stages. By adding perturbation node features, the generalization ability and robustness of the model are improved.

[0027] Furthermore, in the graph convolution operation, the spatial coupling relationship between variables is captured. The specific calculation process is as follows:

[0028]

[0029] Among them, GCN(X G ,Z) represents the graph convolution operation, X G Represents the input of the graph convolution layer. D=∑ j Z j is the degree matrix of the adjacency matrix Z, Ψ is a weight matrix. conv represents the convolution operation, and σ is the activation function.

[0030] Step 2.2: Capture the temporal coupling relationship between variables under the GRU gating mechanism.

[0031] Step 2.3: RMSE is used as the training error evaluation indicator of the source domain model to train the source domain model parameters. The specific loss formula is Loss S as follows:

[0032]

[0033] in and are the i-th source domain samples respectively The model fitting value and true value of , f(·) represents the nonlinear relationship learned by the model, where N is the number of source domain training samples. τ represents the regularization coefficient, Represents element-wise entropy.

[0034] Further, the process of (3) is as follows:

[0035] After the source domain model training is completed, the source domain trained model is obtained, and then the network parameters are fine-tuned based on a small amount of target data. The loss of the specific target domain labeled data is T for:

[0036]

[0037] Where M is the amount of training data in the target domain, represents the jth sample The model's predicted output, is the true value.

[0038] Considering the scarcity of labeled data in the target domain, this may lead to overfitting of the model. The output of the fully connected layer, i.e., the feature map, is used as the new input variable F x , based on the Conditional embedding operator discrepancy (CEOD) module, to measure the target domain labeled data and target domain unlabeled data Differences in conditional distributions at different layers in Represent the feature maps of the target domain training data and test data, respectively. Represents the model's predicted value for the test data. As a semi-supervised paradigm, it is used to capture the available target domain data characteristics. The specific total loss function Loss total As shown below:

[0039]

[0040] in is the model’s prediction of the target domain training data, is the model's predicted value for the target domain test data, β is the coefficient, This is an empirical evaluation operation.

[0041] Further, the process of (4) is as follows:

[0042] The root mean square error (RMSE), mean absolute percentage error (MAE) and mean absolute percentage error (MAPE) are used to evaluate the performance of the model. The specific calculation formula is as follows:

[0043]

[0044]

[0045]

[0046] where y k , They represent the kth true value and predicted value respectively, and the test sample size of the target domain is K. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the DTSGN framework diagram proposed by the present invention;

[0048] Figure 2 It is the sewage case prediction result diagram; (a) is M1→M2, (b) is M1→M3;

[0049] Figure 3 is the box plot of absolute prediction error for the sewage case; (a) is M1→M2, (b) is M1→M3. DETAILED DESCRIPTION

[0050] The present invention will be further described below in conjunction with the accompanying drawings.

[0051] Reference Figure 1 to Figure 3 , a multimodal process soft sensor modeling method based on domain migration spatiotemporal graph network, the method comprising the following steps:

[0052] (1) Obtaining data

[0053] The three modal data of the simulated sewage treatment process are divided into source domain and target domain data.

[0054] (2) Training the source domain model

[0055] The training is performed in an end-to-end manner to capture relevant soft graph structure information that helps predict the output variables. The network composed of each time step is used to predict the output variable, and the prediction loss and graph structure constraints are used as the training loss function to regulate the source domain training process.

[0056] (3) Training target model

[0057] Embedding conditional distribution difference to fine-tune network parameters with prediction loss on a small amount of target data.

[0058] (4) Model performance evaluation

[0059] The root mean square error (RMSE), mean absolute percentage error (MAE) and mean absolute percentage error (MAPE) indicators are used to evaluate the performance of the model.

[0060] Specifically, the process of step (1) is:

[0061] Table 1 Three modes under drought weather

[0062]

[0063] The sewage treatment process is a typical nonlinear, multivariable, and time-consuming process. The present invention uses a simulated sewage treatment plant based on the simulation basic model 1 to verify the effectiveness of the method. First, the dynamic drought weather inflow data is set as the input, and the sampling is performed every 15 minutes, and 672 times are collected in 7 days. Three modes are simulated by changing three flow rates and three oxygen transfer coefficients, which are recorded as M1, M2, and M3 respectively. In order to simulate the real industrial scene, 5% Gaussian noise is added to each mode. Table 1 lists the detailed parameters. The eight variables of process variables, such as soluble inert organic matter, biodegradable matrix, dissolved oxygen, nitrate and nitrite nitrogen, ammonium plus ammonia nitrogen, soluble biodegradable organic nitrogen, alkalinity, and total suspended solids concentration, are used as input variables, and heterotrophic biomass is predicted as a quality variable.

[0064] M1, M2, and M3 are combined in pairs and set as the source domain and target domain respectively. There are 672 source domain training samples, 20 labeled training samples are set for the target domain, and the remaining 652 are test samples.

[0065] Specifically, the process of step (2) is:

[0066] Step 2.1: First, train the spatiotemporal graph network model based on the available source domain data, and add the perturbation matrix S following the Bernoulli distribution to the relationship matrix A captured by the model, so as to train a relationship matrix with better generalization performance.

[0067] The perturbation graph information is mainly based on random vector matrices to achieve self-supervision tasks. Assume that we have a predefined perturbation matrix S = {s ij} i、j＝1,...,m , m represents the matrix dimension, follows the Bernoulli distribution, and generates a perturbation matrix S, in which the elements can change randomly with a certain probability. The specific formula is as follows:

[0068] s ij ~Bernoulli(p)(13)

[0069] P(s ij =1) = p (14)

[0070] P(s ij =0) = 1-p (15)

[0071] Where p is the probability value, Bernoulli(p) means that each element is independently sampled from the Bernoulli distribution, and a perturbation matrix S is obtained. Further, perturbations are added to the graph parameter matrix A of the original model to form a new variable relationship matrix Z, as shown below:

[0072] Z=A+S (16)

[0073] Each element in the matrix S can be randomly adjusted according to the specified probability to ensure that different perturbation effects can be obtained at different training stages. By adding perturbation node features, the generalization ability and robustness of the model are improved.

[0074] Furthermore, in the graph convolution operation, the spatial coupling relationship between variables is captured. The specific calculation process is as follows:

[0075]

[0076] Among them, GCN(X G ,Z) represents the graph convolution operation, X G Represents the input of the graph convolution layer. D=∑ j Z j is the degree matrix of the adjacency matrix Z, Ψ is a weight matrix. conv represents the convolution operation, and σ is the activation function.

[0077] Step 2.2: Further capture the temporal coupling relationship between variables under the gating mechanism of GRU. The specific formula is as follows:

[0078] u i =σ(GCN u ([f i ,h i-1 ],Z i )+b u ) (18)

[0079] r i =σ(GCN r ([f i ,h i-1 ],Z i )+b r ) (19)

[0080] c i =tanh(GCN c ([f i ,(r i *h i-1 ),Z i ])+b c ) (20)

[0081] h i =u i*h i-1 +(1-u i )*c i (twenty one)

[0082] Among them, GCN(·) represents the graph convolution mechanism, f i , Z i They are the output of the graph convolution operation at the i-th moment and the variable relationship matrix learned by the model. u , GCN r and GCN c Represents the input of the update gate, forget gate, and memory unit. u , b r and b c is the bias matrix. tanh(·) and σ(·) represent activation functions. At the i-th moment, h i-1 Represents the state of the hidden layer at the last moment, r i and u i Represent the learning states of the forget gate and the update gate, c i represents the state of the memory unit, h i Represents the output at the current moment.

[0083] Step 2.3: RMSE is used as the training error evaluation indicator of the source domain model to train the source domain model parameters. The specific loss formula is Loss S as follows:

[0084]

[0085]

[0086] in and are the i-th source domain samples respectively The model fitting value and true value of , f(·) represents the nonlinear relationship learned by the model, where N is the number of source domain training samples. τ represents the regularization coefficient, Represents element-wise entropy.

[0087] Specifically, the process of step (3) is:

[0088] After the source domain model is trained, the pre-trained model is obtained, and then the network parameters are fine-tuned based on the prediction loss of a small amount of target data. Let f(x) represent the regression prediction, and the loss of the target domain labeled data is Loss T for:

[0089]

[0090] Considering the scarcity of labeled data in the target domain, this may lead to overfitting of the model. The output of the fully connected layer, i.e., the feature map, is used as the new input variable Fx , based on the Conditional embedding operator discrepancy (CEOD) module, to measure the target domain labeled data and target domain unlabeled data Differences in conditional distributions at different layers It is a semi-supervised paradigm that captures the characteristics of available target domain data. The specific total loss function Loss total As shown below:

[0091]

[0092] in is the model’s prediction of the target domain training data, is the model's predicted value for the target domain test data, β is the coefficient, This is an empirical evaluation operation.

[0093] CEOD is used to calculate the conditional distribution difference between data. The detailed calculation process is as follows:

[0094]

[0095] in, is the model's predicted value for the target domain test data, K is the Gram matrix, which can be calculated using the Gaussian kernel. φ(·) and χ(·) are operation operators. and is the set of operations. η is the regularization coefficient, Tr(·) represents the trace of the calculation matrix, and I is the identity matrix.

[0096] Specifically, the process of step (4) is:

[0097] The following baseline models are used for comparison:

[0098] Model 1: Train the spatiotemporal graph network model based on available labeled data in the source domain.

[0099] Model 2: Train a spatiotemporal graph network model based on available labeled data in the target domain.

[0100] Domain adaption extreme learning machine (DAELM): The model is trained based on sufficient labeled data in the source domain and labeled data in the target domain.

[0101] Conditional distribution deep adaptation regression (CDAR): The source domain model is trained based on a convolutional neural network, the output layer parameters are fine-tuned with target domain labeled data, and conditional distribution difference constraints are added.

[0102] DTSGN: Transfer the parameters of the spatiotemporal graph network model trained in the source domain, then fine-tune the output layer parameters based on the labeled data in the target domain and add conditional distribution difference constraints.

[0103] The root mean square error (RMSE), mean absolute percentage error (MAE) and mean absolute percentage error (MAPE) indicators are used to evaluate the performance of the model. The formula is as follows:

[0104]

[0105] where y k , They represent the kth true value and predicted value respectively, and the test sample size of the target domain is K.

[0106] Table 2 Performance of baseline model and DTSGN in sewage case

[0107]

[0108] The prediction losses of the two specific cases are shown in Table 2. Figure 2 The prediction results and true value distribution of the two cases M1→M2 and M1→M3 are shown in Figure 2. It can be seen that the proposed DTSGN model has the best prediction performance, and the estimation of heterotrophic biomass is more consistent with the true value. Due to the influence of the distribution difference between modalities on the model, the model established in the source domain has a high prediction bias in the target domain, the model generalization performance is poor, and it cannot capture the characteristics of heterotrophic biomass in the target domain well, so the RMSE prediction index is the highest. Based on the small amount of available target domain data, DAELM calculates the model output weight by assigning weights to the domain, but due to the insufficient capture of the spatiotemporal correlation of variables by the model, the prediction performance is worse than DTSGN. Model2 aims to directly train the model based on the labeled data of the target domain. Due to the lack of available data information, the generalization performance of the model in the test data is limited. CDAR extracts variable features based on convolution operations under the condition of conditional distribution differences in embedded data. Due to its insufficient ability to capture the spatiotemporal relationship of variables, the prediction index is higher than DTSGN. Therefore, in all cases, DTSGN shows the advantages of soft sensor modeling in cross-modality.

[0109] The present invention takes into account the spatiotemporal coupling relationship of multimodal process data and the problem of model performance inaccuracy caused by distribution differences between different modes, and proposes a spatiotemporal graph network model based on domain migration. Specifically, the spatiotemporal graph model is trained in the source domain to capture the temporal and spatial coupling relationship between variables. In the spatiotemporal graph network model, the perturbation graph information is mainly based on random vector matrices to achieve self-supervision tasks, which improves the generalization performance and robustness of the model. Further, under the constraint of conditional distribution differences, the model performance is calibrated based on the fine-tuning mechanism, which expands the prediction range of the model, and verifies the advantages of the proposed method in the over-modal case of the sewage treatment process.

[0110] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A multimodal process soft sensor modeling method based on domain migration spatiotemporal graph network, the method comprising the following steps: (1) Obtaining data Obtain source domain and target domain data, and obtain the model input data matrix through the sliding method of time window; (2) Training the pre-trained model Pre-trained models are trained in an end-to-end manner to capture relevant soft graph structure information; The network composed of each time step is used to predict the output variable, and the prediction loss and graph structure constraints are used as the training loss function to regulate the source domain training process; (3) Training target model Embed the conditional distribution difference constraint between the target domain data and adjust the network parameters with the prediction loss of a relatively small amount of target data; (4) Model performance evaluation The root mean square error (RMSE), mean absolute percentage error (MAE), and MAPE indicators are used to evaluate the performance of the model.

2. The multimodal process soft sensor modeling based on domain migration spatiotemporal graph network according to claim 1 is characterized by: In (1), The source domain S has relatively sufficient labeled data fields, are the i-th process data and output data in the source domain respectively; the target domain T has a relatively small amount of labeled data fields, are the jth process data and output data of the target domain respectively; the source domain and target domain have N and M sample sizes respectively, where N>>M, and the target domain test sample The number is K, where l represents labeled data and u represents unlabeled data; the specific process of step (1) is as follows: In order to realize process soft sensor modeling, a sliding window of size w is used to process historical data based on the sliding method of time window, where the data after sliding window in source domain is expressed as represents the source domain process data at time tw, They represent the source domain variable matrix and the predicted key variables at time t respectively; the data after sliding window in the target domain is expressed as represents the target domain process data at time tw, They represent the target domain process variables and prediction key variables at time t respectively.

3. The multimodal process soft sensor modeling based on domain migration spatiotemporal graph network according to claim 2 is characterized by: The process of (2) is as follows: Step 2.1: Train the spatiotemporal graph network model based on the available source domain data, add the perturbation matrix S following the Bernoulli distribution to the relationship matrix A captured by the model, and thus train a relationship matrix Z with better generalization performance; the details are as follows S2.1.1, Assume a predefined perturbation matrix S = {s ij } i、j＝1,...,m , m represents the matrix dimension, follows the Bernoulli distribution, and generates a perturbation matrix S, in which the elements can change randomly with a certain probability. The specific formula is as follows: s ij ~Bernoulli(p)(1) P(s ij =1)=p (2) P(s ij =0)=1-p (3) Where p is the probability value, Bernoulli(p) means that each element is independently sampled from the Bernoulli distribution, and a perturbation matrix S is obtained; S2.1.

2. Add disturbances to the graph relationship matrix A of the original model to form a new model relationship matrix Z, as shown below: Z=A+S (4) Each element in the matrix S can be randomly adjusted according to the specified probability to ensure that different perturbation effects can be obtained at different training stages; S2.1.

3. In the graph convolution operation, the spatial coupling relationship between variables is captured. The specific calculation process is as follows: Among them, GCN(X G ,Z) represents the graph convolution operation, X G Represents the input of the graph convolution layer; D = ∑ j Z j is the degree matrix of the adjacency matrix Z, Ψ is a weight matrix; conv represents the convolution operation, and σ is the activation function; Step 2.2: Capture the temporal coupling relationship between variables under the GRU gating mechanism; Step 2.3: Use RMSE as the training error evaluation indicator of the source domain model to train the source domain model parameters. The specific loss formula is Loss S as follows: in and are the i-th source domain samples respectively The model fitting value and true value of , f(·) represents the nonlinear relationship learned by the model, where N is the number of source domain training samples; τ represents the regularization coefficient, Represents element-wise entropy.

4. The multimodal process soft sensor modeling based on domain migration spatiotemporal graph network according to claim 3 is characterized by: The process of step (3) is as follows: S3.

1. After the source domain model training is completed, the source domain training model is obtained, and then the network parameters are adjusted based on a relatively small amount of target data. The loss of the specific target domain labeled data is T for: Where M is the amount of training data in the target domain, represents the jth sample The model's predicted output, is the true value; S3.

2. Use the output of the fully connected layer, i.e., the feature map, as the new input variable F x , based on the conditional embedding operator difference CEOD module, to measure the target domain labeled data and target domain unlabeled data Differences in conditional distributions at different layers in Represent the feature maps of the target domain training data and test data, respectively. Represents the model's predicted value for the test data; as a semi-supervised paradigm, it is used to capture the available target domain data characteristics. The specific total loss function Loss total As shown below: in is the model’s prediction of the target domain training data, is the model's predicted value for the target domain test data, β is the coefficient, This is an empirical evaluation operation.

5. The multimodal process soft sensor modeling based on domain migration spatiotemporal graph network according to claim 4 is characterized by: The process of (4) is as follows: The root mean square error RMSE, mean absolute percentage error MAE and MAPE indicators are used to evaluate the performance of the model. The specific calculation formula is as follows: where y k , They represent the kth true value and predicted value respectively, and the test sample size of the target domain is K.

Images