Multi-view feature fusion multi-mode process virtual sample generation method

Abstract

The invention is applied to the technical field of industrial process control, and particularly discloses a multi-view image feature fusion multi-mode process virtual sample generation method, which comprises the following steps of: S1, collecting sensor data in an industrial process through an offline detection and distributed control method, and establishing an industrial process database; s2, performing normalization processing on the collected sensor data based on a Z-Score method to obtain a normalized data set; according to the multi-mode process virtual sample generation method based on multi-view image feature fusion, mode discrimination is realized through a Gaussian mixture model, and a measurement learning method of mode preserving embedding is combined, so that the problem of data missing caused by insufficient initial data of a new process, sensor faults or working condition switching in an industrial scene is solved; feature distribution of different process modes can be accurately reflected, and after the feature distribution is combined with an original sample for training, a soft measurement model can capture more comprehensive industrial process dynamic characteristics.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial process control, and in particular to a method for generating multi-mode process virtual samples by fusing multi-view graph features. Background Art

[0002] With the rapid advancement of Industry 4.0 and smart manufacturing, the process industry is accelerating its transformation towards full-process digitalization, networking, and intelligence. Against this backdrop, the demand for real-time data in industrial automation systems is growing exponentially. However, the real-time measurement of key quality variables such as component concentration and reaction efficiency often faces significant bottlenecks: the high deployment cost of high-precision sensors and the long offline measurement latency make it difficult to meet the needs of real-time process monitoring. This has become a key obstacle to achieving the self-perception, self-decision-making, and self-adaptation capabilities of production systems. Soft sensing technology provides a solution to this problem. This technology uses auxiliary variables that can be easily collected online as input and target quality variables as output. By constructing predictive models, it achieves real-time estimation of difficult-to-measure quality variables. Its core modeling methods are divided into first-principles models (FPMs) and data-driven models. FPMs rely on precise mechanisms and are suitable for simple systems; data-driven models rely on artificial intelligence and industrial big data and are more adaptable to high-dimensional nonlinear scenarios.

[0003] As a key enabling technology for the intelligent transformation of process industries, data-driven soft sensing methods have demonstrated tremendous potential. However, due to the high complexity of production systems and the rapid changes in external demands, data collection limitations are common in real-world industrial scenarios. This leads to a serious small sample size problem, significantly reducing modeling accuracy. For example, insufficient data in the early stages of a new process makes it difficult for the model to capture dynamic characteristics; redundant data from conventional operating conditions reduces the effectiveness of feature extraction; sensor failures and operating mode changes lead to missing data and incomplete sample coverage; and the combined effects of high dimensionality, nonlinearity, and noise exacerbate the difficulty of modeling with limited data. Overcoming the small sample size bottleneck is a key challenge in improving soft sensing accuracy. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-mode process virtual sample generation method based on multi-view graph feature fusion to solve the problem raised in the above background technology that due to insufficient accumulated data, the constructed soft measurement model is difficult to accurately capture the dynamic characteristics of the production system, thereby affecting the real-time estimation of the target quality variables.

[0005] To achieve the above object, the present invention provides the following technical solution: a multi-mode process virtual sample generation method based on multi-view image feature fusion, comprising the following steps: S1. Collect sensor data from industrial processes through offline detection and distributed control methods to establish an industrial process database; determine auxiliary variables through mechanism analysis of industrial processes and predictor variables , auxiliary variables are input variables, and predictor variables is the output variable, and the data set in the industrial process database includes auxiliary variables and predictor variables ; S2. Normalize the collected sensor data based on the Z-Score method to obtain a normalized data set ,in Indicates the number of samples; Indicates that auxiliary variables are included and predictor variables The total number of variables of the data set Divide into training set and test set ,in ; S3. Train the Gaussian mixture model (GMM) on the training set Perform pattern discrimination, obtain the probability of each sample belonging to each pattern, take the pattern with the highest probability as the predicted pattern of the sample, and integrate the pattern information into the original data set, and finally generate a data set with pattern annotations ; S4. By adopting a metric learning method based on pattern-preserving embedding, we implement sample balancing between patterns to construct a balanced training dataset. ; S5. Based on the balanced sample set Train a generative network that combines process pattern information and graph network, and use the trained model to generate enhanced samples ; S6. Enhance the sample With the original sample Merge and retrain the soft measurement model to improve the model prediction accuracy; Preferably, the original data pattern prediction process in S3 includes the construction of a GMM model and pattern prediction; The construction process of the GMM model is as follows: First, given labeled training data ; in , for dimensional input variables; , is a single variable output; Assume that the observation data is The probability density function with strong expressive power is constructed by weighted summation and can be described as follows:

[0006] in, Represents the observed data, i.e., the eigenvector, whose dimension is in any direction; is the number of mixed ingredients; It is Gaussian distribution of the mixture weight, satisfying and , Representative The probability density function of a Gaussian distribution is defined as follows:

[0007] in, Represents the dimensions of the data; It is The mean vector of the Gaussian components; No. The covariance matrix of the Gaussian components, Determines the shape and scale of the distribution; represents the determinant of the covariance matrix, Impact normalization factor; The GMM model parameters Estimated by the expectation maximization (EM) algorithm, the specific optimization steps can be described as: First, the values ​​of each parameter are initially randomly selected; Step E: Calculate each data point Belong to The posterior probability of a Gaussian component is calculated as follows:

[0008] Step M: Update parameters. The calculation formula is as follows:

[0009]

[0010]

[0011] in, Indicates that it belongs to The expected number of samples for each component; Then, repeat the E step and M step until the GMM model converges; The model prediction process is as follows: According to the trained GMM, the probability distribution of each sample pattern can be described as:

[0012] in Represents the trained GMM model; is the pattern label vector, Expressed as ,in Indicates the The samples belong to The probability of each pattern, and the pattern with the highest probability is taken as the data pattern , after adding the pattern information to the original dataset, a new dataset is obtained .

[0013] By adopting the above technical solution, through realizing pattern discrimination of GMM and combining it with the metric learning method of pattern-preserving embedding, it is possible to solve the problem of data loss caused by insufficient initial data of new processes, sensor failure or working condition switching in industrial scenarios.

[0014] Preferably, the metric learning method based on pattern embedding preservation in S4 adopts an autoencoder AE, wherein the AE includes an encoder and a decoder, and the encoder is composed of a two-layer feedforward network, and the expression formula is as follows:

[0015] in, For the original sample, is the encoder weight, is the encoder bias, For the optional activation function, select the sigmoid function. is the hidden feature of the output;

[0016] The decoder uses the same architecture as the encoder, expressed as follows:

[0017] in, is a hidden feature, is the decoder weight, is the encoder bias, is the reconstructed sample of the output, In order to characterize the pattern information of the data, a metric learning method of pattern-preserving mapping is added to the decoder loss function to shorten the distance between representations of the same pattern and increase the distance between representations of different patterns. The loss function is expressed as follows:

[0018] in, represents the reconstruction loss; represents the triplet metric learning loss.

[0019] By adopting the above technical solution, the soft sensing model can capture the dynamic characteristics of industrial processes more comprehensively through a balanced training data set, especially improve the modeling ability of rare working conditions and reduce the prediction bias under small sample conditions.

[0020] Preferably, the reconstruction loss and triplet loss in the overall loss of the metric learning method based on pattern-preserving embedding are described as follows:

[0021] in, is the number of samples, It is input samples, is the reconstructed sample; The metric learning objective based on pattern-preserving embedding constructs a discriminative latent space structure, which aggregates similar samples and keeps heterogeneous samples away, and designs a triplet minimization loss function. , the loss function The expression formula is as follows:

[0022]

[0023]

[0024]

[0025] in, The representation model represents the embedded vector representation of the sample; Represents the anchor sample (Anchor), From and anchor point A randomly selected positive sample from the same pattern, Indicates the distance from the anchor point Negative samples randomly selected from different modes; express and The Euclidean distance between , , are the probability vectors of anchor points, positive samples, and negative samples belonging to each mode; is the probability that the anchor point and the positive sample belong to the same pattern; is the probability that the anchor point and the negative sample belong to the same mode.

[0026] By adopting the above technical solution, the reconstruction loss can be used to ensure that the generated samples retain the physical characteristics of the original data and avoid information loss.

[0027] Preferably, the generative network in S5 adopts a generative adversarial network (GAN) architecture, and the generative adversarial network (GAN) includes a generator and a discriminator. The generator uses the probability value of the pattern to which the sample belongs as an input conditional variable to guide the generation of pattern-related data. The framework of the generative model introduces a regressor under the pattern condition and a sample classifier with the pattern as the output target through a regularization term to enhance the generative model's perception of pattern information, and adopts a collaborative training method to train the regression model and the classification model; the discriminator introduces a multi-view graph convolution feature enhancement mechanism, constructs a feature relationship topology map through a feature correlation matrix, and realizes explicit modeling of the constraint relationship between feature nodes.

[0028] By adopting the above technical solution, the generator's ability to generate data for rare working conditions can be enhanced through the collaborative training mechanism by complementing regression and classification tasks.

[0029] Preferably, the generator is described as follows: The pattern probability predicted by the GMM is used as the conditional variable of the generator input to guide the generation of pattern-related data. A regressor under the pattern condition and a sample classifier with the pattern as the output target are introduced to enhance the generative model's ability to perceive pattern information. The mapping function of the regressor is described as follows:

[0030] in, Represents the mapping relationship of the regression model; The mapping function of the pattern classifier is described as follows:

[0031] in Represents the mapping relationship of the classifier; During the generator training process, the mapping relationship between process variables and quality variables under different modes and the modeling performance of the classification relationship are back-propagated to the generator to guide the generator learning. The objective function of the generator is described as follows:

[0032] in, and Represents two weight coefficients used to weigh the two loss components; is the number of virtual samples generated during the network update process; The generator generates Quality variables of samples; is the first sample quality variables; It is generated The data belongs to The probability of a pattern, The classifier predicts The probability of a data pattern.

[0033] By adopting the above technical solution, the accuracy of pattern attribution of generated data can be improved through pattern probability condition input and classifier constraints.

[0034] Preferably, the collaborative training method for training the regression model and the classification model is as follows: Assume that the original generated data is represented by , is the generated data quality variable corrected by the regressor, is the probability of the generated data pattern corrected by the classifier, is the data adjusted based on the regression When updating the classifier, the predicted mode attribution probability is realized by using the regression model and classification model collaborative training framework to achieve knowledge transfer. The interactive optimization process is as follows: The regressor optimizes the mapping relationship between quality variables and outputs the adjusted data representation ; The classifier corrects the pattern probability distribution and outputs the corrected data representation ; Through the cross-training mechanism, the adjusted data is utilized Train the classifier to make the prediction probability Approximating the true probability membership , similarly to training the regressor, The optimization objective functions of the regression model and the classification model are described as follows:

[0035]

[0036] in, The quality variable predicted by the regression model during the co-training process; The first prediction of the classification model in the collaborative training process The samples belong to The probability of a pattern; is the number of virtual samples generated in a single round of iteration.

[0037] By adopting the above technical solution, the quality variables and pattern probabilities of the generated data can be mutually constrained through collaborative training, avoiding the deviation caused by single task optimization.

[0038] Preferably, the multi-view graph convolution feature enhancement mechanism first establishes a dual-channel feature correlation analysis mechanism: The linear correlation characteristics are characterized by the Pearson correlation coefficient, and the resulting linear correlation matrix is ​​used as the edge connection strength of the node to ensure that the graph structure can represent the linear dependency relationship between features; Spearman rank correlation analysis is used to capture the nonlinear correlation characteristics between features, and the obtained nonlinear correlation matrix is ​​used as the edge connection strength of the node to ensure that the graph structure can represent the nonlinear dependency structure between features; On this basis, two learnable weight parameters are introduced , the hidden layer features under the two correlation metrics are adaptively spliced ​​with the original data features to form a multi-view graph feature extraction structure, and the fused features are used as new data features to identify authenticity. The objective function of the discriminator is:

[0039] in, is the gradient penalty term, For real samples, To generate samples.

[0040] By adopting the above technical solution, the problem of difficult modeling of complex correlations between features in industrial data can be solved by using Pearson and Spearman correlation coefficients to characterize linear and nonlinear dependencies respectively.

[0041] Preferably, the Pearson correlation coefficient and the Spearman correlation coefficient are obtained as follows: For the Pearson correlation coefficient: Assume that two paired labeled continuous variable sample sets are represented as ,in is the number of samples, the covariance between the two variables It can be calculated as:

[0042] in, and Respectively of samples peacekeeping dimensional eigenvalues; and It is peacekeeping The mean of the dimension feature; Pearson correlation coefficients between the variables It can be calculated as:

[0043] in and is the standard deviation of two continuous variables; Regarding the Spearman correlation coefficient: The Spearman correlation coefficient is used to capture the nonlinear correlation between two continuous variables, and its calculation formula is:

[0044]

[0045] in, Indicates the number of samples; and express and Rank after sorting from small to large; Indicates the level difference between them.

[0046] By adopting the above technical solution, the linear dependency between features is explicitly modeled through the Pearson coefficient, and the nonlinear relationship is captured by the Spearman coefficient, which can solve the problem of mixed dependency relationships in industrial data being difficult to model.

[0047] Preferably, the gradient penalty is as follows:

[0048] in It belongs to Real samples of the model and generate data Linear interpolation between .

[0049] By adopting the above technical solution, the gradient penalty constraint can make the transition between the generated samples and the real samples in the feature space more natural.

[0050] Compared with the prior art, the present invention has the following advantages: the multi-mode process virtual sample generation method of multi-view image feature fusion: 1. This invention uses a Gaussian mixture model to achieve pattern discrimination and combines it with a metric learning method that preserves pattern embedding. This addresses the data loss problem in industrial scenarios caused by insufficient initial data for new processes, sensor failures, or operating mode switching. It accurately reflects the characteristic distribution of different process patterns. After combining this model with the original samples for training, the soft sensing model can capture a more comprehensive set of dynamic characteristics of industrial processes, improve the modeling capabilities for rare operating conditions, and significantly improve the prediction bias caused by incomplete data coverage under small sample conditions. 2. The present invention introduces pattern-conditioned regressors and classifiers into the generative adversarial network framework, combines it with the multi-view graph convolution feature enhancement mechanism, characterizes feature linear dependence through the Pearson correlation coefficient, captures nonlinear associations through Spearman rank correlation analysis, and constructs a feature relationship topology graph to achieve explicit modeling of complex constraint relationships between industrial process variables, ensuring that the generated virtual samples are highly close to the real industrial data distribution. After collaborative training with the original data, the soft measurement model's feature extraction capability for high-dimensional, nonlinear and noisy industrial data can be enhanced, so that the model's prediction accuracy and robustness under complex working conditions are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Schematic diagram of the process structure of the method of the present invention; Figure 2 This is a schematic diagram of the structural framework of the present invention; Figure 3 This is a schematic diagram of the CTC process flow structure of the present invention; Figure 4 This is a schematic diagram of the t-SNE dimensionality reduction distribution structure of the original data of the present invention; Figure 5 This is a schematic diagram of the t-SNE dimensionality reduction distribution structure of the fused data after adding balanced samples in the present invention; Figure 6 Schematic diagram of the curve structure of matrix concentration prediction and true value in the CTC process after data enhancement by the method of the present invention and other methods. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] See also Figures 1-6 , the present invention provides a technical solution: a multi-mode process virtual sample generation method based on multi-view image feature fusion.

[0054] The first step is to collect sensor data from industrial processes through offline detection and distributed control methods to establish an industrial process database; on this basis, we deeply analyze the mechanism of the industrial process to determine the auxiliary variables and predictor variables , auxiliary variables are input variables, and predictor variables As the output variable, the data stored in the database constitutes a labeled data set , the dataset Include auxiliary variables and predictor variables ; The second step is to Divided into training set and test set, where is the number of samples.

[0055] The second step is to normalize the collected sensor data based on the Z-Score method to obtain a normalized data set. ,in Indicates the number of samples; Indicates that auxiliary variables are included and predictor variables The total number of variables in the data set Divide into training set and test set ,in .

[0056] The third step is to train the Gaussian mixture model (GMM) on the training set Perform pattern discrimination, obtain the probability of each sample belonging to each pattern, take the pattern with the highest probability as the predicted pattern of the sample, and integrate the pattern information into the original data set, and finally generate a data set with pattern annotations ;

[0057] Establish a Gaussian mixture model GMM. The establishment process is as follows:

[0058] The construction process of the GMM model is as follows: The core idea of ​​the Gaussian mixture model (GMM) used is to maximize the joint probability distribution of the observed data under the model. Specifically, the labeled training data is given by the second step ; in , for dimensional input variables; , is a single variable output; Assume that the observation data is The probability density function with strong expressive power is constructed by weighted summation and can be described as follows:

[0059] in, Represents the observed data, i.e., the eigenvector, whose dimension is in any direction; is the number of mixed ingredients; It is Gaussian distribution of the mixture weight, satisfying and , Representative The probability density function of a Gaussian distribution is defined as follows:

[0060] in, Represents the dimensions of the data; It is The mean vector of the Gaussian components; No. The covariance matrix of the Gaussian components, Determines the shape and scale of the distribution; represents the determinant of the covariance matrix, Impact normalization factor; GMM model parameters Estimation is done through the expectation maximization (EM) algorithm. The specific iterative process includes: First, the values ​​of each parameter are initially randomly selected; Step E: Calculate each data point Belong to The posterior probability of a Gaussian component is calculated as follows:

[0061] Step M: Update parameters. The calculation formula is as follows:

[0062]

[0063]

[0064] in, Indicates that it belongs to The expected number of samples for each component; Then, repeat the E step and M step until the GMM model converges; The model prediction process is as follows: After the above iterations until convergence, a trained GMM model can be obtained, and the probability of each sample pattern is assigned as follows:

[0065] in Represents the trained GMM model; is the pattern label vector, Expressed as ,in Indicates the The samples belong to The probability of each pattern, and the pattern with the highest probability is taken as the data pattern , after adding the pattern information to the original dataset, a new dataset is obtained .

[0066] The fourth step is to implement sample balancing between modes by adopting a metric learning method based on pattern-preserving embedding to construct a balanced training dataset. ;

[0067] Specifically, the metric learning method based on pattern embedding preservation adopts an autoencoder AE, which includes an encoder and a decoder. The encoder consists of a two-layer feedforward network, and the expression formula is as follows:

[0068] in, For the original sample, is the encoder weight, is the encoder bias, For the optional activation function, select the sigmoid function. is the hidden feature of the output; The decoder uses the same architecture as the encoder, expressed as follows:

[0069] in, is a hidden feature, is the decoder weight, is the encoder bias, is the reconstructed sample of the output,

[0070] In order to characterize the pattern information of the data, a metric learning method of pattern-preserving mapping is added to the decoder loss function to shorten the distance between representations of the same pattern and increase the distance between representations of different patterns. The loss function is expressed as follows:

[0071] in, represents the reconstruction loss; The reconstruction loss and triplet loss in the overall loss of the pattern-preserving embedding-based metric learning method representing the triplet metric learning loss are described as follows:

[0072] in, is the number of samples, It is input samples, is the reconstructed sample; In the metric learning process, to improve the separability of sample features in the embedding space, relative distance optimization is usually performed by constructing positive and negative sample pairs or triplet sample pairs. However, in actual training, mode collapse is prone to occur, that is, samples of multiple different categories or modalities overlap excessively in the embedding space, resulting in loss of representation ability. Therefore, this paper proposes a mode-aware smoothing mechanism. This mechanism recognizes the similarity between samples and adjusts the geometric relationship between features, thereby enhancing the diversity and discriminability between embedded features. The loss function is as follows:

[0073]

[0074]

[0075]

[0076]

[0077] in, The representation model represents the embedded vector representation of the sample; Represents the anchor sample (Anchor), From and anchor point A randomly selected positive sample from the same pattern, Indicates the distance from the anchor point Negative samples randomly selected from different modes; express and The Euclidean distance between

[0078] , , are the probability vectors of anchor points, positive samples, and negative samples belonging to each mode; is the probability that the anchor point and the positive sample belong to the same pattern; is the probability that the anchor point and the negative sample belong to the same mode, use Train the above-mentioned pattern-aware metric learning model. After the training converges, use the model to encode the training data to obtain hidden features. , randomly select two features belonging to the same mode, interpolate, and decode to obtain new samples in the corresponding mode as mode-balanced samples, and get .

[0079] Step 5: Based on the balanced sample set Train a generative network that combines process pattern information and graph network, and use the trained model to generate enhanced samples ; Build and train the generative model. The specific steps are as follows: (1) Based on the construction of the discriminator for graph network feature extraction, a dual-channel feature correlation analysis mechanism is first established: the linear correlation characteristics are characterized by the Pearson correlation coefficient, and the obtained linear correlation matrix is ​​used as the edge connection strength of the node to ensure that the graph structure can represent the linear dependency relationship between features; the nonlinear correlation characteristics between features are captured by Spearman rank correlation analysis, and the obtained nonlinear correlation matrix is ​​used as the edge connection strength of the node to ensure that the graph structure can represent the nonlinear dependency structure between features; on this basis, two learnable weight parameters are introduced , the hidden layer features under the two correlation metrics are adaptively spliced ​​with the original data features to form a multi-view graph feature extraction structure. The two correlation coefficients are described as follows: For the Pearson correlation coefficient: Assume that two sets of continuous variable samples with paired labels are represented as ,in is the number of samples, the covariance between the two variables It can be calculated as:

[0080] in, and Respectively of samples peacekeeping dimensional eigenvalues; and It is peacekeeping The mean of the dimension feature; Pearson correlation coefficient between variables It can be calculated as:

[0081] in and is the standard deviation of two continuous variables; For Spearman correlation coefficient: The Spearman correlation coefficient is used to capture the nonlinear correlation between two continuous variables. Its calculation formula is:

[0082]

[0083] in, Indicates the number of samples; and express and Rank after sorting from small to large;

[0084] Indicates the level difference between them; The fused features are used as new data features to identify authenticity. The objective function of the discriminator is:

[0085] in, is the gradient penalty term, For real samples, To generate samples, The gradient penalty is as follows:

[0086] in It belongs to Real samples of the model and generate data Linear interpolation between .

[0087] (2) Construction of pattern-related generator The generative network adopts the generative adversarial network (GAN) architecture. The generative adversarial network (GAN) includes a generator and a discriminator. The generator uses the probability value of the pattern to which the sample belongs as the input conditional variable to guide the generation of pattern-related data. The framework of the generative model introduces a regressor under the pattern condition and a sample classifier with the pattern as the output target through the regularization term to enhance the generative model's ability to perceive pattern information. The mode probability predicted by GMM is used as the conditional variable of the generator input to guide the generation of mode-related data. A regressor under the mode condition and a sample classifier with the mode as the output target are introduced to enhance the generative model's ability to perceive mode information. The relationship between process variables and quality variables under the mode information condition is as follows:

[0088] in, Represents the mapping relationship of the regression model; The mapping function of the pattern classifier is described as follows:

[0089] in Represents the mapping relationship of the classifier; During the generator training process, the mapping relationship between process variables and quality variables under different modes and the modeling performance of the classification relationship are back-propagated to the generator to guide the generator learning. The objective function of the generator is described as follows:

[0090] in, and Represents two weight coefficients used to weigh the two loss components; is the number of virtual samples generated during the network update process; The generator generates Quality variables of samples; is the first sample quality variables; It is generated The data belongs to The probability of a pattern, The classifier predicts The probability of a data pattern.

[0091] (3) Construction of collaborative training

[0092] The regression model and the classification model are trained using a collaborative training method. The discriminator introduces a multi-view graph convolution feature enhancement mechanism, constructs a feature relationship topology through the feature correlation matrix, and realizes the explicit modeling of the constraint relationship between feature nodes. After the generation model completes one round of iteration, the generated data is input into the two models to improve their generalization ability. The collaborative training process of the regression model and the classification model is as follows: Assume that the original generated data is represented as , is the generated data quality variable corrected by the regressor, is the probability of the generated data pattern corrected by the classifier, is the data adjusted based on the regression When updating the classifier, the predicted mode attribution probability is realized by using the regression model and classification model collaborative training framework to achieve knowledge transfer. The interactive optimization process is as follows: Regressor optimizes the mapping relationship between quality variables and outputs the adjusted data representation ; The classifier corrects the pattern probability distribution and outputs the corrected data representation ; Through the cross-training mechanism, the adjusted data is utilized Train the classifier to make the prediction probability Approximating the true probability membership , similarly to training the regressor, The optimization objective functions of the regression model and the classification model are described as follows:

[0093]

[0094] in, The quality variable predicted by the regression model during the co-training process; The first prediction of the classification model in the collaborative training process The samples belong to The probability of a pattern; is the number of virtual samples generated in a single round of iteration; It ensures that the regression and classification models can extract knowledge of reliable supervisory information during the iteration process and pass it to the generator, thereby helping the generator learn regression knowledge and classification information from the input data, improving the quality of generated data and the generalization ability of the prediction model.

[0095] Step 6: Use the data obtained in step 4 The generated network is trained iteratively until it reaches a Nash equilibrium.

[0096] The seventh step is to use the trained generative model to generate several virtual sample data, which have similar statistical characteristics and process correlations with the original data. The generated virtual samples are fused with the original training data to construct an enhanced training data set. Based on the enhanced training data set, the soft measurement model is retrained to finally establish the quality variable About process variables The mapping relationship is expressed as:

[0097] in, is the soft measurement model function obtained by training the enhanced dataset mentioned above.

[0098] The implementation case of the present invention uses the root mean square error RMSE and the determination coefficient Verify the prediction effect, the smaller the RMSE value is The larger the value of , the smaller the prediction error of the proposed method, that is, the better the prediction performance of the soft sensor modeling method. Its calculation formula is as follows:

[0099]

[0100] in, represents the number of samples, represents the predicted output, The actual output value of the table, and Represents the mean of the sample output values. The lower the RMSE value, A higher value indicates a higher accuracy of the model prediction.

[0101] The performance of the method proposed in the present invention is illustrated below with the help of a specific industrial example of a chlortetracycline fermentation process. In the chlortetracycline fermentation process, the chlortetracycline substrate concentration is a very important key indicator in the feedback fermentation control process. However, the chlortetracycline substrate concentration cannot be detected online at present. In order to improve the chlortetracycline fermentation control level, it is necessary to perform soft measurement modeling of the chlortetracycline substrate concentration. Table 1 shows the nine auxiliary variables selected for the key predictor variable chlortetracycline substrate concentration.

[0102] Table 1 Description of auxiliary variables

[0103] This experiment collected 13 batches of data from a fermentation tank. The first 8 batches were used as training sets, including 200 data points; the remaining 5 batches were used as test sets, including 127 data points. The network structure and initialization parameters of different generative models were determined through repeated experiments: In order to verify the effectiveness of the method proposed in this paper, four other data generation methods are compared: (1) GPR: Gaussian process regression, which uses a mother covariance function with a noise term; (2) SVAE: Sample generation based on supervised variational autoencoder; (3) WGAN_gp: WGAN with gradient penalty for sample generation; (4) VA_WGAN: A generative model combining VAE and WGAN-gp; (5) MR_GAN: A related generative adversarial network based on regression modeling (MR-GAN) for soft sensor modeling based on data augmentation; (6) MGGAN_gp: the method proposed in this invention.

[0104] According to the experimental results, in this case, when the data expansion amount is 160, the prediction performance reaches the best. The experimental results are as follows: Table 2 Influence of different data enhancement methods on the soft sensor modeling accuracy in CTC chemical process

[0105] Table 2 compares the performance improvements of the soft sensor prediction model using five sample generation methods when generating 160 samples. Analysis of Table 1 shows that these existing models can improve model prediction performance to a certain extent. The samples generated by SVAE, VA_WGAN, WGAN_gp, and MR_GAN improve prediction accuracy by 13.56%, 19.55%, 15.73%, and 19.02%, respectively. The proposed MGGAN_gp reduces the root mean square error from 0.4374 to 0.3288, an improvement of 24.83%. This demonstrates the effectiveness of the proposed MGGAN_gp sample generation method in the chlortetracycline CTC chemical process. This case study demonstrates that the present invention has excellent sample generation capabilities.

[0106] Working principle: Industrial sensor data is collected through offline detection and distributed control, a database is established, the data is normalized based on the Z-Score method, and the training set and test set are divided. The training set is subjected to pattern discrimination through the Gaussian mixture model (GMM), and finally a data set with pattern annotations is generated. By adopting a metric learning method based on pattern-preserving embedding, sample balancing between patterns is implemented to construct a balanced training data set. Based on the balanced sample set, a generative network that integrates process pattern information and graph network is trained, and the trained model is used to generate enhanced samples. The enhanced samples are merged with the original samples to retrain the soft measurement model to improve the model prediction accuracy.

[0107] While the embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that various changes, modifications, substitutions, and alterations can be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. A multi-modal process virtual sample generation method based on multi-view feature fusion, characterized by: The following steps are involved: S1. Collect sensor data from industrial processes through offline detection and distributed control methods to establish an industrial process database; determine auxiliary variables through mechanism analysis of industrial processes and predictor variables , auxiliary variables are input variables, and predictor variables is the output variable, and the data set in the industrial process database includes auxiliary variables and predictor variables ; S2. Normalize the collected sensor data based on the Z-Score method to obtain a normalized data set ,in Indicates the number of samples; Indicates that auxiliary variables are included and predictor variables The total number of variables of the data set Divide into training set and test set ,in ; S3. Train the Gaussian mixture model (GMM) on the training set Perform pattern discrimination, obtain the probability of each sample belonging to each pattern, take the pattern with the highest probability as the predicted pattern of the sample, and integrate the pattern information into the original data set, and finally generate a data set with pattern annotations ; S4. By adopting a metric learning method based on pattern-preserving embedding, we implement sample balancing between patterns to construct a balanced training dataset. ; S5. Based on the balanced sample set Train a generative network that combines process pattern information and graph network, and use the trained model to generate enhanced samples ; S6. Enhance the sample With the original sample Merge and retrain the soft measurement model to improve the model prediction accuracy.

2. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 1 is characterized by: The original data pattern prediction process in S3 includes the construction of the GMM model and pattern prediction; The construction process of the GMM model is as follows: First, given labeled training data ; in , for dimensional input variables; , is a single variable output; Assume that the observation data is The probability density function with strong expressive power is constructed by weighted summation and can be described as follows: ； in, Represents the observed data, i.e., the eigenvector, whose dimension is in any direction; is the number of mixed ingredients; It is Gaussian distribution of the mixture weight, satisfying and , Representative The probability density function of a Gaussian distribution is defined as follows: ； in, Represents the dimensions of the data; It is The mean vector of the Gaussian components; No. The covariance matrix of the Gaussian components, Determines the shape and scale of the distribution; represents the determinant of the covariance matrix, Impact normalization factor; The GMM model parameters Estimated by the expectation maximization (EM) algorithm, the specific optimization steps can be described as: First, the values ​​of each parameter are initially randomly selected; Step E: Calculate each data point Belong to The posterior probability of a Gaussian component is calculated as follows: ； Step M: Update parameters. The calculation formula is as follows: ； ； ； in, Indicates that it belongs to The expected number of samples for each component; Then, repeat the E step and M step until the GMM model converges; The model prediction process is as follows: According to the trained GMM, the probability distribution of each sample pattern can be described as: ； in Represents the trained GMM model; is the pattern label vector, Expressed as ,in Indicates the The samples belong to The probability of each pattern, and the pattern with the highest probability is taken as the data pattern , after adding the pattern information to the original dataset, a new dataset is obtained .

3. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 1 is characterized by: The metric learning method based on pattern embedding preservation in S4 adopts an autoencoder AE, which includes an encoder and a decoder. The encoder is composed of a two-layer feedforward network, and the expression formula is as follows: ； in, For the original sample, is the encoder weight, is the encoder bias, For the optional activation function, select the sigmoid function. is the hidden feature of the output; The decoder uses the same architecture as the encoder, expressed as follows: ； in, is a hidden feature, is the decoder weight, is the encoder bias, is the reconstructed sample of the output, In order to characterize the pattern information of the data, a metric learning method of pattern-preserving mapping is added to the decoder loss function to shorten the distance between representations of the same pattern and increase the distance between representations of different patterns. The loss function is expressed as follows: ； in, represents the reconstruction loss; represents the triplet metric learning loss.

4. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 3 is characterized by: The reconstruction loss and triplet loss in the overall loss of the metric learning method based on pattern-preserving embedding are described as follows: ； in, is the number of samples, It is input samples, is the reconstructed sample; The metric learning objective based on pattern-preserving embedding constructs a discriminative latent space structure, which aggregates similar samples and keeps heterogeneous samples away, and designs a triplet minimization loss function. , the loss function The expression formula is as follows: ； ； ； ； in, The representation model represents the embedded vector representation of the sample; Represents the anchor sample (Anchor), From and anchor point Randomly selected positive samples from the same model, Indicates the distance from the anchor point Negative examples randomly selected from different modes; express and The Euclidean distance between , , are the probability vectors of anchor points, positive samples, and negative samples belonging to each mode; is the probability that the anchor point and the positive sample belong to the same pattern; is the probability that the anchor point and the negative sample belong to the same mode.

5. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 1 is characterized by: The generative network in S5 adopts a generative adversarial network (GAN) architecture, which includes a generator and a discriminator. The generator uses the probability value of the pattern to which the sample belongs as an input conditional variable to guide the generation of pattern-related data. The framework of the generative model introduces a regressor under the pattern condition and a sample classifier with the pattern as the output target through a regularization term to enhance the generative model's perception of pattern information, and adopts a collaborative training method to train the regression model and the classification model; the discriminator introduces a multi-view graph convolution feature enhancement mechanism, constructs a feature relationship topology map through a feature correlation matrix, and realizes explicit modeling of the constraint relationship between feature nodes.

6. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 5 is characterized by: The generator is described as follows: The pattern probability predicted by the GMM is used as the conditional variable of the generator input to guide the generation of pattern-related data. A regressor under the pattern condition and a sample classifier with the pattern as the output target are introduced to enhance the generative model's ability to perceive pattern information. The mapping function of the regressor is described as follows: ； in, Represents the mapping relationship of the regression model; The mapping function of the pattern classifier is described as follows: ； in Represents the mapping relationship of the classifier; During the generator training process, the mapping relationship between process variables and quality variables under different modes and the modeling performance of the classification relationship are back-propagated to the generator to guide the generator learning. The objective function of the generator is described as follows: ； in, and Represents two weight coefficients used to weigh the two loss components; is the number of virtual samples generated during the network update process; The generator generates Quality variables of samples; is the first sample quality variables; It is the generated The data belongs to The probability of a pattern, The classifier predicts The probability of a data pattern.

7. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 5 is characterized by: The collaborative training process of the regression model and the classification model is as follows: Assume that the original generated data is represented as , is the generated data quality variable corrected by the regressor, is the probability of the generated data pattern corrected by the classifier, is the data adjusted based on the regression When updating the classifier, the predicted mode attribution probability is realized by using the regression model and classification model collaborative training framework to achieve knowledge transfer. The interactive optimization process is as follows: The regressor optimizes the mapping relationship between quality variables and outputs the adjusted data representation ; The classifier corrects the pattern probability distribution and outputs the corrected data representation ; Through the cross-training mechanism, the adjusted data is utilized Train the classifier to make the prediction probability Approximating the true probability membership , similarly to training the regressor, The optimization objective functions of the regression model and the classification model are described as follows: ； ； in, The quality variable predicted by the regression model during the co-training process; The first prediction of the classification model in the collaborative training process The samples belong to The probability of a pattern; is the number of virtual samples generated in a single round of iteration.

8. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 5 is characterized by: The multi-view graph convolution feature enhancement mechanism first establishes a dual-channel feature correlation analysis mechanism: The linear correlation characteristics are characterized by the Pearson correlation coefficient, and the resulting linear correlation matrix is ​​used as the edge connection strength of the node to ensure that the graph structure can represent the linear dependency relationship between features; Spearman rank correlation analysis is used to capture the nonlinear correlation characteristics between features, and the obtained nonlinear correlation matrix is ​​used as the edge connection strength of the node to ensure that the graph structure can represent the nonlinear dependency structure between features; On this basis, two learnable weight parameters are introduced , the hidden layer features under the two correlation metrics are adaptively spliced ​​with the original data features to form a multi-view graph feature extraction structure, and the fused features are used as new data features to identify authenticity. The objective function of the discriminator is: ； in, is the gradient penalty term, For real samples, To generate samples.

9. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 8, characterized in that: The Pearson correlation coefficient and Spearman correlation coefficient are obtained as follows: For the Pearson correlation coefficient: Assume that two paired labeled continuous variable sample sets are represented as ,in is the number of samples, the covariance between the two variables It can be calculated as: ; in, and Respectively of samples peacekeeping dimensional eigenvalues; and It is peacekeeping The mean of the dimension feature; Pearson correlation coefficients between the variables It can be calculated as: ； in and is the standard deviation of two continuous variables; Regarding the Spearman correlation coefficient: The Spearman correlation coefficient is used to capture the nonlinear correlation between two continuous variables, and its calculation formula is: ； ； in, Indicates the number of samples; and express and Rank after sorting from small to large; Indicates the level difference between them.

10. The multi-modal process virtual sample generation method based on multi-view feature fusion according to claim 8, characterized in that: The gradient penalty is as follows: ； in It belongs to Real samples of the model and generate data Linear interpolation between .

Images