A multi-process fault monitoring and diagnosis method

Abstract

The present invention relates to a multi-operating process fault monitoring and diagnosis method, comprising the following steps: using variational inference to learn a robust metric model in a kernel latent space to obtain low-dimensional nonlinear latent variables and model parameters in the kernel latent space corresponding to the data; and performing GMM multi-operating process monitoring based on robust metric learning using variational inference in the kernel latent space. The present invention embeds kernel space weakly supervised metric learning and latent variable models into the GMM modeling process and uses Student's t-distribution to model non-Gaussian noise. The method focuses on improving the subsequent GMM performance through metric learning methods, thereby enhancing the GMM's ability to describe nonlinear, uncertain, non-Gaussian noisy, and multi-operating process datasets.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial process monitoring and diagnosis, and specifically relates to a multi-operating process fault monitoring and diagnosis method based on kernel latent space robust metric learning GMM. Background Art

[0002] To achieve optimal control, system reliability, and process safety, real-time process monitoring technology has garnered widespread attention from both academia and industry. Multivariate statistical process monitoring (MSPM) is unconstrained by process mechanistic models and the lack of process knowledge. It can monitor processes using only process measurement data collected by a distributed control system. Over the past decade, latent variable models (LVMs) have become the most popular technique in quality prediction and process monitoring. Common LVM methods, such as principal component analysis (PCA), partial least squares (PLS), slow feature analysis (SFA), and canonical vector analysis (CVA), can transform high-dimensional, correlated process data into a low-dimensional latent feature space, eliminating the impact of redundant data and variables on modeling [References 1-6]. These low-dimensional latent variables represent a significant portion of process variation and can be used to develop monitoring and regression models, providing process anomaly warnings and quality predictions. However, due to factors such as equipment aging, raw material fluctuations, and shifts in market demand, the operating conditions of many industrial processes often fluctuate continuously. Processes with multiple operating conditions are referred to as multi-condition processes. Currently, research on multi-condition processes has been conducted, primarily focusing on the following approaches: 1) Unsupervised clustering methods: Common approaches include fuzzy mean clustering (FCM) and hierarchical clustering. These methods partition complex data sets into multiple clusters, each representing a specific operating condition. Process monitoring is achieved by identifying the cluster to which new process data belongs [References 1, 2, 7, 8]. 2) Multi-model modeling methods within a probabilistic framework: These methods use local MSPM methods to establish monitoring models for each operating condition. They then employ hybrid modeling, model switching, and ensemble learning to establish a global monitoring model for complex multi-condition processes. Common hybrid monitoring modeling methods include hybrid PCA and its Bayesian regularized hybrid PCA, maximum likelihood hybrid PCA, robust hybrid probabilistic PCA, Gaussian mixture models (GMMs), multi-subspace partitioning methods, and dynamic latent variable model switching [References 9-12]. These methods have clear physical implications, effectively accounting for both local and global process information and accounting for the uncertainty associated with varying operating conditions. These methods have achieved promising results in practice. 3) Local Learning Modeling Methods: Typical methods include just-in-time learning modeling and adaptive local modeling [References 13-17]. These methods are relatively simple and flexible, and can embed new modalities into monitoring models. 4) Variable Partitioning Methods: These methods divide the variable space into multiple strongly correlated variable subspaces, emphasizing the local information of process variables [Reference 18]. However, these methods do not consider the global information of the process, resulting in the loss of variable-related information between local subspaces.

[0003] For most industrial processes, process variables exhibit nonlinear relationships, inherent uncertainty, and strong nonlinear characteristics. GMM is a classic unsupervised soft clustering method that uses a finite number of Gaussian distributions to fit complex data distributions. It can handle nonlinear, uncertain, and non-Gaussian datasets, exhibits good physical properties, and is widely used in multi-process monitoring and soft sensing [References 2, 3, 8, 11]. However, in practical applications, high-dimensional process data often leads to singularities in the covariance matrix of the sub-Gaussian distribution of the GMM model. Redundant process variables not only increase the computational burden of GMM training but also make it more prone to local extremes, severely impacting GMM model performance. While learning GMM model parameters using variational inference and Monte Carlo methods can address the singularity of the Gaussian distribution covariance in GMM models to a certain extent, redundant variables and data nonlinearity remain significant factors that affect GMM performance. To address these issues, existing GMM modeling methods often first transform the high-dimensional data into a low-dimensional feature space using PCA or KPCA, and then construct the GMM model in this low-dimensional feature space [References 2, 3]. This type of method does not utilize the geometric structure information between samples and cannot extract effective latent variable features with the ability to identify working conditions. Although using KPCA for data preprocessing can effectively improve GMM performance, the kernel function parameter values of KPCA are still determined based on manual experience and do not consider the scale characteristics of different feature variables, thereby reducing the effectiveness of extracting latent variables. In addition, choosing the appropriate number of Gaussian components (GCs), that is, model selection, is also an important issue in GMM learning. Inappropriate GCs can lead to overfitting problems. Commonly used hybrid model selection methods are based on minimizing a certain criterion to achieve model selection. The main criterion methods include Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Absolute Incremental Log-Likelihood (AIL), Minimum Information Length (MML), and other criteria [References 1-3,8].

[0004] It is understood that each Gaussian distribution in the GMM model is obtained by calculating the Mahalanobis distance between a sample and its mean. Kernel-based latent vector extraction methods ultimately boil down to calculating the Mahalanobis distance between samples in a high-dimensional feature space. Metric learning methods leverage the sample information provided by the dataset to adaptively learn an appropriate distance metric, thereby mapping high-dimensional data samples into a low-dimensional discriminant latent variable space. This minimizes the distance between samples under the same operating condition and maximizes the distance between samples under different operating conditions. While existing metric learning methods have achieved promising results in many fields, they often focus on obtaining a hidden similarity metric between samples, paying less attention to improving the performance of subsequent learning tasks. This leads to a disconnect between metric learning and predictive model learning. Most metric learning methods fail to consider input data noise or simply assume that data noise is white [References 2, 3, 16, 19]. In reality, industrial process data is often affected by various factors, and data noise often exhibits a non-Gaussian distribution. Therefore, simply assuming white noise is often inappropriate. Furthermore, the dimensionality of the latent space is typically determined using cross-validation, which imposes a significant computational burden on large-scale and high-dimensional data. Summary of the Invention

[0005] The purpose of the present invention is to design a multi-process fault monitoring and diagnosis method based on kernel latent space robust metric learning GMM to address the shortcomings of the above-mentioned existing technologies.

[0006] In machine learning, clustering and classification algorithms such as the fuzzy c-means clustering (FCM) algorithm and support vector machine (SVM) often use distance metrics to describe and determine data similarity. Considering the complexity of the data sample dimension and distribution, choosing an appropriate distance metric is key to ensuring algorithm performance. Common metrics include Mahalanobis distance, linear mapping distance metric, nonlinear distance metric, and local distance metric. Most metric learning focuses on Mahalanobis distance. in, matrix Since the matrix M is non-singular and positive definite, it can be decomposed into M = WW T , r=rank(M). Therefore, the above formula can be written as d M (x i ,x j )=||W T (x i -x j )||2. Clearly, the Mahalanobis distance is equivalent to the Euclidean distance after linear mapping of the data samples. Metric learning adaptively discovers metrics that reflect the data distribution, keeping similar samples as close as possible and heterogeneous samples as far apart as possible, thus better representing the similarity between samples.

[0007] Metric learning can be categorized into supervised, unsupervised, weakly supervised, and semi-supervised metric learning [References 20-22]. Supervised metric learning constructs similarity or dissimilarity constraints based on sample label information to build an optimization model. Common algorithms include large-margin nearest neighbor (LMNN), information-theoretic metric learning (ITML), neighbor component analysis (NCA), and sparse Bayesian metric learning in latent space [References 20, 23]. Unsupervised metric learning assumes that label information is unavailable, only side information (usually in the form of similar and dissimilar sets). Its goal is to discover low-dimensional manifold structures from the training data while preserving the geometric structure of the data. Typical unsupervised metric learning methods include principal component analysis (PCA), multidimensional scaling (MDS), isotropic feature maps (ISOMAP), locally linear embedding (LLE), and sparse high-dimensional metric learning (SDML). To address the problem of insufficient labeled samples, semi-supervised metric learning methods fully label samples and use unlabeled samples to construct constraints to enhance metric learning performance, combining the advantages of supervised and unsupervised learning. Typical semi-supervised methods have emerged, including Laplace regularized metric learning (LRML), regularized semi-supervised metric learning (RSSML) using local topology and ternary constraints, semi-supervised metric learning for learning symmetric positive definite matrices, and SERAPH, a semi-supervised metric learning paradigm with super-sparseness [References 24, 25]. Weakly supervised metric learning algorithms utilize weaker data point information than supervised metric learning. Unlike supervised metric learning algorithms, weakly supervised metric learning utilizes tuples of similar or dissimilar data points for learning, rather than directly using labeled data. For example, Davood Zabihzadeh (2019) proposed a sparse Bayesian metric learning method for latent space. This method finds a more effective transformation by learning low-dimensional manifold hidden variable noise, mapping the original samples into the latent space, reducing the distance between similar samples and increasing the distance between dissimilar samples.

[0008] While linear metric learning methods are effective for linearly separable problems, they are less effective for nonlinear structural problems. Nonlinear metric learning using kernel methods is currently the primary approach in nonlinear metric learning research. These methods map samples to a feature space using an approximate kernel function corresponding to a high-dimensional feature map, and then leverage existing metric learning methods to learn new metrics in this feature space [Reference 26]. For example, the kernel TML method, which uses kernel principal component analysis for kernelization, is a kernel classification framework for metric learning. This framework not only unifies methods such as LMNN and ITML but also provides several improved support vector machine (SVM) methods. These methods first determine the kernel function parameters empirically and assume that the data noise is Gaussian white noise, resulting in low robustness, which directly affects the performance of subsequent metric learning methods.

[0009] Given a data set consisting of N samples For simplicity, the sample is centered. GMM can be viewed as a linear combination of Gaussian density functions, which can better describe the distribution of many real data. The mathematical representation of GMM is

[0010]

[0011] Here, K represents the number of Gaussian components, π i is a mixing factor and satisfies θ j ={μ j ,Σ j} represents the mean μ associated with Gaussian component j j and the covariance matrix Σ j , j=1,2,…,K. Let X=[x1,x2,…,x N ] is the matrix composed of training data, where it is assumed that the observed data point x i The goal of GMM is to find the optimal parameter Θ' so that the likelihood function Maximum. Under the assumption that the data are independently sampled from the same distribution, the log-likelihood function is expressed as In order to simplify the likelihood representation, the hidden indicator representation set z is introduced ij ∈{0,1} so that h ij =1 means x i Sampled from Gaussian component j, otherwise h ij =0 means x i is sampled from other Gaussian components. Then, p(H|Θ') is written as

[0012]

[0013] Here, K represents the number of Gaussian components. Given GMM parameters Θ' and hidden variables Then the business form of the likelihood function is

[0014]

[0015] Therefore, the log-complete likelihood function can be written as follows

[0016]

[0017] The optimal solution of formula (4) can be obtained by using the Expectation-Maximization (EM) algorithm.

[0018] From a probabilistic point of view, h ij Represents instance xi The probability of belonging to Gaussian component j is in the form of h ij =p(j|x i ,Θ). Formula (4) can be rewritten as follows

[0019]

[0020] in, is the multivariate Gaussian density function, E(h ij )=p(h ij =1|x i ,Θ') is the posterior distribution probability of the category indicator latent variable, and satisfies

[0021] The above optimization problem is a nonlinear problem, and the EM algorithm is generally used to iteratively optimize the model to find the optimal parameter Θ'*. The E step of the EM algorithm calculates the mixing factor π by the posterior distribution probability of the category indicator latent vector j According to the Bayesian formula, the posterior distribution probability is calculated by the following formula

[0022]

[0023] The iterative update form of the mixing factor is obtained by solving the following optimization problem

[0024]

[0025] By constructing the Lagrange equation and j The derivative is equal to 0. According to the constraints, it is easy to get π j Iterative update equation

[0026]

[0027] In the M-step of the EM algorithm, the parameter μ j ,Σ j The estimates are as follows:

[0028]

[0029]

[0030] Here, Θ' is the model parameter obtained in the previous step, Σ ij =(x i -μ j )(x i -μ j ) T , Indicates the number of samples contained in the j-th Gaussian component, j = 1, 2, …, K.

[0031] As a classic machine learning algorithm, GMM has attracted widespread attention due to its advantages, including clear physical significance, simple computation, and ability to describe complex data distributions. In practical industrial process monitoring applications, process data is often high-dimensional, random, and redundant, with a small amount of labeled data and a large amount of unlabeled data. These issues can easily lead to singular covariance matrices and overfitting in GMM models. While some researchers have proposed addressing these issues by leveraging data geometry, variational inference, Dirichlet processes, and KPCA, a series of research and application results have been achieved, significantly promoting the application of GMM in industrial process monitoring. Given the complex nature of industrial process environments, extracting effective low-dimensional latent variables from high-dimensional data containing complex noise and nonlinearity to improve the accuracy of GMM models remains a key issue in GMM multi-process monitoring research. Based on these considerations, this paper integrates kernel space weakly supervised metric learning with latent variable models to extract effective low-dimensional latent variables from complex process data and construct a GMM model, enabling data cluster analysis and sample distribution modeling. The present invention also uses the Student's t-distribution to model the non-Gaussian noise in process data, and automatically determines the latent variable dimension using variational inference (VI) methods. This patent focuses more on improving the performance of subsequent GMMs using metric learning methods, and enhancing GMM's ability to describe nonlinear, uncertain, non-Gaussian noise, and multi-condition data sets.

[0032] Choosing the appropriate number of Gaussian components (GCs), or model selection, is also a key issue in GMM learning. Inappropriate GCs can lead to overfitting. Commonly used mixed model selection methods implement model selection based on minimizing a certain criterion. The main criterion methods include the Akaike Information Criterion (AIC), the Bayesian Information Criterion (BIC), the Absolute Incremental Log-Likelihood (AIL), the Minimum Information Length (MML), and other criteria [References 27, 28]. The present invention uses the Bayesian Information Criterion (BIC) to determine the GMM model structure: BIC = -2ln(L) + kln(N), where N is the number of samples, k is the number of estimated free parameters, and L is the maximum value of the likelihood function of the estimated model. The smaller the value, the higher the model effect. Detailed implementation of the GMM algorithm can be found in references [2, 3].

[0033] In order to achieve the above purpose, the technical solution of the present invention is as follows:

[0034] A multi-operating process fault monitoring and diagnosis method based on kernel latent space robust metric learning GMM includes the following steps:

[0035] S1. Use variational inference methods to learn a robust metric model in the kernel latent space and obtain the low-dimensional nonlinear latent variables and model parameters of the kernel latent space corresponding to the data;

[0036] S101, using kernel techniques to map input process data to a high-dimensional feature space associated with the kernel function, so that the feature space data is linearly separable; go to step S102;

[0037] S102, constructing a robust metric learning latent variable extraction method based on variational inference in the high-dimensional feature space, using variational inference model parameter learning to automatically determine the latent variable dimension; go to step S103;

[0038] S103, establishing a GMM model based on the latent variables extracted in the kernel latent space to perform cluster modeling analysis on the data; go to step S104;

[0039] S104, select samples with high credibility in the cluster according to the GMM model clustering results to generate sample constraint pairs, or generate sample constraint pairs based on prior knowledge, generate a constrained optimization problem to obtain the optimal kernel function parameters; go to step S105;

[0040] S105, repeat the above steps until the algorithm converges; go to step S2;

[0041] S2, GMM multi-condition process monitoring based on robust metric learning with kernel latent space variational inference;

[0042] S201, assuming a given process data sample set The robust metric learning method of variational inference in the kernel latent space is used to obtain the latent variables of the corresponding low-dimensional subspace, which is denoted as Go to step S202;

[0043] S202, based on latent variable set Call the GMM algorithm to obtain the GMM clustering model; generate a constrained optimization problem based on the GMM clustering results to obtain the optimal kernel function parameters; on this basis, use the robust metric learning method of kernel latent space variational inference to generate a new latent variable set Reconstruct the GMM model in the new low-dimensional space; go to step S303;

[0044] S203, iteratively executing the process optimization algorithm model of S301 and S302 above until the GMM converges; constructing global process monitoring statistics based on the GMM clustering results and using them for fault diagnosis based on the contribution of the variables.

[0045] The present invention embeds kernel space weakly supervised metric learning and latent variable model into the GMM modeling process, and uses Student's t-distribution to model non-Gaussian noise. It pays more attention to the improvement of subsequent GMM performance by metric learning methods, so as to enhance GMM's ability to describe nonlinear, uncertain, non-Gaussian noise, and multi-operating condition data sets.

[0046] The technical solutions further refined by the present invention are as follows:

[0047] In the step S1, the kernel implicit space variational inference robust metric learning method is as follows:

[0048] 1) Set the maximum implicit space dimension d, parameter γ, and side information to determine the initial values v and v' of the model hyperparameters m = 1, 2, …, d, i = 1, 2, …, N;

[0049] 2) Randomly initialize the posterior distributions q(ρ), q(U), q(U'), q(α) and q(W) through the values of v and v'. Randomly draw the initial transformation matrix W from the posterior distribution q(W), and t = d;

[0050] 3) Set the iteration number t = 0;

[0051] 4) When t < maxitem, let t = t + 1, i = 1, …, N. At this time, calculate the conditional probabilities q(z i i and z' i |y i ) and q(z' i |y i ) of the low-dimensional non-linear latent variable features z i and z' i . Here, the latent variables z N and z' T are calculated by the following formula

[0052] z = Wφ(x) + e (11)

[0053] where φ(x) = [k(x, x1), k(x, x2), …, k(x, x i )] T is the high-dimensional feature mapping related to the kernel function, k(x, x -i ) represents the kernel function satisfying the Mercer condition. Take the Gaussian kernel function k(x, x1) = exp(-(x, x1) i Λ(x, x1) / 2), and Λ is a diagonal matrix; is the linear mapping matrix that maps the data x from the high-dimensional feature φ(x) to the low-dimensional implicit space. N represents the dimension of the high-dimensional feature φ(x), d is the dimension of the low-dimensional implicit space and satisfies d ≤ N, and the noise e is an independent student t-distribution random variable;<00***441>

[0054] According to the variational inference theory, the optimal approximate solution of the posterior distribution is solved in the following form

[0055] Here, Θ = {Z, Z', U, U', α, ρ, W} represents the set of latent variables Θ -i Indicates removing the hidden variable Θ from the hidden variable set Θ i The set of remaining latent variables, ln p(y,Θ|X,X') represents the complete probability distribution, which is in the form of

[0057]

[0058] Based on variational inference theory, considering the approximate distribution q(Z,Z'), we have

[0059]

[0060] in, and are random variables u i 、u' i expectations, are the expectations of variables ρ and W respectively, T represents the transpose of a matrix or vector, and C is a constant; The parameter γ controls the speed of change of the distance between z and z'; p(z|W,x,ρ,u)=N(z|Wφ(x),(uρ) -1 I d ) is a normal distribution containing latent variable u, Wφ(x) and (uρ) -1 I d are the mean and covariance of the random variable z, I d is a d×d identity matrix; the Gaussian distribution p(z|W,x,ρ,u) is hierarchically constructed by introducing latent variables u and obeys the Student t-distribution latent variable z i The conditional probability of

[0061] p(z|W,x,ρ,v)=st(z|Wφ(x),ρI d ,v) (12)

[0062] in, is the Gamma function, Wφ(x) and ρ are the local parameter and scale parameter of the t-distribution, respectively, and v is the degree of freedom parameter that controls the shape of the distribution; p(z'|W,x,ρ,u') has the same properties as p(z|W,x,ρ,u);

[0063] According to formula (27), q(z i ,z' i |y i =1) can be obtained

[0064]

[0065] From the above we can see that q(z i |y i=1) obeys Gaussian distribution, that is

[0066]

[0067] Thus we can get the mean and variance of the Gaussian distribution:

[0068]

[0069]

[0070] When y i = 0, noting that p(y = 0 | z, z') = 1-p(y = 1 | z, z'), taking the derivative of q(Z, Z') yields the following relationship

[0071]

[0072] According to the Bayesian posterior formula, it has the following form

[0073]

[0074] make The above formula is transformed into

[0075]

[0076] Since q(z i |y i =0) =∫q(z i ,z' i |y i =0)dz' i ,So

[0077]

[0078] Notice and q(y i =1|z i ) are all Gaussian distributions, then their linear combination is still Gaussian distribution; according to r i The definition of Then the above formula can be rewritten as

[0079]

[0080] Gaussian distribution Its mean and covariance distribution are as follows

[0081]

[0082]

[0083] Therefore, for q(z' i |yi =1) and q(z' i |y i =0) The method of obtaining is similar to the above, Gaussian distribution The mean and variance of :

[0084]

[0085]

[0086] Gaussian distribution Its mean and covariance distribution are as follows

[0087]

[0088]

[0089] According to variational inference theory, the posterior distribution approximation q(U) can be decomposed into Notice ln q(u i ) is in the form of

[0090]

[0091] Notice So

[0092]

[0093]

[0094] Then u i and u i The expected form of the logarithm is

[0095]

[0096]

[0097] in, is the common di-gamma function. i )'s variational approximation q(u' i ), u' i and u' i The expected form of the logarithm is

[0098]

[0099]

[0100] Its form is q(u i), the update form of the posterior distribution parameters is similar to equations (48) and (49), which will not be repeated here. In addition, for the distribution p(u' i ) approximation q(u' i ), which is of the form Then the distribution parameter update behavior

[0101]

[0102]

[0103] Using formula (26) to derive q(W), we can get the following form:

[0104]

[0105] Notice w m φ(x' i ) is a scalar, then

[0106]

[0107] From the above formula, we can see that ln q(w m ) is w m is a quadratic function, so q(w m ) obeys a Gaussian distribution, and its mean and covariance are as follows

[0108]

[0109]

[0110] For high-dimensional data, the Nystrom method can be used to obtain the matrix Low rank Approximation, then the matrix inverse is calculated by Woodbury formula, that is

[0111]

[0112] This way the computational complexity becomes O(p 2 d), so calculate It only takes O(pd), and the computational complexity of the entire process is the sum of the above two steps;

[0113] Consider q(α), which can be decomposed into Then q(a m )The optimal solution is

[0114]

[0115] Note that q(a m ) obeys the Gamma distribution The parameter update form is as follows

[0116]

[0117]

[0118] here,

[0119] Considering q(ρ), the derivative of formula (26) can be obtained

[0120]

[0121] in, q(ρ)=Ga(ρ|a ρ ,b ρ ), then the parameter update of q(ρ) is as follows

[0122]

[0123]

[0124] The optimal parameter v can be obtained by the following equation:

[0125]

[0126] Similarly, by solving the equation Get the optimal solution of v', where v' is the random variable u' n Hyperparameters of the distribution;

[0127] If the variational lower bound converges to meet the requirements, the algorithm terminates.

[0128] In the above step 2), initialization: sampling from the Gamma distribution according to the hyperparameters as a m (m=1,2,…,d) and Initial value, ρ = d, W = W0, generally W0 is obtained by using PCA The processing determines the number of iterations t=1,v,v', and the kernel function parameter Λ=I.

[0129] In step S2, the specific implementation process of the GMM multi-operating process monitoring method of robust metric learning based on kernel latent space variational inference is as follows:

[0130] S301, given sample Maximum latent space dimension d, parameter γ, number of iterations st = 1, maximum number of iterations maxiter, convergence threshold ε, kernel function parameter Λ, side information To determine the initial values of model hyperparameters v, v',

[0131] S302, according to the robust metric learning method of running kernel latent space variational inference, obtain the latent variable set and related model parameters Θ in the low-dimensional feature space st ;

[0132] S303, use GMM algorithm to calculate the latent variable set Clustering is performed, and the clustering parameter set is recorded as where μ c and Σ c Represent the mean and covariance matrix of the cth cluster, π c represents the prior probability of the cth cluster, each cluster represents a working condition, and each Gaussian component represents the cluster distribution;

[0133] S304, according to the clustering results Determine sample z i The category to which it belongs, and select samples with high credibility to generate side information in, is the hidden variable z i The posterior probability of belonging to the cth cluster, c = 1, 2, ..., K; each class of sample set is sorted in descending order according to the probability of the class to which the sample belongs, and is recorded as Let Z = [Z1, Z2, ..., Z K ],N c is the number of samples in the cth cluster;

[0134] S305, adjust the Gaussian kernel function parameter Λ: let Z / Z c Indicates moving from Z to Z c Then, the data set is sorted in descending order according to the probability of the cluster c to which it belongs. For each sample set {Z c ,Z / Z c}, c=1,2,…,K; from Z c Take out the samples, where β=η / 8,η=1,2,…,8, Indicates taking a value close to βN c The maximum integer, the corresponding original data set and the probability of the cluster c are recorded as U' c , from Z / Z c Take out the samples, the corresponding original data set and the probability of belonging to non-cluster c are recorded as N' c =card(X' c ), Function card(X' c ) represents a set The number of elements in

[0135] Based on the principle of minimizing the local distance of the sample set and maximizing the non-local distance of the sample set, the samples of the same type are made as close as possible and the samples of different types are made as far away as possible, so as to achieve the purpose of separating samples of different categories. The optimal Gaussian kernel function parameters are obtained by optimizing the problem.

[0136]

[0137] in, is a Gaussian kernel function, the diagonal matrix Λ=diag(Λ1,Λ2,…,Λ D' ) is the kernel function parameter, D' is the dimension of the original space input data, and the function diff(Λ) represents the weighted intra-class local distance between similar data samples in the feature space Weighted inter-class distance between samples of different classes The difference here, Represents a high-dimensional feature map corresponding to the Gaussian kernel function and satisfies The purpose of introducing weights is to pay more attention to the samples at the boundary of the categories and improve the separation of the boundary samples. The following gradient descent method is used to solve the optimal kernel function parameter Λ for the above optimization problem. * , the gradient descent update form is

[0138]

[0139]

[0140] Where t' is the step length, represents the Λ of the g+1th and gth iterations d' value, g is the number of iterations, diff(Λ g ) The value of the function diff(Λ) at the g-th iteration is given by Available

[0141]

[0142] The initial value of the parameter is dst d' represents the average distance between all sample d'th dimension variables, and the termination condition of the gradient descent algorithm is set to |diff(Λ g )-diff(Λ g+1 )|≤10 -6 ;

[0143] S306, select samples with higher feasibility based on the GMM clustering results to generate constraint pairs (X st+1 ,X' st+1 ,y st+1 ), and call the robust metric learning algorithm of kernel latent space variational inference to find the optimal parameter Θst+1 ;

[0144] S307, estimate the value of the objective function ln p(y st+1 ,Θ st+1 |X st+1 ,X' st+1 );

[0145] S308, st←st+1

[0146] Until the condition |ln p(y st-1 ,Θ st-1 |X st-1 ,X' st-1 )-ln p(y st ,Θ st |X st ,X' st )|≤ε or st≥maxiter is satisfied and the algorithm ends, and the optimal model parameters Θ',Θ are obtained;

[0147] If the condition is not met |ln p(y st-1 ,Θ st-1 |X st-1 ,X' st-1 )-ln p(y st ,Θ st |X st ,X' st )|≤ε or st≥maxiter, then continue to execute steps S305-S308;

[0148] S309, fault detection:

[0149] For the new monitoring sample x*, first calculate its corresponding latent variable z*=Wφ(x*) according to the projection matrix W belonging to each Gaussian component C c (c=1,2,…,K) and then determine that the monitoring sample x* belongs to the Gaussian component C c The membership degree of

[0150]

[0151] The distance between the latent variable z* and each Gaussian component This measure follows the approximate χ with d degrees of freedom 2 distribution, so the local probability of sample z* in the cth Gaussian component can be estimated as

[0152]

[0153] Construct a comprehensive statistic of the following form

[0154]

[0155] This statistic integrates the posterior probabilities of all Gaussian components and accurately represents the probability that the monitoring sample x* is within the boundary. Therefore, whether the current working condition is normal can be determined by whether the comprehensive monitoring statistic is less than the predetermined confidence level of 100(1-a)%.

[0156] S310, fault diagnosis:

[0157] For monitoring variable Ξ l (1≤l≤D'), its contribution rate in each Gaussian component can be calculated by the following first-order partial derivative, that is

[0158]

[0159] in, So, regarding the monitoring variable Ξ l The overall contribution of

[0160]

[0161] If the current working condition is in an abnormal state, calculate the contribution of each monitoring variable and select the variable with the maximum contribution, that is,

[0162]

[0163] From formula (67), it can be considered that the variable that causes the abnormal working condition of the process is variable Ξ l* .

[0164] Compared with the prior art, the present invention has the following advantages:

[0165] (1) The present invention is based on the Bayesian variational framework and iteratively calculates the posterior distribution of model parameters and hyperparameters. The model can learn the parameter non-Gaussian white noise of the low-dimensional latent space and find a more effective transformation matrix, thereby improving the robustness of the model.

[0166] (2) Automatically discover the latent space dimension and the sparsity of the solution, overcoming the problem of heavy computational burden brought by the cross-validation method for determining the latent space dimension for large-scale and high-dimensional data. At the same time, it brings similar samples in the latent space closer together and increases the distance between dissimilar samples, effectively avoiding the overfitting problem.

[0167] (3) Unlike the Mahalanobis metric distance, this algorithm has good linear scalability;

[0168] (4) The present invention uses an iterative method to determine the optimal kernel function parameters and optimize the low-dimensional latent space, which not only overcomes the problem that the existing unsupervised and semi-supervised learning methods can only empirically determine the kernel function parameters, but also effectively improves the performance of the subsequent GMM model and effectively improves the description ability of complex nonlinear data, thereby improving the level of multi-condition process monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0169] The present invention will be further described below in conjunction with the accompanying drawings.

[0170] Figure 1 This is a schematic diagram of the robust metric learning probability model structure in the present invention. The arrows in the figure represent the dependency relationship between variables. The arrow indicates that the variable depends on the variable at the end of the arrow. For example, variable u depends on variable v. y is the output of the constrained data pair (x, x'), and its value indicates whether the sample data pair is similar. The latent variable u and the latent variable u' can convert the noise that obeys the Student t-distribution into white noise with a Gaussian distribution, which is beneficial to solving the model optimization problem and improving the modeling ability of complex data. The vector φ and the vector φ' respectively represent the eigenvectors generated by x and x', converting the nonlinear data regression modeling problem into a linear regression modeling problem. The latent variables z and z' are the projections of the high-dimensional data φ and φ' in the low-dimensional feature space, respectively, and the corresponding projection matrix is W. v, v' and γ are all model hyperparameters. The latent variable α controls the projection matrix to be W distribution, and adaptively determines the dimension of the latent variable space. The latent variable ρ and the parameter γ are the conditional probability parameters of given y with respect to z and z'. DETAILED DESCRIPTION

[0171] The present invention will be described in detail below with reference to the embodiments and the accompanying drawings.

[0172] Example 1 Robust Metric Learning GMM Multi-Condition Process Monitoring with Kernel Latent Space Variational Inference

[0173] 1. Robust metric learning model for kernel latent space

[0174] The main idea of the robust metric learning method based on kernel latent space variational inference is to use the kernel technique to map the input process data to a high-dimensional feature space related to the kernel function, so that the feature space data is linearly separable; construct a latent variable extraction method based on robust metric learning based on variational inference in the high-dimensional feature space to automatically determine the dimension of the latent variables; establish a GMM model based on the extracted latent variables for data clustering analysis and data distribution modeling; obtain the optimal kernel function parameters based on the clustering results of the GMM model; and repeat the above steps until the algorithm converges.

[0175] Assume that the label observation data is Binary variable y i ∈{0,1} represents the sample pair (xi,x' i ), When (xi,x' i ) is a similar term, then y i =1, otherwise 0. Then assume that the element data is mapped to the low-dimensional space in the form of

[0176] z=Wφ(x)+e (11)

[0177] Among them, φ(x)=[k(x,x1),k(x,x2),…,k(x,x N )] T is a high-dimensional feature map associated with the kernel function, k(x,x i ) represents a kernel function that satisfies the Mercer condition. Typical kernel functions include Gaussian kernel function and binomial kernel function. In this invention, the Gaussian kernel function k(x, x1)=exp(-(x, x1) T Λ(x,x1) / 2) is used as the kernel function, Λ is a diagonal matrix, and T represents the transpose of the matrix or vector; The linear mapping matrix that maps data x from high-dimensional features φ(x) to low-dimensional latent space, N represents the dimension of high-dimensional features φ(x), d is the dimension of low-dimensional latent space and satisfies d≤N, and noise e is an independent Student t-distributed random variable that can represent Gaussian and non-Gaussian noise, improving the ability to extract nonlinear features from complex noise data, thereby improving the robustness of the model. Note that the kernel matrix needs to be centered in actual use. The conditional probability p(z|W,x,ρ) of the latent variable z is -1 ,v) is

[0178] p(z|W,x,ρ,v)=st(z|Wφ(x),ρI d ,v) (12)

[0179]

[0180] is the Gamma function, Wφ(x) and ρ are the local parameter and scale parameter of the t-distribution respectively, v is the degree of freedom parameter, d is the dimension of the latent variable z, D is the dimension of the original input data space, z is the nonlinear latent variable feature shown in formula (11), and T is the matrix or vector transpose.

[0181] The Student t-distribution can be constructed hierarchically using a Gaussian distribution with a latent variable u, i.e.

[0182] p(z|W,x,ρ,u)=N(z|Wφ(x),(uρ) -1 I d ) (13)

[0183] Among them, the prior distribution of u is Gamma distribution Here the Gamma function is Where a and b are the shape and scale parameters of the function respectively, and N(z|Wφ(x),(uρ) -1 I d ) indicates that the variable z has a mean of Wφ(x) and a variance of (uρ) -1 I d Gaussian distribution, I d is a d×d identity matrix. As can be seen from the above formula, the t-distribution can be interpreted as a mixture of infinite Gaussian distributions, and u controls the noise level of the variable z.

[0184] Based on the idea of metric learning, that is, after the original data is mapped to the low-dimensional latent space, similar samples are closer and dissimilar samples are farther away, the probability distribution of the output binary variable y is defined as

[0185] p(y|z,z')=p(y=1|z,z')p(y=0|z,z') (14)

[0186]

[0187] It can be seen from the above formula that if two latent variables z, z' are consistent with each other, then their similarity probability is equal to 1, otherwise its value decreases exponentially with the square distance between the two latent variables, and its value change rate is controlled by the parameter γ.

[0188] Similar to the probabilistic PCA method, the prior probability of the transformation matrix W is set to

[0189]

[0190] Here, w m is the mth row vector of W, and the hyperparameter a m Indicates the precision of the distribution, m=1,2,…,d, I D is the D×D identity matrix. For the Bayesian model, define α=[a1,a2,…,a d ] T and ρ have the following prior forms

[0191]

[0192] p(ρ)=Ga(ρ|a ρ ,b ρ ) (18)

[0193] Here, the hyperparameters of the Gamma distribution can be calculated by estimating the posterior distribution using the variational method. a ρ ,, b ρ are all hyperparameters. During the estimation process, if the parameters When the value approaches infinity, the corresponding vector wm will be removed from the model. Therefore, the actual dimension of the latent space will be determined by a m The value is automatically determined, reducing the difficulty of manually setting the dimension of the latent space.

[0194] According to the model shown in formula (14), the conditional probability of the output variable for the original input variable is

[0195]

[0196] Obviously, the posterior probability distribution of the original data pair is equivalent to the Gaussian kernel function of the latent variable. By optimizing the parameters (W, u, ρ, γ), the accuracy of the classifier will be greatly improved. The optimal model parameters can be obtained by maximizing the likelihood function p(y|X, X', Θ), that is,

[0197]

[0198] Where Θ* is the optimal model parameter obtained after solving the optimization problem, Θ* is the parameter to be optimized, p is the probability representation (commonly used in mathematics), y is the label vector composed of the labels of the sample set data, and |X, is the data pair (x i , x' i ) i The set of X' is the data pair (x i , x' i ) in x' i A collection of components.

[0199] Since it is impossible to directly find a closed-form solution that maximizes the likelihood function represented by the Student's t-distribution, MCMC and variational Bayesian inference methods are commonly used to learn the posterior distribution of the learning parameters. Due to computational efficiency and convergence issues with MCMC, the VI method is used to learn the above model parameters.

[0200] 2. Model parameter learning for variational inference

[0201] First, define the latent variable set Δ={Δ1,Δ2,…,Δ n'}, and the remaining model parameters are denoted as Θ, then the logarithm of the likelihood function is expressed as

[0202]

[0203] Here, the variational distribution q(Δ) represents an arbitrary distribution with respect to Δ, and p(Δ|X,Θ) represents the true posterior distribution with respect to Δ. represents the Kullback-Leibler divergence between q(Δ) and p(Δ|X,Θ). When q(Δ)=p(Δ|X,Θ), the relative entropy is 0. Obviously, F(q(Δ),Θ) is the lower bound of the likelihood function ln p(X|Θ). Maximizing F(q(Δ),Θ) will cause KL(q(Δ)||p(Δ|X,Θ)) to approach 0, that is, q(Δ) is approximately p(Δ|X,Θ). Therefore, the optimal solution of q(Δ) can be obtained by the following optimization problem

[0204]

[0205] Here, q * (Δ) represents the variational decomposition of q(Δ). Based on the mean field theory, q(Δ) can be decomposed into the product of latent variables Then the variational lower bound F(q(Δ),Θ) is transformed into the following form for each variation

[0206] F(q(Δ),Θ)=∫q(Δ1)q(Δ2)…q(Δ n' )(ln p(Δ,X|Θ))dΔ1dΔ2…dΔ n' -∫q(Δ)lnq(Δ)dΔ+const

[0207] Here, const is considered as a constant as an independent term independent of Δ. Solve F(q(Δ), Θ) and take the derivative for all variational distributions. Then the optimal solution for each variational distribution is given by

[0208]

[0209] in, represents ln p(Δ,X|Θ) versus Δ-Δ i' Expectations, here Δ-Δ i' Represents the set Δ to delete the element Δ i' The optimal solution of q(S) and q(U) is similar to the above.

[0210] Let the hidden variable set Θ={Z,Z',U,U',α,ρ,W}, The remaining model parameters are recorded as Δ, so that the posterior distribution approximation q(Θ) of the latent variable can be decomposed into the following form

[0211] q(Θ)=Πq(Z,Z')q(U)q(U')q(α)q(W)q(ρ) (24)

[0212] Then the optimal posterior distribution approximation solution is solved by the following form

[0213]

[0214] Here, Θ = {Z, Z', U, U', α, ρ, W} represents the set of latent variables, Θ j-i Indicates removing the hidden variable Θ from the hidden variable set Θ i After the set, ln p(y,Θ|X,X') represents the complete probability distribution.

[0215] according to Figure 1 The probability model shown can be written in the form of a complete probability distribution

[0216]

[0217] Figure 1 The standard of prior probability parameters is simplified in this paper, and only the training data, model parameters, latent variables and parameters that need to be updated are marked. The total likelihood distribution function and the posterior distribution form of latent variables are based on Figure 1 get.

[0218] According to variational theory, the optimal solution q(Θ) can be obtained by taking the derivative of the latent variable posterior approximation using the above formula.

[0219] The specific process of finding the optimal solution q(Θ) is as follows:

[0220] 1) Consider the latent variables Z, Z'

[0221] Derivative q(Z,Z'), when y i =1, we can get the following relationship

[0222]

[0223] in, is the random variable u i expectations, are the expectations of the corresponding variables.

[0224] Note that the posterior approximation q(z i |y i =1) can be obtained by the following formula

[0225]

[0226] make Then in formula (28)

[0227]

[0228] Therefore q(z i |y i =1) is converted into the following form

[0229]

[0230] From the above we can see that q(z i |y i =1) obeys Gaussian distribution, that is

[0231]

[0232] Thus we can get the mean and variance of the Gaussian distribution:

[0233]

[0234]

[0235] When y i = 0, noting that p(y = 0 | z, z') = 1-p(y = 1 | z, z'), taking the derivative of q(Z, Z') yields the following relationship

[0236]

[0237] According to the Bayesian posterior formula, it has the following form

[0238]

[0239] make The above formula is transformed into

[0240]

[0241] Since q(z i |y i =0) = ∫q(zi,z' i |yi=0)dz' i ,So

[0242]

[0243] Notice and q(y i =1|z i ) are all Gaussian distributions, then their linear combination is still Gaussian distribution; according to r i The definition of Then the above formula can be rewritten as

[0244]

[0245] Its mean and covariance distribution are as follows

[0246]

[0247]

[0248] S202. Consider q(W)

[0249] The derivative of formula (26) with respect to q(W) can be obtained as follows:

[0250]

[0251] Notice w m φ(x' i ) is a scalar, then

[0252]

[0253]

[0254] From the above formula, we can see that ln q(w m ) is w m is a quadratic function, so q(w m ) obeys a Gaussian distribution, and its mean and covariance are as follows

[0255]

[0256]

[0257] For high-dimensional data, the Nystrom method can be used to obtain the matrix Low rank Approximation, then the matrix inverse is calculated by Woodbury formula, that is

[0258]

[0259] This way the computational complexity becomes O(p 2 d), so calculate It only takes O(pd), and the computational complexity of the entire process is the sum of the above two steps.

[0260] S203. Consider the posterior distribution approximation q(U) and q(U')

[0261] According to variational inference theory, the posterior distribution approximation q(U) can be decomposed into Notice ln q(u i ) is in the form of

[0262]

[0263] Notice So

[0264]

[0265]

[0266] Then u i and u i The expected form of the logarithm is

[0267]

[0268]

[0269] in, is a common di-gamma function, for q(u i '), its form is the same as q(u i ), the update form of the posterior distribution parameters is similar to Equations (48) and (49), and will not be repeated here.

[0270] S204. Consider q(ρ), and take the derivative of formula (26) to get

[0271]

[0272] in, q(ρ)=Ga(ρ|a ρ ,b ρ ), then the parameter update of q(ρ) is as follows

[0273]

[0274]

[0275] S205. Consider q(α), which can be decomposed into Then q(a m )The optimal solution is

[0276]

[0277] Note that q(a m ) obeys the Gamma distribution The parameter update form is as follows

[0278]

[0279]

[0280] here,

[0281] S206. Consider v and v'

[0282] The optimal parameter v can be obtained by solving the following optimization problem:

[0283]

[0284] The above optimization problem is a nonlinear optimization problem, and the optimal solution of v can be easily obtained using Matlab functions. Similarly, the optimal solution of v' can be obtained by solving an optimization problem similar to Equation (57).

[0285] Algorithm 1 Kernel Latent Space Variational Inference Robust Metric Learning Algorithm Process

[0286] Type hyperparameter initial values v, v', m=1,2,…,d,i=1,2,…,N

[0287] 2. Initialization: through v, v', a ρ ,b ρ Randomly initialize the posterior distribution q(ρ), q(U), q(U'), q(α) and q(W) with the value, and randomly extract the initial transformation matrix W from the posterior distribution q(W), t=d

[0288] Sampling from a Gamma distribution according to the hyperparameters as a m (m=1,2,…,d) and Initial value, ρ = d, W = W0, generally W0 is obtained by using PCA Processing is determined, the number of iterations t = 1, v, v', the kernel function parameter Λ = I

[0289] 3. Set the number of iterations t = 0

[0290] 4while t <maxitem

[0291] t=t+1

[0292] for i=1,…,N do

[0293] Update q(z) using the formula ()-() i |y i ) and q(z' i |y i )

[0294] Update q(u using the formula ()-() i ) and q(u' i )

[0295] endfor

[0296] for i=1,…,N do

[0297] Update q(z using equations (32)-(33), (39)-(40) and similar forms i |y i ) and q(z' i |y i)

[0298] Update q(u using equations (49)-(50) and similar forms i ) and q(u' i )

[0299] end for

[0300] for m=1,…,d do

[0301] Update q(w m )

[0302] Update q(a) using equations (55)-(56) m )

[0303] end for

[0304] Update q(ρ) using equations (52)-(53)

[0305] Solve the optimization problem shown in formula (57) to obtain the optimal v and v'

[0306] If the variational lower bound converges to

[0307] Then terminates while

[0308] end if

[0309] end while

[0310] Algorithm 1 is a robust metric learning algorithm for kernel latent space variational inference, which constructs a robust probabilistic metric learning model from the input data side information. Extracting low-dimensional nonlinear latent features Algorithm 1 not only improves the robustness of the model and eliminates the influence of data noise on latent variable extraction, but also ensures the extraction of low-dimensional nonlinear latent variable sets in high-dimensional feature space. In summary, Algorithm 1 solves the problem that high-dimensional nonlinear redundant data makes it difficult for GMM to accurately divide data modes and the covariance of Gaussian sub-models is singular, effectively improving the performance of GMM.

[0311] In actual implementation, the original data set is first clustered using clustering methods such as k-means or GMM, and samples with high credibility in the cluster are selected to generate sample constraint pairs. Sample constraint pairs can also be generated based on prior knowledge.

[0312] 3. GMM Multi-condition Process Monitoring with Robust Metric Learning Based on Kernel Latent Space Variational Inference

[0313] Assume that a given process data sample set The robust metric learning method of variational inference in the kernel latent space is used to obtain the latent variables of the corresponding low-dimensional subspace, which is denoted as Based on latent variable set Call the GMM algorithm to obtain the GMM clustering model; generate constraints based on the GMM clustering results to adjust the kernel function parameters; on this basis, use the robust metric learning method of kernel latent space variational inference to generate a new set of latent variables A GMM model is constructed in the new low-dimensional space. The process optimization algorithm model of S301 and S302 is iteratively executed until the GMM converges. A global process monitoring statistic is constructed based on the GMM clustering results and used for fault diagnosis based on the contribution of the variables.

[0314] It can be seen from the above algorithm implementation process that the multi-condition monitoring modeling method proposed in the present invention closely integrates the kernel latent space robust metric learning method with GMM modeling to better improve the subsequent model performance issues.

[0315] The implementation process of the GMM multi-condition process monitoring method based on robust metric learning of kernel latent space variational inference in Algorithm 2 is as follows:

[0316] Iteration number maxiter, convergence threshold ε, kernel function parameter Λ, side information Determine the initial values of model hyperparameters v, v', a ρ ,b ρ , m=1,2,…,d, i=1,2,…,N;

[0317] Step 3: Repeat

[0318] Step 4: Run the robust metric learning algorithm of kernel latent space variational inference according to Algorithm 1 to obtain the latent variable set and related model parameters Θ in the low-dimensional feature space. st ;

[0319] Step 5: Use the GMM algorithm to calculate the latent variable set Clustering is performed, and the clustering parameter set is recorded as Here μ c and Σ c Represent the mean and covariance matrix of the cth cluster, π c represents the prior probability of the cth cluster, each cluster represents a working condition, and each Gaussian component represents the cluster distribution;

[0320] Step 6: According to the clustering results Determine sample z i The category to which it belongs, and select samples with high credibility to generate side information in, is the hidden variable z i The posterior probability of belonging to the cth cluster, c = 1, 2, ..., K. Arrange each class of sample set in descending order according to the probability of the class to which the sample belongs, and record it as Let Z = [Z1, Z2, ..., Z K ],N c is the number of samples in the cth cluster;

[0321] Step 7: Adjust the Gaussian kernel function parameter Λ: Let Z / Z c Indicates moving from Z to Z c Then, the data set is sorted in descending order according to the probability of the cluster c to which it belongs. For each sample set {Z c ,Z / Z c}, c=1,2,…,K; from Z c Take out the samples, where β=η / 8,η=1,2,…,8, Indicates taking a value close to βN c The maximum integer, the corresponding original data set and the probability of the cluster c are recorded as U' c , from Z / Z c Take out the samples, and the corresponding original data set and the probability of belonging to non-cluster c are recorded as N' c =card(X' c ), function Representing a collection The number of elements in

[0322] Based on the principle of minimizing the local distance of the sample set and maximizing the non-local distance of the sample set, the samples of the same type are made as close as possible and the samples of different types are made as far away as possible, so as to achieve the purpose of separating samples of different categories. The optimal Gaussian kernel function parameters are obtained by optimizing the problem.

[0323]

[0324] in, is a Gaussian kernel function, the diagonal matrix Λ=diag(Λ1,Λ2,…,Λ D' ) is the kernel function parameter, D' is the dimension of the original space input data, and the function diff(Λ) represents the weighted intra-class local distance between similar data samples in the feature space Weighted inter-class distance between samples of different classes The difference here, Represents a high-dimensional feature map corresponding to the Gaussian kernel function and satisfies The purpose of introducing weights is to give more attention to samples at the boundary of categories and improve the separation of samples at the boundary. The following gradient descent method is used to solve the above optimization problem to obtain the optimal kernel function parameter Λ * , the gradient descent update form is

[0325]

[0326]

[0327] Where t' is the step length, represents the Λ of the g+1th and gth iterations d' value, g is the number of iterations, diff(Λ g ) The value of the function diff(Λ) at the g-th iteration is given by Available

[0328]

[0329] The initial value of the parameter is dst d' represents the average distance between all sample d'th dimension variables, and the termination condition of the gradient descent algorithm is set to |diff(Λ g )-diff(Λ g+1 )|≤10 -6 .

[0330] Step 8: Select samples with higher feasibility based on the GMM clustering results to generate constraint pairs (X st+1 ,X' st+1 ,y st+1 ), and call the robust metric learning algorithm of kernel latent space variational inference to find the optimal parameter Θ st+1

[0331] Step 9: Estimate the value of the objective function ln p(y st+1 ,Θ st+1 |X st+1 ,X' st+1 )

[0332] Step 10: st←st+1

[0333] until|ln p(y st-1 ,Θ st-1 |X st-1 ,X' st-1 )-ln p(y st ,Θ st |X st ,X' st )|≤ε or st≥maxiter

[0334] step11: Get the optimal model parameters Θ', Θ

[0335] Step 12: Fault detection (building monitoring statistics):

[0336] For the new monitoring sample x*, first calculate its corresponding latent variable z*=Wφ(x*) according to the projection matrix W belonging to each Gaussian component C c (c=1,2,…,K), and then determine that the monitoring sample x* belongs to the Gaussian component C c The membership degree of

[0337]

[0338] The distance between the latent variable z* and each Gaussian component This measure follows the approximate χ with d degrees of freedom 2 distribution, so the local probability of sample z* in the cth Gaussian component can be estimated as

[0339]

[0340] Construct a comprehensive statistic of the following form

[0341]

[0342] This statistic combines the posterior probabilities of all Gaussian components and accurately represents the probability that the monitoring sample x* is within the bounds. Therefore, whether the current operating condition is normal can be determined by whether this comprehensive monitoring statistic is less than a predetermined confidence level of 100(1-a)%. In practice, the confidence level is generally set to 95%.

[0343] Step 13 Fault diagnosis (calculation of monitoring variable contribution):

[0344] For monitoring variable Ξ l (1≤l≤D'), its contribution rate in each Gaussian component can be calculated by the following first-order partial derivative, that is

[0345]

[0346] in So, regarding the monitoring variable Ξ l The overall contribution of

[0347]

[0348] If the current working condition is in an abnormal state, calculate the contribution of each monitoring variable and select the variable with the maximum contribution, that is,

[0349]

[0350] From formula (67), it can be considered that the variable that causes the abnormal working condition of the process is variable Ξ l*

[0351] Algorithm 2 is divided into the following steps:

[0352] 1. Use the Bayesian variational inference algorithm to find the optimal robust metric learning model parameters and latent variable set (Algorithm 1 implementation);

[0353] 2. Use the GMM algorithm to perform clustering in the kernel latent space, and select samples with high credibility based on the clustering results to generate pairwise constraints;

[0354] 3. Solve the constrained optimization problem to obtain the optimal kernel function parameters;

[0355] 4. Go to step 1 to iteratively obtain the optimal kernel function parameters and kernel space latent variable set until the algorithm converges.

[0356] This algorithm is mainly composed of four stages, namely, kernel latent space robust metric learning to discover low-dimensional subspace, GMM clustering, fault monitoring and fault diagnosis. The optimal subspace and GMM clustering model are obtained through the iteration of robust low-dimensional subspace and GMM clustering, which better realizes the modeling of nonlinear and complex noise process data; abnormal working conditions are monitored by constructing monitoring statistics, and the key variables causing abnormal working conditions can be determined by the contribution of monitoring variables, thus realizing fault monitoring and diagnosis of multi-working condition processes.

[0357] In summary, Algorithm 1 of this invention uses variational inference to learn a robust distance metric in the kernel latent space, which is the foundation and core of the entire method. Algorithm 2 iteratively calls Algorithm 1, uses the results of Algorithm 2 to generate an optimization problem, solves the optimization problem to obtain the optimal kernel function parameters, and thus obtains the optimal latent variable space. The GMM clustering results in the kernel latent space generate fault monitoring statistics to monitor faults, and diagnose any faults.

[0358] In addition to the above embodiments, the present invention may also have other implementations. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection required by the present invention.

[0359] References:

[0360] [1]J.Yu.A nonlinear kernel Gaussian mixture model based on inferentialmonitoring approach for fault detection and diagnosis of chemical processes.Chem.Eng.Sci.,68(1):506-519,2012.

[0361] [2] Ren Shijin. Research on key technologies for industrial process fault monitoring, Postdoctoral research report, Zhejiang University, 2019

[0362] [3] Sun Xianchang, Tian Xuemin, Zhang Ni. A multi-process fault diagnosis method based on GMM. Computers and Applied Chemistry, 2014, 31(1): 33-39

[0363] [4]Zhichao,Xuefeng Yan.Complex dynamic process method based on slowfeature analysis model of muilti-subspace partition.ISA Transactions,95:68-91,2019.

[0364] [5] Yue Wang, Chunhui Zhao. Probabilistic fault diagnosis method based on the combination of nest-loop fisher discriminant analysis and analysis of relative changes. Control Engineering Practice 68 (2017) 32–45

[0365] [6] Junhua Zheng, Jinlin Zhu, Guangjie Chen, Zhihuan Song, ZhiqiangGe. Data-driven supervised fault diagnosis methods based on latent variable models: a comparative study. Chemometrics and Intelligent Laboratory Systems, 18715:41-52, 2019.

[0366] [7]Bei Wang,Zhichao Li,Zhenwen Dai,Neil Lawrence,Xuefeng Yan.Data-driven mode identification and unsupervised fault detection for nonlinearmultimode processes.IEEE Transactions on Industrial informatics,16(6):3651-3660,2020.

[0367] [8]Reza Sharifi,Reza Langari.Nonlinear sensor fault diagnosis usingmixture of probabilistic PCA models.Mechanical Systems and Signal Processing,8515:638-650,2017.

[0368] [9]Le Zhou,Jiaqi Zheng,Zhiqiang Ge,Zhihuan Song,ShengdaoShan.Multimode process monitoring based on switching autoregressive dynamiclatent variable model.IEEE Transactions on Industrial ELECTRONICS,65(10):8184-8195,2018.

[0369]

[10] Le Yao,Zhiqiang Ge.Scalable Semisupervised GMM for Big DataQuality Prediction in Multimode Processes.IEEE Transactions on IndustrialElectronics,66(5):1 231-1241,2019.

[0370]

[11] Weiming Shao,Zhiqiang Ge,Zhihuan Song,Jingbo Wang.SemisupervisedRobust Modeling of Multimode Industrial Processes for Quality VariablePrediction Based on Student's t-Mixture Model.IEEE Transactions on IndustrialInformatics,16(5):896-910,2020

[0371]

[12] Qingqiang Sun,Zhiqiang Ge.Deep Learning for Industrial KPIPrediction:When Ensemble Learning Meets Semi-Supervised Data.IEEETransactions on Industrial Informatics,2020,DOI:10.1109 / TII.2020.2969709.

[0372]

[13] Ziqi Zhu,Jiayuan Zhang,Jixin Zou,Chunhua Deng.Multi-kernelGaussian process latent variable regression model for high-dimensionalsequential data modeling.Neurocomputing 348(2019)3–15.

[0373]

[14] Weiming Shao,Zhiqiang Ge,Zhihuan Song,Le Zhou.Data-DrivenPredictive Model Based on Locally Weighted Bayesian Gaussian Regression,2019IEEE 8th Data Driven Control and Learning Systems Conference(DDCLS)

[0374]

[15] Xiaofeng Yuan,Zhiqiang Ge,Biao Huang,Zhihuan Song,YalinWang.Semisupervised JITL Framework for Nonlinear Industrial Soft SensingBased on Locally Semisupervised Weighted PCR.IEEE Transactions on IndustrialInformatics,13(2):,2017.

[0375]

[16] Jinlin Zhu,Zhiqiang Ge,Zhihuan Song,Jingxiang Liu,Tao Liu,JunghuiChen.Sequential local-based Gaussian mixture model for monitoring multiphasebatch processes.Chemical Engineering ScienceVolume,181:101-113,2018

[0376]

[17] Weiming Shao,Zhiqiang Ge,Zhihuan Song.Bayesian Just-in-TimeLearning and Its Application to Industrial Soft Sensing.IEEE Transactions onIndustrial Informatics,10.1109 / TII.2019.2950272.

[0377]

[18] Zhichao,Xuefeng Yan.Complex dynamic process method based on slowfeature analysis model of muilti-subspace partition.ISA Transactions,95:68-91,2019.

[0378]

[19] Weiming Shao,Zhiqiang Ge,Zhihuan Song.Softer-sensor developmentfor processes with multiple operating models based on semisupervied Gaussianmixture regression.IEEE Transaction on Control Systems Technology,DOI:10.11.9 / TCST.2018.2856845.

[0379]

[20] Davood Zabihzadeh,Reza Monsefi,Hadi Sadoghi Yazdi.Sparse Bayesianapproach for metric learning in latent space.Knowledge-Based Systems,178:11-24,2019

[0380]

[21] S.Xiang,F.Nie,C.Zhang,Learning a mahalanobis distance metric fordata clustering and classification,Pattern Recognit.41(12) (2008)3600–3612.

[0381]

[22] M.Der,L.K.Saul,Latent coincidence analysis:A hidden variablemodel for distance metric learning,in:Advances in Neural InformationProcessing Systems,2012,pp.3230–3238.

[0382]

[23] M.Harandi,M.Salzmann,R.Hartley,Joint dimensionality reduction andmetric learning:A geometric take,in:International Conference on MachineLearning(ICML),EPFL-CONF-229290,2017

[0383]

[24] Q.Qian, R.Jin, S.Zhu, Y.Lin, Fine-grained visual categorization via multistage metric learning, in: Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition, 2015, pp.3716–3724.

[0384]

[25] P.Zadeh, R.Hosseini, S.Sra, Geometric mean metric learning, in: International Conference on Machine Learning, 2016, pp.2464–2471.

[0385]

[26] Li Xin. Research on nearest neighbor credit scoring model based on metric learning. Doctoral dissertation, Shanghai University, 2017.

[0386]

[27] Huang T.,Peng H.,Zhang K..Model selection for Gaussian mixturemodels.Statistica Sinica, 2017:147-169.

[0387]

[28] Ma JY,Jiang JJ,Liu CY,etc.Feature guided Gaussian mixturemodel with semisupervised EM and local geometric constraint for retinal imageregistration.Information Sciences,2017,417:128-142.

Claims

1. A multi-condition process fault monitoring and diagnosis method, characterized in that: The following steps are involved: S1. Use variational inference methods to learn a robust metric model in the kernel latent space and obtain the low-dimensional nonlinear latent variables and model parameters of the kernel latent space corresponding to the data; S101, using kernel techniques to map input process data to a high-dimensional feature space associated with the kernel function, so that the feature space data is linearly separable; go to step S102; S102, constructing a robust metric learning latent variable extraction method based on variational inference in the high-dimensional feature space, using variational inference model parameter learning to automatically determine the latent variable dimension; go to step S103; S103, establishing a GMM model based on the latent variables extracted in the kernel latent space to perform cluster modeling analysis on the data; go to step S104; S104, select samples with high credibility in the cluster according to the GMM model clustering results to generate sample constraint pairs, or generate sample constraint pairs based on prior knowledge, generate a constrained optimization problem to obtain the optimal kernel function parameters; go to step S105; S105, repeat the above steps until the algorithm converges; go to step S2; Among them, the robust metric learning method of kernel latent space variational inference is as follows: 1) Set the maximum latent space dimension d, parameter γ, and secondary information To determine the initial values of model hyperparameters v, v', a ρ ,b ρ , m=1,2,…,d, i=1,2,…,N; 2) Through v, v', a ρ ,b ρ The posterior distribution q(ρ), q(U), q(U'), q(α) and q(W) are randomly initialized with the value of , and the initial transformation matrix W is randomly drawn from the posterior distribution q(W), τ = d; 3) Set the number of iterations t = 0; 4) When t<maxitem, let t=t+1, i=1,…,N, q(z i |y i =1) obeys a Gaussian distribution with a mean and variance of: Gaussian distribution Its mean and covariance distribution are as follows For q(z' i |y i =1) and Gaussian distribution The mean and variance of : Gaussian distribution Its mean and covariance distribution are as follows u i and u i The expected form of the logarithm is in, is a common di-gamma function, for the distribution p(u' i )'s variational approximation q(u' i ), u' i and u' i The expected form of the logarithm is q(w m ) obeys a Gaussian distribution, and its mean and covariance are as follows For high-dimensional data, the Nystrom method can be used to obtain the matrix Low rank Approximation, then the matrix inverse is calculated by Woodbury formula, that is q(α m ) obeys the Gamma distribution The parameter update form is as follows here, q The parameter update of (ρ) is as follows The optimal parameter v can be obtained by solving the following optimization problem: Similarly, by solving the equation Get the optimal solution of v', where v' is the random variable u' n Hyperparameters of the distribution; If the variational lower bound converges to meet the requirements, the algorithm terminates; S2, GMM multi-condition process monitoring based on robust metric learning with kernel latent space variational inference; S201, assuming a given process data sample set The robust metric learning method of variational inference in the kernel latent space is used to obtain the latent variables of the corresponding low-dimensional subspace, which is denoted as Go to step S202; S202, based on latent variable set Call the GMM algorithm to obtain the GMM clustering model; generate a constrained optimization problem based on the GMM clustering results to obtain the optimal kernel function parameters; on this basis, use the robust metric learning method of kernel latent space variational inference to generate a new latent variable set Reconstruct the GMM model in the new low-dimensional space; go to step S203; S203, iteratively executing the process optimization algorithm model of S201 and S202 above until the GMM converges; constructing global process monitoring statistics based on the GMM clustering results and using them for fault diagnosis based on the contribution of the variables; The specific implementation process of the GMM multi-condition process monitoring method based on robust metric learning of kernel latent space variational inference is as follows: ⑴Given sample Maximum latent space dimension d, parameter γ, number of iterations st = 1, maximum number of iterations maxiter, convergence threshold ε, kernel function parameter Λ, side information To determine the initial values of model hyperparameters v, v', a ρ ,b ρ ; ⑵According to the robust metric learning method of running kernel latent space variational inference, the latent variable set and related model parameters Θ in the low-dimensional feature space are obtained st ; ⑶ Use GMM algorithm to calculate the latent variable set Clustering is performed, and the clustering parameter set is recorded as where μ c and Σ c Represent the mean and covariance matrix of the c-th cluster, π c represents the prior probability of the cth cluster, each cluster represents a working condition, and each Gaussian component represents the cluster distribution; ⑷ According to the clustering results Determine sample z i The category to which it belongs, and select samples with high credibility to generate side information in, is the hidden variable z i The posterior probability of belonging to the cth cluster, c = 1, 2, ..., K; each class of sample set is sorted in descending order according to the probability of the class to which the sample belongs, and is recorded as Let Z = [Z1, Z2, ..., Z K ],N c is the number of samples in the cth cluster; ⑸ Adjust the Gaussian kernel function parameter Λ: let Z / Z c Indicates moving from Z to Z c Then, the data set is sorted in descending order according to the probability of the cluster c to which it belongs. For each sample set {Z c ,Z / Z c }, c=1,2,…,K; from Z c Take out the samples, where β=η / 8,η=1,2,…,8, Indicates taking a value close to βN c The maximum integer, the corresponding original data set and the probability of the cluster c are recorded as U' c , from Z / Z c Take out the samples, and the corresponding original data set and the probability of belonging to non-cluster c are recorded as N' c =card(X' c ), function Representing a collection The number of elements in Based on the principle of minimizing the local distance of the sample set and maximizing the non-local distance of the sample set, the samples of the same type are made as close as possible and the samples of different types are made as far away as possible, so as to achieve the purpose of separating samples of different categories. The optimal Gaussian kernel function parameters are obtained by optimizing the problem. in, is a Gaussian kernel function, the diagonal matrix Λ=diag(Λ1,Λ2,…,Λ D' ) is the kernel function parameter, D' is the dimension of the original space input data, and the function diff(Λ) represents the weighted intra-class local distance between similar data samples in the feature space Weighted inter-class distance between samples of different classes The difference here, Represents a high-dimensional feature map corresponding to the Gaussian kernel function and satisfies The purpose of introducing weights is to pay more attention to the samples at the boundary of the categories and improve the separation of the boundary samples. The following gradient descent method is used to solve the optimal kernel function parameter Λ for the above optimization problem. * , the gradient descent update form is Where τ' is the step size, represents the Λ of the g+1th and gth iterations d' value, g is the number of iterations, diff(Λ g ) The value of the function diff(Λ) at the g-th iteration is given by Available The initial value of the parameter is dst d' represents the average distance between all sample d'th dimension variables, and the termination condition of the gradient descent algorithm is set to |diff(Λ g )-diff(Λ g+1 )|≤10 -6 ; (6) According to the GMM clustering results, select samples with higher feasibility to generate constraint pairs (X st+1 ,X' st+1 ,y st+1 ), and call the robust metric learning algorithm of kernel latent space variational inference to find the optimal parameter Θ st+1 ; ⑺ Estimate the value of the objective function lnp(y st+1 ,Θ st+1 |X st+1 ,X' st+1 ); ⑻st←st+1 Until the condition |ln p(y st-1 ,Θ st-1 |X st-1 ,X' st-1 )-lnp(y st ,Θ st |X st ,X' st The algorithm ends after )|≤ε or st≥maxiter is satisfied; Get the optimal model parameters Θ', Θ; ⑼Fault detection: For the new monitoring sample x*, first calculate its corresponding latent variable z*=Wφ(x*) according to the projection matrix W belonging to each Gaussian component C c (c=1,2,…,K) and then determine that the monitoring sample x* belongs to the Gaussian component C c The membership degree of The distance between the latent variable z* and each Gaussian component This measure follows the approximate χ with d degrees of freedom 2 distribution, so the local probability of sample z* in the cth Gaussian component can be estimated as Construct a comprehensive monitoring statistic in the following form Whether the current working condition is normal can be determined by whether the comprehensive monitoring statistic is less than a predetermined confidence level of 100(1-a)%. ⑽Fault diagnosis: For monitoring variables Ξ l (1≤l≤D'), its contribution rate in each Gaussian component can be calculated by the following first-order partial derivative, that is in, So, regarding the monitoring variable Ξ l The overall contribution of If the current working condition is in an abnormal state, calculate the contribution of each monitoring variable and select the variable with the maximum contribution, that is, From formula (67), it can be considered that the variable that causes the abnormal working condition of the process is variable Ξ l* .

Images