Liquid rocket engine variational inference learning and diagnosis method containing input noise multi-class Gaussian process

Abstract

The invention provides an input noise multi-class Gaussian process-containing liquid rocket engine variational inference learning and diagnosis method, which comprises the steps of introducing noise and establishing a multi-class Gaussian process model with noise, converting a data classification problem into a posterior approximation problem based on Gaussian variational inference, and adopting a small-batch data approximation method to obtain an input noise multi-class Gaussian process-containing liquid rocket engine variational inference learning and diagnosis result. And performing optimization estimation on the variation lower bound, obtaining approximate posteriori distribution by maximizing the variation lower bound, and establishing a classification prediction model. According to the method, the multi-class Gaussian process is used as a bottom layer classifier model, additive Gaussian noise is introduced, a variation inference method is used for approximating posterior distribution of model hidden variables, and the classifier model containing input noise is optimized, so that fault diagnosis is carried out on the liquid rocket engine, and the fault classification accuracy of the liquid rocket engine containing the input noise is improved.

Description

Technical Field

[0001] The present invention belongs to the field of fault diagnosis, and more specifically, relates to a variational inference learning and diagnosis method for a liquid rocket engine containing a multi-category Gaussian process with input noise. Background Art

[0002] The Gaussian Process (GP) is a common probabilistic modeling tool in machine learning. It provides probabilistic outputs based on a Bayesian framework and is suitable for handling tasks with uncertainty. The uncertainty of a GP algorithm can be reflected in the predicted distribution of the target variable, which may be affected by noise in the model. GP algorithms are typically used to solve regression or simple binary classification problems. When the task involves multiple categories, the single-category Gaussian Process needs to be extended to accommodate multi-category situations.

[0003] Multi-class classification tasks are widespread across various fields, including industry and science. When machine learning algorithms tackle such problems, they typically assume that the data in the dataset is real and noise-free. However, this assumption theoretically ignores the impact of measurement noise on the data. While ignoring noise can offer some advantages in algorithm design and iteration time, if the unaccounted-for noise significantly impacts the generation process of the input data, the prediction results for the multi-class classification problem may be suboptimal. Conversely, incorporating noise in the input data into prior knowledge and combining it with inductive biases can make the algorithm more robust, better able to adapt to the variability and uncertainty of real-world data, and often achieve better prediction results. Therefore, in multi-class classification problems, considering noise and incorporating it into the algorithm design and training process generally yields superior results. Summary of the Invention

[0004] The present invention proposes a variational inference learning and diagnosis method for liquid rocket engines based on a multi-category Gaussian process with input noise. The multi-category Gaussian process is used as the underlying classifier model, additive Gaussian noise is introduced, and the variational inference method is used to approximate the posterior distribution of the model's latent variables. The classifier model with input noise is optimized, and new classification predictions are made based on this.

[0005] The purpose of the present invention is to improve the multi-category Gaussian process classification method, so as to improve the accuracy of multi-category Gaussian process classification with input noise.

[0006] To achieve the above objectives, the present invention provides a variational inference learning method for a multi-category Gaussian process with input noise, comprising the following steps:

[0007] Step 1: Introduce noise and establish a multi-classification Gaussian process model with noise;

[0008] Step 2: Based on Gaussian variational inference, the true Bayesian posterior and the approximate posterior are linked through the KL divergence, the data classification problem is transformed into an approximate posterior problem, and the KL divergence and variational lower bound are defined;

[0009] Step 3: Convert the calculation of KL divergence into an optimization problem of the variational lower bound, and use the small batch data approximation method to optimize the unbiased estimate of the variational lower bound;

[0010] Step 4: Maximize the variational lower bound to obtain an approximate distribution. Use the approximate posterior value to replace the exact posterior value in the category label prediction model. The optimized model predicts the label result based on the new instance data, and uses the approximate posterior instead of the actual posterior to establish the classification prediction model.

[0011] set up is the data matrix of noise observation, and X is the data matrix of noise-free input, with the following relationship: in, is the observed noisy data, x i is the real data without noise, ∈ i is the Gaussian noise added to the real data. Assuming that the noise is independent, then V i is a diagonal noise matrix of dimension d×d. The following likelihood function can be obtained: Observed data So x i is the mean value, V i is the normal distribution with covariance matrix. Specify the noise-free observation data x i With the following prior distribution: Where I is the identity matrix. When s is a large constant, the distribution is similar to the uninformative uniform distribution, so we assume s = 1000.

[0012] Therefore, the joint distribution of all observed variables and latent variables in the model is:

[0013]

[0014] In the formula is the observation data matrix, X=(x1,...,x N ) T is the actual noise-free input matrix, y is the observation label vector, F=(f1,...,f N ) T is the actual noise-free input calculation matrix, U=(u 1 ,...,u C ) T is the calculation matrix of pseudo data points. i Is the input x i The implicit function f at c(·) is the function value vector, c=1,...,C is the number of category labels. c Is the input x i The Gaussian process function value matrix at , f c =(f c (x1),...,f c (x N )) T .u c is the matrix of Gaussian process function values at the pseudo data points.

[0015] Furthermore, in order to solve the optimal parameters of the latent variables, based on Gaussian variational inference, the true Bayesian posterior and the approximate posterior are linked through the KL divergence, the data classification problem is transformed into an approximate problem of the posterior, and the KL divergence and variational lower bound are defined.

[0016] The posterior distribution formula when the observed variables are known can be derived from the joint distribution formula: Solving the posterior distribution requires calculating the marginal likelihood In practice, the marginal likelihood is difficult to solve and its computational cost is extremely high. Therefore, variational inference is used to transform the inference problem into an optimization problem to avoid solving the marginal likelihood.

[0017] Variational inference optimizes parameters by minimizing the distance between the variational approximation q and the true posterior distribution. The distance is calculated using the KL divergence. Let the variational approximation cluster be q(X,F,U), then the KL divergence between it and the true posterior can be written as:

[0018]

[0019] Where, is a constant term. Requiring the minimum value of KL divergence is equivalent to minimizing the values of the first two terms on the right side of the equation. Its negative value is called the variational lower bound, which is recorded as

[0020] The variational approximation q(X,F,U) contains latent variables X,F,U, where X is the actual noise-free input matrix, which is independent of the latent variables F,U. Thus, q(X,F,U)=q(F,U)q(X), where q(F,U)=q(F|U)q(U)=p(f c |u c )q(U), the variational function of q(U) obeys the Gaussian posterior distribution Where c = 1, ..., C, the variational function of q(X) obeys the Gaussian posterior distribution After finishing, we can get:

[0021]

[0022] Finally, substitute the calculated p(X,X,y,F,U) and q(X,F,U) into The simplified variational lower bound is:

[0023]

[0024] Furthermore, the calculation of KL divergence is converted into a variational lower bound The optimization problem is solved by using a small batch data approximation method to optimize the unbiased estimate of the variational lower bound.

[0025] An unbiased estimate of the first expectation of the variational lower bound is obtained in is the number of mini-batch data instances.

[0026] The second term of the variational lower bound is also computed using mini-batch estimation:

[0027]

[0028] The third and fourth terms of the variational lower bound are the KL divergence between the two Gaussian distributions, calculated as follows:

[0029]

[0030] in is f c (·) at point Z c The covariance matrix at m c and S c is the Gaussian posterior distribution The mean and variance of .

[0031]

[0032] Furthermore, by maximizing the variational lower bound, an approximate distribution q is obtained. The approximate posterior value is used to replace the exact posterior value in the classification prediction model. The optimized model is then optimized according to the new instance data x. * Predict its label result y * , using the approximate posterior instead of the actual posterior to establish the following classification prediction model:

[0033]

[0034] Where, Given noisy data The real data x * The posterior distribution of (noise-free) is the normalized product of the prior and the likelihood, so it can be solved in closed form and is Gaussian, that is:

[0035]

[0036] In the formula V * At data point x * The diagonal matrix with Gaussian noise variance at . Because the setting s = 1000 is quite large, in fact

[0037] Using the Monte Carlo method, Extract samples and simplify the calculation of p(y * |x * )Integral in the prediction formula, we can get:

[0038]

[0039] Where S is the number of samples drawn, is the sth x generated by sampling * Sample. The right side of the formula integrates f and u. The integral related to u can be calculated analytically using Gaussian cumulative distribution, and the integral related to f is calculated using the Gauss-Hermite quadrature method. The prediction formula is as follows:

[0040]

[0041] In the formula Φ(·) is a standard Gaussian cumulative probability distribution, and

[0042]

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. The present invention can introduce data noise into the multi-category Gaussian process classification algorithm, adopt variational inference as an optimization method, and approximate the posterior distribution of the model.

[0045] 2. The present invention derives and introduces input noise to optimize the prediction performance of the multi-category Gaussian process classification algorithm model.

[0046] 3. Compared with the traditional noise-free multi-category Gaussian process classification algorithm, the noise-introduced multi-category Gaussian process classification algorithm proposed in the present invention can reduce the classification prediction error, and the negative log-likelihood is much smaller than that of the traditional algorithm, and the degree of fitting to the true posteriori is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Flowchart of the variational inference learning and diagnosis method for liquid rocket engines with multi-category Gaussian processes containing input noise;

[0048] Figure 2The box plots show the average negative log-likelihood (NLL) of the traditional noise-free multi-class Gaussian process classification algorithm (MGP) and the noise-introduced multi-class Gaussian process variational inference classification algorithm (VI-MGP) as the noise variance increases.

[0049] Figure 3 The box plot of the average error of the traditional noise-free MGP classification algorithm and the noise-introduced VI-MGP classification algorithm as the noise variance increases;

[0050] Figure 4 Fault prediction confusion matrix for a liquid rocket engine example. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0052] Example

[0053] This example proposes a variational inference learning method for a multi-class Gaussian process with input noise and applies it to the fault classification and diagnosis of liquid rocket engines. The fault dataset and preprocessing are as follows:

[0054] Liquid rocket engine failures can be categorized by the stage of failure: startup, steady-state, and shutdown. During steady-state operation, failures are primarily controlled by the fluid and mechanical motion of the supply system, and the energy conversion process between the gas generator and the combustion chamber. Major failures in this state include pipeline leaks or blockages, and turbopump system failures.

[0055] The experimental data is derived from steady-state test data of a liquid rocket engine. It includes five common pipeline fault types and 21 associated characteristic attributes. The fault types and characteristic attributes are shown in Tables 1 and 2.

[0056] Table 1 Steady-state fault types of liquid rocket engines

[0057]

[0058] Table 2 Feature attributes and corresponding symbols

[0059]

[0060] The dataset contains monitoring data from five different pipeline fault types over different time periods, totaling 5400 fault data points. 90% of these data points were used for training, and the remaining 10% were used for testing. The fault data were randomly permuted and repeated 100 times for the experiment. Considering that the wide range of values observed during real-world training can affect algorithm convergence, the observed liquid rocket engine attributes were normalized to a standard value with a mean of 0 and a variance of 1. In the experiments, noise with a mean of 0 and variances of 0.1, 0.25, and 0.5 was injected into the original data, forming four sets of comparative experiments alongside the original noise-free data. Training was performed with a batch size of 50 and an upper limit of 1000 iterations. The number of dummy data points was set to 5% of the total data volume.

[0061] The steps for implementing classification prediction are as follows:

[0062] Introduce noise and establish a multi-classification Gaussian process model with noise, set is the data matrix of noise observation, and X is the data matrix of noise-free input, with the following relationship: in, is the observed noisy data, x i is the real data without noise, ∈ i is the Gaussian noise added to the real data. Assuming that the noise is independent, then V i is a diagonal noise matrix of dimension d×d. The following likelihood function can be obtained: Observed data So x i is the mean value, V i is the normal distribution with covariance matrix. Specify the noise-free observation data x i With the following prior distribution: Where I is the identity matrix. When s is a large constant, the distribution is similar to the uninformative uniform distribution, so we assume s = 1000.

[0063] Therefore, the joint distribution of all observed variables and latent variables in the model is:

[0064]

[0065] In the formula is the observation data matrix, X=(x1,...,x N ) T is the actual noise-free input matrix, y is the observation label vector, F=(f1,...,f N ) T is the actual noise-free input calculation matrix, U=(u 1 ,...,u C ) T is the calculation matrix of pseudo data points.i Is the input x i The implicit function f at c (·) is the function value vector, c=1,...,C is the number of category labels. c Is the input x i The Gaussian process function value matrix at , f c =(f c (x1),...,f c (x N )) T .u c is the matrix of Gaussian process function values at the pseudo data points.

[0066] Furthermore, in order to solve the optimal parameters of the latent variables, based on Gaussian variational inference, the true Bayesian posterior and the approximate posterior are linked through KL divergence, the data classification problem is transformed into an approximate problem of the posterior, and the variational lower bound is defined.

[0067] The posterior distribution formula when the observed variables are known can be derived from the joint distribution formula: Solving the posterior distribution requires calculating the marginal likelihood In practice, the marginal likelihood is difficult to solve and its computational cost is extremely high. Therefore, variational inference is used to transform the inference problem into an optimization problem to avoid solving the marginal likelihood.

[0068] Variational inference optimizes parameters by minimizing the distance between the variational approximation q and the true posterior distribution. The distance is calculated using the KL divergence. Let the variational approximation cluster be q(X,F,U), then the KL divergence between it and the true posterior can be written as:

[0069]

[0070] Where, is a constant term. Requiring the minimum value of KL divergence is equivalent to minimizing the values of the first two terms on the right side of the equation. Its negative value is called the variational lower bound, which is recorded as

[0071] The variational approximation q(X,F,U) contains latent variables X,F,U, where X is the actual noise-free input matrix, which is independent of the latent variables F,U. Thus, q(X,F,U)=q(F,U)q(X), where q(F,U)=q(F|U)q(U)=p(f c |u c )q(U), the variational function of q(U) obeys the Gaussian posterior distribution Where c = 1, ..., C, the variational function of q(X) obeys the Gaussian posterior distribution After finishing, we can get:

[0072]

[0073] Finally, substitute the calculated p(X,X,y,F,U) and q(X,F,U) into The simplified variational lower bound is:

[0074]

[0075] Furthermore, the calculation of KL divergence is converted into a variational lower bound We use a small batch data approximation method to optimize the unbiased estimate of the variational lower bound.

[0076] An unbiased estimate of the first expectation of the variational lower bound is obtained in is the number of mini-batch data instances.

[0077] The second term of the variational lower bound is also computed using mini-batch estimation:

[0078]

[0079] The third and fourth terms of the variational lower bound are the KL divergence between the two Gaussian distributions, calculated as follows:

[0080]

[0081] in is f c (·) at point Z c The covariance matrix at m c and S c is the Gaussian posterior distribution The mean and variance of .

[0082]

[0083] Furthermore, by maximizing the variational lower bound, an approximate distribution q is obtained. The approximate posterior value is used to replace the exact posterior value in the category label prediction model. The optimized model is optimized according to the new instance data x * Predict its label result y * , using the approximate posterior instead of the actual posterior to obtain the following prediction formula:

[0084]

[0085] Where, It gives new data The posterior distribution of the actual attribute of is the normalized product of the prior and the likelihood. Due to the special nature of its form, it can be calculated in closed form, and the result has a Gaussian structure:

[0086]

[0087] In the formula V * At data point x * The diagonal matrix with the variance of Gaussian noise at . Because the setting s = 1000 is quite large, in fact

[0088] Using the Monte Carlo method, Extract samples and simplify the calculation of p(y * |x * )Integral in the prediction formula, we can get:

[0089]

[0090] Where S is the number of samples drawn, f s,* =(f 1 (x s,* ),...,f C (x s,* )) T , f cs,* =f c (x s,* ), x s,* is the generated sample. The right side of the formula integrates f and u. The integral related to u can be calculated analytically using Gaussian cumulative distribution, and the integral related to f is calculated using Gauss-Hermite quadrature method. The prediction formula is as follows:

[0091]

[0092] In the formula Φ(·) is a standard Gaussian cumulative probability distribution, and

[0093]

[0094] Figure 2 、 3The boxplots compare the mean error and mean negative log-likelihood (NLL) of the two methods using real data. As the variance of the noise added to the real data increases, the classification error and negative log-likelihood of both methods increase simultaneously. When the noise variance is 0.5, the noise significantly impacts the input data, resulting in an error rate of nearly 15% for both methods. Introducing an excessively high noise variance parameter results in poor model predictions. When no input noise is added, that is, when the variance is 0, the mean error is still lower than that of the traditional MGP method. A reasonable explanation for this is that the real data of liquid rocket engine failures already contains a certain amount of noise, making this algorithm more advantageous when used with this method. When an appropriate variance is introduced into the real data, the mean error of the algorithm with input noise is slightly lower than that of the traditional algorithm, and the mean negative log-likelihood is 32.3% lower than that of the traditional algorithm, with a more concentrated distribution. This indicates that the model with added noise better fits the true posterior distribution of the data.

[0095] Figure 4 This is the confusion matrix for fault classification prediction for a liquid rocket engine example. The model correctly predicts a high percentage of five engine pipeline faults, with calculated accuracy rates of 94.3%, 93.7%, 89.8%, 92.9%, and 92.4% for each type of fault, respectively. These results demonstrate that the proposed variational inference learning method for a multi-class Gaussian process with input noise has excellent classification and prediction performance.

[0096] It should be pointed out that the above is only one of the specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes and substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered within the scope of protection of the present invention.

Claims

1. A variational inference learning and diagnostic method for liquid rocket engines using a multi-class Gaussian process with input noise, characterized by: include: Step 1: Introduce noise and establish a multi-classification Gaussian process model with noise; Step 2: Based on Gaussian variational inference, the true Bayesian posterior and the approximate posterior are linked through the KL divergence, the data classification problem is transformed into an approximate posterior problem, and the KL divergence and variational lower bound are defined; Step 3: Convert the calculation of KL divergence into an optimization problem of the variational lower bound, and use the small batch data approximation method to optimize the unbiased estimate of the variational lower bound; Step 4: Maximize the variational lower bound to obtain an approximate distribution. Use the approximate posterior value to replace the exact posterior value in the category label prediction model. The optimized model predicts the label result based on the new instance data, and uses the approximate posterior instead of the actual posterior to establish the classification prediction model.

2. The variational inference learning and diagnosis method for liquid rocket engines with a multi-class Gaussian process containing input noise according to claim 1 is characterized in that: In step 1: set up is the data matrix of noise observation, and X is the data matrix of noise-free input, with the following relationship: in, is the observed noisy data, x i is the real data without noise, ∈ i is the Gaussian noise added to the real data. Assuming that the noise is independent, then V i is a diagonal noise matrix of dimension d×d. The following likelihood function can be obtained: Observed data So x i is the mean value, V i is the normal distribution with covariance matrix. Specify the noise-free observation data x i With the following prior distribution: Where I is the identity matrix. When s is a large constant, the distribution is similar to the uninformative uniform distribution, so we assume s = 1000. Therefore, the joint distribution of all observed variables and latent variables in the model is: In the formula is the observation data matrix, X=(x1,...,x N ) T is the actual noise-free input matrix, y is the observation label vector, F=(f1,...,f N ) T is the actual noise-free input calculation matrix, U=(u 1 ,...,u C ) T is the calculation matrix of pseudo data points. i Is the input x i The implicit function f at c (·) is the function value vector, c=1,...,C is the number of category labels. c Is the input x i The Gaussian process function value matrix at , f c =(f c (x1),...,f c (x N )) T .u c is the matrix of Gaussian process function values at the pseudo data points.

3. The variational inference learning and diagnosis method for liquid rocket engines with a multi-class Gaussian process containing input noise according to claim 2 is characterized in that: In step 2: In order to solve the optimal parameters of the latent variables, based on Gaussian variational inference, the true Bayesian posterior and the approximate posterior are linked through the KL divergence, the data classification problem is transformed into an approximate posterior problem, and the KL divergence and variational lower bound are defined. The joint distribution formula in step 1 can be used to derive the posterior distribution formula when the observed variables are known: Solving the posterior distribution requires calculating the marginal likelihood In practice, the marginal likelihood is difficult to solve and its computational cost is extremely high. Therefore, variational inference is used to transform the inference problem into an optimization problem to avoid solving the marginal likelihood. Variational inference optimizes parameters by minimizing the distance between the variational approximation q and the true posterior distribution. The distance is calculated using the KL divergence. Let the variational approximation cluster be q(X,F,U), then the KL divergence between it and the true posterior can be written as: Where, is a constant term. Requiring the minimum value of KL divergence is equivalent to minimizing the values of the first two terms on the right side of the equation. Its negative value is called the variational lower bound, which is recorded as The variational approximation q(X,F,U) contains latent variables X,F,U, where X is the actual noise-free input matrix, which is independent of the latent variables F,U. Thus, q(X,F,U)=q(F,U)q(X), where q(F,U)=q(F|U)q(U)=p(f c |u c )q(U), the variational function of q(U) obeys the Gaussian posterior distribution Where c = 1, ..., C, the variational function of q(X) obeys the Gaussian posterior distribution After finishing, we can get: Finally, substitute the calculated p(X,X,y,F,U) and q(X,F,U) into The simplified variational lower bound is:

4. The variational inference learning and diagnosis method for liquid rocket engines with a multi-category Gaussian process containing input noise according to claim 3 is characterized in that: In step 3: Convert the calculation of KL divergence to a variational lower bound The optimization problem is solved by using a small batch data approximation method to optimize the unbiased estimate of the variational lower bound. An unbiased estimate of the first expectation of the variational lower bound is obtained in is the number of mini-batch data instances. The second term of the variational lower bound is also computed using mini-batch estimation: The third and fourth terms of the variational lower bound are the KL divergence between the two Gaussian distributions, calculated as follows: in is f c (·) at point Z c The covariance matrix at m c and S c is the Gaussian posterior distribution The mean and variance of .

5. The variational inference learning and diagnosis method for liquid rocket engines with multi-category Gaussian processes containing input noise according to claim 4 is characterized in that: In step 4: By maximizing the variational lower bound, an approximate distribution q is obtained. The approximate posterior value is used to replace the exact posterior value in the classification prediction model. The optimized model is then optimized based on the new instance data x. * Predict its label result y * , using the approximate posterior instead of the actual posterior to establish the following classification prediction model: Where, Given noisy data The real data x * The posterior distribution of (noise-free) is the normalized product of the prior and the likelihood, so it can be solved in closed form and is Gaussian, that is: In the formula V * At data point x * The diagonal matrix with Gaussian noise variance at . Because the setting s = 1000 is quite large, in fact Using the Monte Carlo method, Extract samples and simplify the calculation of p(y * |x * )Integral in the prediction formula, we can get: Where S is the number of samples drawn, is the sth x generated by sampling * Sample. The right side of the formula integrates f and u. The integral related to u can be calculated analytically using Gaussian cumulative distribution, and the integral related to f is calculated using the Gauss-Hermite quadrature method. The prediction formula is as follows: In the formula Φ(·) is a standard Gaussian cumulative probability distribution, and

Images