A health state prediction method based on adaptive bayesian deep learning

Abstract

This invention relates to a health status prediction method based on adaptive Bayesian deep learning under multi-source uncertainty. It introduces adaptive model-free techniques into the Bayesian deep learning framework to fully leverage the capabilities of the prediction model. First, a model-free dropout method is employed to quantify cognitive uncertainty. This method can automatically learn the dropout rate and distribution type, better capturing highly nonlinear degradation characteristics. Second, an arbitrary polynomial chaos expansion (aPc) method is used to quantify stochastic uncertainty. This method avoids introducing additional subjectivity from limited samples or sparse information into the assumed distribution. Finally, a health status prediction framework based on adaptive Bayesian deep learning is proposed. Variational inference is used to construct the network loss function and conduct training, unifying the quantification of cognitive uncertainty and stochastic uncertainty in a model-free manner, thereby simultaneously performing mean and interval predictions of health status.

Description

Technical Field

[0001] This invention provides a health status prediction method based on an adaptive Bayesian deep learning framework. It is a health status prediction method that considers the quantification of uncertainty and can obtain point and interval estimates of the system health status. It belongs to the field of fault prediction and health management (PHM) technology. Background Technology

[0002] The exponential increase in hardware, software, personnel, and organizational characteristics of modern complex industrial systems has greatly increased system complexity, which in turn increases the likelihood of system failure. Failures during long-term operation often lead to unacceptable losses of personnel and property. Prognostics and health management (PHM) technology can ensure the safe and stable operation of complex industrial systems. In PHM, State of Health (SOH) prediction is the foundation of health management, requiring accurate and reliable input for health management decisions.

[0003] With the development of sensor and computer technologies, modern industrial systems increasingly utilize network-connected devices. Various sensor suites generate massive amounts of multidimensional data, leading to the widespread application of data-driven methods, such as deep learning, in health status prediction. However, deep learning models, due to their black-box nature, require evaluation for reliability and effectiveness, as their predictions are affected by data noise and model errors. Existing deep learning-based health status prediction models mostly provide point estimates, ignoring the impact of uncertainty; however, such predictions are often unreliable for inference and decision-making.

[0004] Deep learning-based health status prediction requires quantifying two types of uncertainty: cognitive uncertainty caused by the model's reasoning itself and stochastic uncertainty caused by data noise. Cognitive uncertainty stems from insufficient understanding of neural network architecture, model parameters, and hyperparameters due to limited data information; this can be eliminated with sufficient data containing rich information. Stochastic uncertainty is an inherent property of data and cannot be eliminated even with more data. Both types of uncertainty in deep learning require a unified architecture for quantification and comprehensive processing to improve the reliability of prediction results. However, almost all current methods quantify uncertainty in deep learning by introducing model-based hypotheses and priors, which limits the modeling and predictive capabilities of these methods.

[0005] In view of this, the present invention proposes a model-free adaptive Bayesian deep learning framework that comprehensively considers random uncertainty and cognitive uncertainty for health status prediction, and realizes the probability prediction of health status by incorporating multi-source uncertainty, which can provide reliable decision support for health management. Summary of the Invention

[0006] This invention presents a health status prediction method based on adaptive Bayesian deep learning. It introduces adaptive model-free techniques into the Bayesian deep learning framework to fully leverage the predictive model's capabilities. First, a model-free dropout method is employed to quantify cognitive uncertainty. This method automatically learns the dropout rate and distribution type, better capturing highly nonlinear degradation features. Second, an arbitrary polynomial chaos expansion (aPc) method is used to quantify stochastic uncertainty. This method avoids introducing additional subjectivity from limited samples or sparse information into the assumed distribution. Finally, a health status prediction framework based on adaptive Bayesian deep learning is proposed. Variational inference is used to construct the network loss function and conduct training, unifying the quantification of cognitive and stochastic uncertainties in a model-free manner. The method flow described in this invention is as follows: Figure 1 As shown, it specifically includes the following:

[0007] Step 1: Targeting the dataset with multidimensional input features and one-dimensional health status representation , Given the time series sample length, the dataset is divided into training, test, and validation sets, and preprocessed with normalization and other methods.

[0008] Step 2: For a A deep neural network with layers is defined as follows: layer( )have The _ neuron, the _ ... neurons ( The corresponding dropout mask parameter Related intermediate hidden variables and mapping functions;

[0009] Step 3: Adjust dropout mask parameters using intermediate hidden variables and mapping functions. Model it so that its distribution type can adapt to change as the data changes;

[0010] Step 4: Prior distribution of parameters for the dropout mask Modeling is performed, and parameter estimation is carried out using the maximum likelihood estimation method;

[0011] Step 5: Calculate the network weight parameters after adding the dropout mask. Constructing variational distributions in variational inference ;

[0012] Step 6: Based on the dataset Construct a multivariate polynomial output model;

[0013] Step 7: For the polynomial coefficients in the output model The solution is obtained using the Galerkin projection method or the random response surface method; for the multidimensional orthogonal basis functions in the output model Construct orthogonal basis functions of arbitrary distributions using the aPc method. ;

[0014] Step 8: Train the aPc model using the training set and establish a mapping relationship between multidimensional input features and health status representations considering random uncertainty;

[0015] Step 9: Perform data augmentation on the test set, input the augmented data into the trained aPc model, and predict the probabilistic information features of the health status representation.

[0016] Step 10: Construct a prediction distribution model under Bayesian deep learning. Based on the variational distribution in cognitive uncertainty quantification and the distribution information characteristics in stochastic uncertainty quantification, construct the network loss function using variational inference methods.

[0017] Step 11: Set network hyperparameters, train a Bayesian deep learning model using dropout and L2 regularization based on the network loss function, and use stochastic gradient descent to adjust the parameter set during training. Optimization is performed to obtain the optimal model weights, and the corresponding strain distribution can be used to approximate the posterior distribution. Dropout is not disabled during the testing phase to preserve the probabilistic features of the healthy state.

[0018] Step 12: Use the Monte Carlo sampling method to output the results of the mean prediction and interval prediction of health status, decompose the two types of uncertainty, and characterize the total uncertainty, random uncertainty and cognitive uncertainty.

[0019] In step 2, the first The intermediate hidden variables of the layer are defined as follows It follows an intermediate distribution intermediate hidden variables and They can be connected through a mapping function. Related:

[0020] (1)

[0021] In the formula, It is an arbitrary function that satisfies the following conditions: 1) its range is [0,1]; 2) it is monotonically and differentiable within its domain, such as the Sigmoid function. It satisfies the following: 1) It can be converted into a standard distribution with differentiable parameters; 2) It is an arbitrary distribution that is easy to sample, such as Gaussian distribution, Weibull distribution, logistic distribution, etc.

[0022] In step 3, based on the definitions of intermediate latent variables and mapping functions, It can satisfy Any form:

[0023] (2)

[0024] In the formula, It is the inverse function of the mapping function.

[0025] by and The effectiveness of this invention is illustrated using the Sigmoid function as an example. The distribution can be represented as:

[0026] (3)

[0027] Distribution It can be adjusted and Adaptive approximation of all types of dropout distributions.

[0028] In step 4, the parameters of the variational distribution and It can learn and infer from the data, adaptively adjusting the dropout rate. The prior distribution of the parameters can be defined as:

[0029] (4)

[0030] In the formula, It is the first Multivariable Gaussian hidden state parameters of the layer.

[0031] Assumption and They follow Gaussian and inverse gamma distributions, respectively. and The maximum likelihood estimate can be approximated by a multilayer perceptron with Gaussian output:

[0032] (5)

[0033] (6)

[0034] In the formula, and It is the first Layer parameters Corresponding weights and biases, and It is the first Layer parameters Corresponding weights and biases.

[0035] Multivariable Gaussian Hidden State Parameters By maximizing Reasoning:

[0036] (7)

[0037] In step 5, the network weight parameters after adding the dropout mask... It can be represented as:

[0038] (8)

[0039] Definition of the first Layer parameter set for:

[0040] (9)

[0041] The variational distribution in variational reasoning is:

[0042] (10)

[0043] In step 6, the multivariate polynomial model can be represented as:

[0044] (11)

[0045] In the formula, These are polynomial coefficients. It is the number of coefficients, and the truncation order. and feature dimensions related, They are multidimensional orthogonal basis functions, under the condition that the input features are independent. It can be represented as the product of one-dimensional orthogonal basis functions. :

[0046] (12)

[0047] In the formula, It is a multivariate index. Random variable corresponding to order The polynomial is:

[0048] (13)

[0049] In the formula, yes Medium coefficient.

[0050] In step 9, the distribution model of the health status representation can be represented by a two-parameter function, namely... .

[0051] In step 10, the predicted distribution model can be expressed as:

[0052] (14)

[0053] According to Bayes' theorem, the posterior distribution It can be represented as:

[0054] (15)

[0055] Using a variational distribution Remove approximate posterior distribution ,calculate and KL divergence between:

[0056] (16)

[0057] In the formula, the third term on the right is related to the model parameters. Irrelevant, the lower bound of evidence is defined according to the first and second items:

[0058] (17)

[0059] In the formula, the first term can be approximated by Monte Carlo integration, and the second term can be regarded as the L2 regularization term of the model parameters, which is minimized. and The objective of the KL divergence can be transformed into maximizing the ELBO, and the network's loss function can then be expressed as:

[0060] (18)

[0061] In the formula, It is the model decay coefficient, variational distribution. Determined according to equation (10), Based on the quantification of aPc random uncertainty, the health status characterization quantity follows a two-parameter function. Taking the normal distribution as an example, the loss function can be expressed as:

[0062] (19)

[0063] In step 11, given the dataset Under the conditions, The predicted distribution under sub-sampling can be represented as:

[0064] (20)

[0065] In the formula, , and The number of samples.

[0066] In step 12, the health status representation for each sample can be expressed as: ,but The mean predicted under each sampling can be expressed as:

[0067] (twenty one)

[0068] The variance, which characterizes the total uncertainty, can be expressed as:

[0069] (twenty two)

[0070] The variance, which characterizes random uncertainty, can be expressed as:

[0071] (twenty three)

[0072] The variance representing cognitive uncertainty can be expressed as:

[0073] (twenty four)

[0074] This invention is a health status prediction method based on adaptive Bayesian deep learning, and its advantages and effects are as follows:

[0075] 1. This invention employs a model-free form of dropout technique to quantify cognitive uncertainty, which can automatically learn the dropout rate and distribution type, thereby better capturing highly nonlinear degradation characteristics;

[0076] 2. This invention uses the aPc method to quantify random uncertainty and realize the propagation of uncertainty from input to output, thus avoiding the introduction of additional subjectivity from limited samples or sparse information into the assumed distribution;

[0077] 3. This invention constructs a health status prediction framework based on adaptive Bayesian deep learning, which unifies and quantifies cognitive uncertainty and stochastic uncertainty in a model-free manner, and can provide probabilistic prediction results of health status, providing reliable decision support for health management. Attached Figure Description

[0078] Figure 1 This is a flowchart of the method described in this invention;

[0079] Figure 2 The results of health status prediction by different methods are shown, where (a) represents the traditional LSTM NN method (without considering uncertainty quantification); (b) represents the homoscedastic Bayesian LSTM NN method, which uses traditional dropout to capture cognitive uncertainty; (c) represents the heteroscedastic Bayesian LSTM NN method, which uses traditional dropout to capture cognitive uncertainty and treats the output as a Gaussian distribution to capture random uncertainty; and (d) represents the method proposed in this invention.

[0080] The labels and symbols in the diagram are explained as follows:

[0081] The dropout mask represents the temporary dropout mask;

[0082] model-free means no model;

[0083] aPc represents an arbitrary chaotic polynomial expansion. Detailed Implementation

[0084] Lithium-ion batteries, as complex electrochemical systems, are widely used in aerospace, transportation, and other industries. This invention utilizes the lithium-ion battery degradation dataset from the University of Maryland's Center for Advanced Life Cycle Engineering (CALCE), commonly used in health status prediction, to illustrate the implementation process. Six lithium-ion batteries (numbered CS33-CS38) were used with two different discharge currents to reflect the uncertain charging and discharging environment in actual lithium-ion battery operation. First, a constant current mode of 0.55A was used for charging until the voltage reached 4.2V. Then, constant voltage charging was maintained until the charging current dropped below 0.05A. Afterward, discharge was carried out at a constant current of 0.55A or 1.1A (0.55A for CS33 and CS34, and 1.1A for CS35-CS38) until the lithium-ion battery voltage dropped to 2.7V. This dataset records the voltage, current, capacity, and other characteristics of each charge-discharge cycle, with capacity considered as the health status characteristic of step 1. For voltage and current, degradation-related physical quantities can be extracted, and constant current charging time and constant voltage drop time are selected as input features for step 1. The equal voltage drop time corresponds to a discharge voltage of 3.8V to 3.4V. The state of health (SOH) of a lithium battery is defined as follows:

[0085] (25)

[0086] In the formula, It is the capacity corresponding to the current loop. This is the rated capacity.

[0087] Four indicators were selected to measure the effectiveness of health status prediction, including root mean square error (RMSE), coefficient of determination (R²), and other indicators. 2 The primary evaluation metric is the predictive performance of the mean; the Predictive Interval Coverage (PICP) and Predictive Interval Average Bandwidth (PINAW) primarily evaluate the predictive performance of the interval. The calculation methods for these four metrics are as follows:

[0088] (1) Root Mean Square Error (RMSE)

[0089] (26)

[0090] (2) Coefficient of determination (R) 2 )

[0091] (27)

[0092] (3) Predicted Interval Coverage (PICP)

[0093] (28)

[0094] (4) Prediction Interval Average Bandwidth (PINAW)

[0095] (29)

[0096] In the formula, This is the actual capacity. It is the maximum capacity. It is the minimum capacity. It is the average capacity. It is the predicted capacity. It is the upper confidence bound of the prediction. It is the lower confidence bound of the prediction.

[0097] The effectiveness of this invention was verified by using a Long Short-Term Memory Neural Network (LSTMNN), a commonly used model in prediction problems, as the benchmark model. Three other methods were selected to compare the prediction results of the proposed method: M1 represents the traditional LSTMNN method (without considering uncertainty quantification); M2 represents the homoscedastic Bayesian LSTMNN method, which uses traditional dropout to capture cognitive uncertainty; M3 represents the heteroscedastic Bayesian LSTMNN method, which uses traditional dropout to capture cognitive uncertainty and treats the output as a Gaussian distribution to capture random uncertainty; M4 represents the method proposed in this invention. Life cycle data from CS34 to CS38 were used as the training set, and the last 300 data sets in the training set were designated as the test set. Life cycle data from CS33 were used as the validation set. Prediction started from the 100th cycle, and one-step prediction of health status was performed using the four methods. Table 1 shows the model parameter settings for the prediction methods. The four methods were set with the same hyperparameters such as the number of hidden layers, batch size, number of layers, number of iterations, and learning rate.

[0098] Table 1 Prediction Method Model Parameter Settings

[0099] parameter numerical values Number of hidden layers 128,64 number of floors 2 Batch size 64 Iteration rounds 100 Initial learning rate 0.005

[0100] The SGD optimizer is used to find the weights and biases of all layers. A dynamic learning rate is set, which decays by a factor of 10 every 50 generations. For each method, the average of 5 trials is taken as the prediction result, and the confidence level of the interval prediction is set to 95%. Figure 2 The prediction results of four methods are shown. It is clear from the figure that the method proposed in this invention is the best among all methods, both for mean prediction and interval prediction. Adaptively quantifying uncertainty can obtain effective confidence interval prediction results, while also providing the generalization ability of the deep learning model, thereby improving the accuracy of mean prediction.

[0101] Table 2 shows the RMSE, R... 2 The results of PICP and PINAW are shown in Table 2. As can be seen from Table 2, the method (M4) proposed in this invention has the smallest RMSE and the largest R... 2The highest PICP indicates good predictive performance for both mean and interval prediction. Compared to the traditional LSTM NN method (M1), M4 not only obtains interval estimation prediction results but also further improves the prediction accuracy of point estimation. The Bayesian prior increases the generalization ability of the deep learning model, proving that effectively quantifying uncertainty is beneficial for both mean and interval prediction. Compared to the homoscedastic Bayesian LSTM NN method (M2), the heteroscedastic Bayesian LSTM NN method (M3) has a smaller RMSE and R0. 2 Compared to M3, M2 has a larger PINAW, indicating a narrower confidence interval. However, a smaller PICP suggests that the interval does not adequately cover the true values, resulting in poor mean prediction. In prediction problems, random uncertainty needs to be reliably quantified because the dependency between features and labels has inherent randomness, often manifesting as heteroscedasticity. Compared to M3, M4 performs better in both mean and interval prediction. By adaptively quantifying random and cognitive uncertainty to reduce limitations on the modeling capabilities of the prediction model, the accuracy of mean and interval predictions for health states can be improved, thus providing a basis for decision support in state-based health management.

[0102] Table 2 Comparison of Prediction Results

[0103] Prediction methods RMSE <![CDATA[R 2 ]]> PICP PINAW M1 0.039 0.922 / / M2 0.042 0.911 0.183 0.129 M3 0.032 0.947 0.031 0.695 M4 0.027 0.964 0.764 0.138

Claims

1. A health status prediction method based on adaptive Bayesian deep learning, characterized in that, Includes the following steps: Step 1: Targeting the dataset with multidimensional input features and one-dimensional health status representation For lithium battery health status prediction, the multidimensional input features are constant current charging time and equal voltage drop time feature data, and the one-dimensional health status characterization is the lithium battery health status. The dataset is divided into training, testing, and validation sets based on the time series sample length, and then normalized. Step 2: For a A deep neural network with layers is defined as follows: layer( )have The _ neuron, the _ ... neurons ( The corresponding dropout mask parameter Related intermediate hidden variables and mapping functions; Step 3: Adjust dropout mask parameters using intermediate hidden variables and mapping functions. Model it so that its distribution type changes adaptively with the data; Step 4: Prior distribution of parameters for the dropout mask Modeling is performed, and parameter estimation is carried out using the maximum likelihood estimation method; Step 5: Calculate the network weight parameters after adding the dropout mask. Constructing variational distributions in variational inference ; Step 6: Based on the dataset Construct a multivariate polynomial output model; Step 7: For the polynomial coefficients in the output model The solution is obtained using the Galerkin projection method or the random response surface method; for the multidimensional orthogonal basis functions in the output model , for For 3D random variables, construct orthogonal basis functions of arbitrary distribution using the arbitrary polynomial chaotic expansion method. , It is the order; Step 8: Train the aPc model using the training set and establish a mapping relationship between multidimensional input features and health status representations considering random uncertainty; Step 9: Perform data augmentation on the test set, input the augmented data into the trained aPc model, and predict the probabilistic information features of the health status representation. Step 10: Construct a prediction distribution model under Bayesian deep learning. Based on the variational distribution in cognitive uncertainty quantification and the distribution information characteristics in stochastic uncertainty quantification, construct the network loss function using variational inference methods. Step 11: Set network hyperparameters, train a Bayesian deep learning model using dropout and L2 regularization based on the network loss function, and use stochastic gradient descent to adjust the parameter set during training. Optimize to obtain the optimal model weights, and use the corresponding sub-distribution to approximate the posterior distribution. Do not turn off dropout during the testing phase to retain the probabilistic features of the healthy state. Step 12: Use the Monte Carlo sampling method to output the results of the mean prediction and interval prediction of health status, decompose the two types of uncertainty, and characterize the total uncertainty, random uncertainty and cognitive uncertainty.

2. The health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, In step 2 The intermediate hidden variables of the layer are defined as follows It follows an intermediate distribution intermediate hidden variables and They are connected through a mapping function Related: (1) In the formula, It is an arbitrary function that satisfies the following conditions: 1) its range is [0,1]; 2) it is monotonically and differentiable within its domain. It satisfies the following: 1) It has the ability to be converted into a standard distribution with differentiable parameters; 2) It is an arbitrary distribution that is easy to sample.

3. The health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, In step 3 It is to satisfy Any form: (2) In the formula, It is the inverse function of the mapping function, when obey , When it is the Sigmoid function, The distribution is represented as: (3) Distribution By adjusting and Adaptive approximation of all types of dropout distributions.

4. The health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, The prior distribution of the parameters in step 4 is defined as follows: (4) In the formula, It is the first Multivariable Gaussian hidden state parameters of the layer and They follow Gaussian and inverse gamma distributions, respectively. and The maximum likelihood estimation is approximated by a multilayer perceptron with Gaussian output: (5) (6) In the formula, and It is the first Layer parameters Corresponding weights and biases, and It is the first Layer parameters Corresponding weights and biases, By maximizing Reasoning: (7) In the formula, It is a multivariable Gaussian hidden state parameter.

5. The health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, The network weight parameters after adding the dropout mask in step 5 Represented as: (8) In the formula, It is the first The original network parameters of the first layer are defined. Layer parameter set for: (9) Then variational reasoning for: (10) In the formula, It is a variational distribution.

6. The health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, The multivariate polynomial model in step 6 is expressed as follows: (11) In the formula, These are polynomial coefficients. It is the number of coefficients, and the truncation order. and feature dimensions related, They are multidimensional orthogonal basis functions, under the condition that the input features are independent. Represented as the product of one-dimensional orthogonal basis functions : (12) In the formula, It is a multivariate index. Random variable corresponding to order The polynomial is: (13) In the formula, yes Medium coefficient.

7. The health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, The distribution model of the health status characterization in step 9 is represented by a two-parameter function, namely... .

8. The health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, The distribution model predicted in step 10 is expressed as follows: (14) According to Bayes' theorem, the posterior distribution Represented as: (15) Using a variational distribution Remove approximate posterior distribution ,calculate and KL divergence between: (16) In the formula, With model parameters Irrelevant, according to and Define the lower bound of evidence: (17) In the formula, Approximation using Monte Carlo integration. Treat it as an L2 regularization term for the model parameters, minimize and If the objective of calculating the KL divergence is transformed into maximizing the ELBO, then the network's loss function is expressed as: (18) In the formula, It is the model decay coefficient, variational distribution. Determined according to equation (10), Based on the quantification of aPc random uncertainty, the health status characterization quantity follows a two-parameter function. Taking the normal distribution as an example, the loss function is expressed as follows: (19)。 9. A health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, In step 11 The predicted distribution under sub-sampling is represented as follows: (20) In the formula, , and The number of samples.

10. A health status prediction method based on adaptive Bayesian deep learning according to claim 1, characterized in that, In step 12 The mean of the predictions under each sampling is expressed as: (21) Represented as: (22) Represented as: (23) Represented as: (24) In the formula, It is the variance that characterizes the total uncertainty. It is the variance that characterizes random uncertainty. It is the variance that characterizes cognitive uncertainty.

Images