Engine residual life prediction method based on deep survival analysis

Abstract

The invention discloses an engine residual life prediction method based on deep survival analysis, and belongs to the technical field of industrial equipment monitoring and equipment health prediction management. According to the method, an engine residual life prediction model based on deep survival analysis is constructed, a nonlinear relation between multi-source sensor signals and engine residual life distribution is modeled, and real-time online prediction of the engine RUL is achieved through a survival analysis method for the complex situation that part of signals are deleted or an RUL label is lacked. According to the method, the deleted data can be effectively processed, potential data information is fully mined through dynamic optimization of the prediction process, the data utilization rate is maximized, accurate RUL prediction and quantification of uncertainty of the RUL prediction are realized based on Weibull distribution modeling, the life distribution of the engine can be accurately predicted, and the prediction precision of the residual life of the engine is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial equipment monitoring and equipment health prediction management, and relates to an online prediction technology for the remaining service life of engine equipment, and specifically to an online prediction method for the remaining service life of an engine based on a Bayesian deep survival analysis network. By constructing a Bayesian deep survival analysis network model, signal fusion is performed on multi-source sensor degradation signals with right censoring, and the remaining service life of the engine is predicted online. Background Art

[0002] In engineering applications, the monitoring system based on the equipment operating status can monitor the operating signals closely related to the health status of the equipment in real time, and accurately predict the remaining life by establishing a degradation model to determine the best maintenance time, showing great potential in preventing equipment failures and reducing maintenance costs. With the rapid development of sensor technology and information technology, the degradation signals collected during the operation of the equipment can be transmitted to the cloud server in real time through the wireless communication network for in-depth analysis. The convenience of data acquisition makes the data-driven approach easier to implement, and at the same time, provides a data basis for real-time predictive maintenance of equipment. Predicting the remaining useful life (RUL) of a device based on its current health status specifically refers to predicting the time when a system or component fails or no longer performs its expected function. Existing RUL prediction methods are usually based on the degradation signal of the device and perform regression analysis on multi-source time series data. However, in practical applications, there is often a lack of complete degradation signal data during the operation of the device. More devices are in service and their exact failure time is unknown. We can regard this signal as right-censored data. Such devices can only obtain partial degradation signals and have no remaining useful life labels, which makes accurate prediction of remaining useful life complicated. We call this type of data "time-to-event" (TTE) data, which is used to describe the duration from the starting point to the occurrence of an event (such as equipment failure or censoring). This type of data is widely used in multiple engineering and manufacturing scenarios, such as medical, aviation, and railways. According to statistics, the censored data often exceeds 95%. Although the widespread application of sensor equipment has improved data availability and is conducive to the fusion of TTE data and high-dimensional degradation signals, existing RUL prediction methods only rely on complete data for modeling, often ignoring the important information contained in the missing signals and making it difficult to effectively process the missing data that lacks regression labels, resulting in biased results. This poses a major challenge to the accurate prediction of remaining useful life. Therefore, more effective methods are urgently needed to make full use of the missing data to improve the accuracy of the prediction. Summary of the invention

[0003] In order to overcome the deficiencies of the above-mentioned prior art, the present invention proposes an online prediction method for the remaining useful life of an engine based on a Bayesian deep survival analysis network. The present invention constructs an engine remaining useful life prediction model (Bayesian LSTM-SURV) based on deep survival analysis, the purpose of which is to establish a nonlinear relationship between multi-source sensor signals with right censoring and the engine RUL, and to achieve real-time online prediction of the engine RUL through a survival analysis method for complex situations where some signals are censored or lack remaining useful life (RUL) labels.

[0004] Specifically, the online prediction model for the remaining useful life of an engine proposed in the present invention transfers the information of the training data to the test data through the Bayesian method, making full use of the degradation information of the censored data and the complete data, and taking into account the uncertainty of the life, adopts the Weibull distribution to model the life distribution, and innovatively designs a loss function based on the survival analysis theory to capture the degradation information contained in the TTE data, thereby establishing a nonlinear relationship between the multidimensional time series degradation signal and the RUL. On the one hand, this model overcomes the RUL prediction modeling problems caused by signal censoring and label missing, fully exploits the degradation information contained in the censored data, and significantly improves the accuracy of the prediction; on the other hand, it enhances the ability to quantify uncertainty, providing a more robust solution for real-time RUL prediction.

[0005] The method of the present invention includes two stages: (a) in the offline stage, the degradation signal of multiple sensors is modeled based on the Bayesian mixed effect model and the Bayesian LSTM-SURV model is trained to construct a nonlinear relationship between the degradation signal and the remaining life; (b) in the online stage, the degradation signal is dynamically updated by Bayesian, and the RUL of the engine is predicted in real time based on the trained Bayesian LSTM-SURV model. Specifically, in the offline stage, first, the degradation signal of the engine is fitted by the Bayesian mixed effect model to obtain the model parameters to characterize the degradation trend of the signal; secondly, a Bayesian LSTM-SURV model is established, and the model input is a complete signal with a label and a censored signal without a label. The output of the model does not choose to focus on the specific value of RUL, but considers that the life of the engine follows the Weibull distribution, thereby capturing the probability characteristics of the unit degradation process changing over time, such as the survival probability. In addition, this work introduces a survival analysis method to effectively process TTE data. TTE data includes complete degradation data of faulty equipment and censored data of in-service equipment. Traditional methods often cannot fully utilize the potential information contained in censored data, resulting in limited model prediction performance. For this reason, the present invention innovatively proposes a loss function of the Bayesian LSTM-SURV model in combination with the survival analysis theory, so that the model can not only utilize complete data, but also fully mine the degradation information contained in censored data, thereby improving the prediction accuracy of the model. In the online stage, the present invention proposes a signal update framework based on the Bayesian method to characterize the degradation signal, remove noise, and migrate the features of the training data to the test data to achieve signal update of in-service engines. Subsequently, the Bayesian LSTM-SURV model uses the updated degradation signal for real-time RUL prediction. In summary, the signal update framework based on the Bayesian method proposed in the present invention not only dynamically updates the signal, but also quantifies the uncertainty in the degradation process, thereby significantly improving the accuracy of the remaining service life prediction, reducing the prediction bias, and improving the reliability and generalization ability of the model in practical applications.

[0006] Based on the Bayesian method, the present invention combines the survival analysis theory and deep neural network to propose a Bayesian LSTM-SURV model, assuming that the engine life obeys the Weibull distribution, and constructs a nonlinear relationship between the multi-sensor degradation signal and the engine RUL. In the offline stage, the degradation signal is first fitted by the Bayesian mixed effect model, the prior distribution followed by the parameters is estimated, and the Bayesian LSTM-SURV model is designed in combination with the survival analysis, so that it can predict the life distribution using complete data and missing data. In the online stage, the degradation signal is modeled and denoised based on the Bayesian method, and the features of the training data are transferred to the test data. By updating the posterior distribution of the signal in real time, accurate prediction of the RUL is achieved.

[0007] The technical solution provided by the present invention is as follows:

[0008] The engine remaining life prediction method based on deep survival analysis includes the following steps:

[0009] 1) Aircraft engine service signal collection and data representation: Collect the data Y of the J-dimensional degradation signal of I engine i , for each engine i, record its event occurrence time T i and the indicator variable d i , where T i represents the engine failure occurrence time or censoring time, d i indicates that the engine is faulty (1) or missing (0), then the complete observation data O of the ith engine is i = {Y i ,T i ,d i}.

[0010] 2) Constructing a Bayesian mixed effects model to model multivariate degradation signals:

[0011] 21) Use the Bayesian mixed effect model to characterize the engine degradation process and denoise the training set data;

[0012] 22) A two-stage approach is used to estimate model parameters.

[0013] 3) Constructing engine remaining life prediction model and model training based on deep survival analysis:

[0014] The engine remaining life prediction model based on deep survival analysis includes an input layer, a deep survival analysis (LSTM-SURV) network layer and an output layer.

[0015] 31) Model input: Multi-sensor degradation signals are used as the input of the model, and sliding window technology is used to process multi-sensor degradation signals (time series). In this process, the time series batches are normalized to a preset fixed length L to ensure that the multi-sensor degradation signal data maintains the same length L;

[0016] 32) Deep survival analysis network layer: This layer combines the long short-term memory network layer, the random dropout layer and two fully connected layers to learn the degradation information represented by the multi-sensor degradation signals. Among them, the long short-term memory network layer can process time series data, extract effective features from the signals of multiple sensors, and capture the time dependence in the degradation process; the present invention also designs a new loss function based on the survival analysis theory, so that the long short-term memory network layer can fully learn the degradation information contained in the complete data and the missing data; the random dropout layer prevents overfitting by randomly discarding neurons, and the fully connected layer is used to further fuse and process the extracted features;

[0017] 33) Model output: Assuming that the life of all units follows the Weibull distribution, the output of the model is set to the Weibull distribution parameters λ and k that the engine life obeys.

[0018] 34) Offline model training:

[0019] In the offline stage, based on the constructed Bayesian LSTM-SURV model, the denoised training set data in step 2) is used for training to fully learn the degradation information contained in the complete data and the missing data, achieve accurate prediction of RUL, and update the model parameters of the neural network.

[0020] 4) Online parameter update and remaining service life prediction:

[0021] After the model training parameters are obtained by grid search in the offline stage, the remaining service life of the in-service engine i can be predicted. * , by combining historical data with individual information, that is, using the Bayesian mixed effect model learned from the training data set, taking the model fitting parameters as prior information, and combining the sensor degradation signal of the in-service engine as observation information, the posterior distribution of the parameters is calculated through the Bayesian formula to update the Bayesian mixed effect model parameters, thereby realizing information migration from training data to test data. The update process is based on the Bayesian method, taking the Bayesian mixed effect model learned in the training phase as prior information, combining the sensor observation data (likelihood function) of the in-service engine, and updating the posterior distribution of the model parameters through the Bayesian formula. Specifically, based on the Bayesian method, the posterior distribution of sensor j of engine i is The mean is and the variance is The multivariate normal distribution of the in-service engine i can be used to update the degradation signal of the in-service engine i using the posterior distribution, and then input into the Bayesian LSTM-SURV model to obtain the Weibull distribution estimation parameters of the in-service engine i life and So calculate

[0022] 5) Construct model evaluation indicators for model evaluation:

[0023] In order to evaluate the performance of the Bayesian LSTM-SURV model proposed in the present invention, the present invention uses the root mean square error (RMSE) as the model evaluation index. RMSE can effectively evaluate the prediction accuracy of the model by quantifying the deviation between the predicted value and the actual value. In addition, the present invention compares the proposed model with the existing time-dependent survival neural network (TSNN) and DeepSurv model to compare the prediction performance of the three models on the same data set.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] With the development of sensor technology, the monitoring system based on the equipment operation status can monitor the operation signals closely related to the health status of the equipment in real time and perform predictive maintenance. However, the existing RUL prediction methods are mostly based on regression analysis of degradation signals. These methods only rely on complete data for modeling, ignoring the important information contained in the missing data, affecting the accuracy of the prediction. Therefore, these methods are not enough to handle missing data that lack regression labels. The present invention provides an online prediction model method for the remaining service life of the engine based on a Bayesian deep survival analysis network, which models the nonlinear relationship between multi-sensor degradation signals and engine RUL, and effectively improves the accuracy and reliability of the prediction. In the offline stage, the degradation signal is fitted by a Bayesian mixed effect model, the prior distribution of the parameters is estimated, and the Bayesian LSTM-SURV engine remaining service life online prediction model is constructed in combination with survival analysis, so that it can use complete data and missing data to predict the life distribution; in the online stage, the Bayesian method is used to model and denoise the degradation signal, and the features of the training data are migrated to the test data, and the posterior distribution of the signal is updated in real time, thereby realizing real-time RUL prediction. The technical advantages of the present invention include:

[0026] (1) By combining survival analysis and neural networks, a new loss function is designed to capture the degradation information in complete and censored data, thereby modeling the nonlinear relationship between degradation signals and remaining useful life, fully considering the problem of censored signals and missing RUL labels;

[0027] (2) The present invention adopts the Bayesian method to extract information from the training data and transfer it to the test data prediction process, dynamically optimize the prediction results, and fully explore the potential information of the data, thereby maximizing the use of existing data;

[0028] (3) Different from the traditional method of directly predicting the RUL value, the present invention assumes that the engine life follows the Weibull distribution. By modeling the life distribution and survival function, it is possible not only to calculate and predict the RUL value but also to quantify the uncertainty of these prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic flow chart of the model method provided by the present invention.

[0030] Figure 2 This is a framework diagram of the model method provided by the present invention.

[0031] Figure 3 This is a schematic diagram of the long short-term memory network (LSTM) structure adopted in the present invention.

[0032] Figure 4 The embodiment 1 of the present invention has different p 1 (complete data ratio) and p 2 Schematic diagram of the root mean square error RMSE comparison results of training data under (loss time ratio).

[0033] Figure 5 It is a schematic diagram of the conditional probability density function of the Weibull distribution predicted by 6 engines randomly selected from the test data set in Example 1 of the present invention.

[0034] Figure 6 is the engine c in embodiment 1 of the present invention at t 0 Schematic diagram of the actual and predicted conditional survival curves at times =40, 45, 50, and 55.

[0035] Figure 7 The embodiment 1 of the present invention is 1 = 0.7 (complete data ratio) and p 2 =0.6 (censoring time ratio), P under different actual remaining useful life levels CI (Actual life span L i Graph of the proportion of predictions that fall within the 90% confidence interval.

[0036] Figure 8 Schematic diagram of Bayesian update of partial signals of engine i in embodiment 2 of the present invention.

[0037] Fig. 9 is different from p in Example 2 of the present invention 1 (complete data ratio) and p 2 Schematic diagram of the RMSE comparison results of training data under (loss time ratio).

[0038] Fig.10The embodiment 2 of the present invention is 1 = 0.7 (complete data ratio) and p 2 =0.6 (censoring time ratio), P under different actual remaining useful life levels CI (Actual life span L i Graph of the proportion of predictions that fall within the 90% confidence interval. DETAILED DESCRIPTION

[0039] The present invention will be further described below by way of embodiments in conjunction with the accompanying drawings, but the scope of the present invention is not limited in any way.

[0040] The present invention provides an online prediction model method for the remaining useful life of an engine based on a Bayesian deep survival analysis network, which models the nonlinear relationship between multi-sensor degradation signals and the engine RUL, effectively improving the accuracy and reliability of the prediction. In the offline stage, the degradation signal is fitted by a Bayesian mixed effect model, the prior distribution of the parameters is estimated, and a Bayesian LSTM-SURV online prediction model for the remaining useful life of the engine is constructed in combination with survival analysis, so that it can use complete data and missing data to predict the life distribution. In the online stage, the Bayesian method is used to model and denoise the degradation signal, and the features of the training data are transferred to the test data, and the posterior distribution of the signal is updated in real time, thereby achieving real-time RUL prediction, and a model evaluation index is constructed for model evaluation. Figure 1 A schematic flow chart of the model method provided by the present invention.

[0041] The specific implementation of the present invention is as follows:

[0042] 1) Aircraft engine service signal collection and data representation:

[0043] The present invention is concerned with estimating the event occurrence time T for each engine i given a J-dimensional degradation signal. i Assume there are I engines, each engine i has J degradation signals, and each degradation signal has n i Observation values, the specific data are as follows:

[0044]

[0045] For the convenience of representation, the present invention assumes that the J degradation signals of each engine are observed at the same time point. Therefore, Y consists of J-dimensional time series data, based on which the remaining service life prediction is performed. Given an engine i, the event occurrence time can be expressed as:

[0046]

[0047] Among them, C Rrepresents the right censored set, L i represents the failure time of complete data, C i represents the end point of the censored data, and at the same time, the indicator variable d i To indicate whether the engine is deleted, that is:

[0048]

[0049] Among them, d i Represents the survival state, that is, if engine i is in L i If the fault occurs, then d i The value is 1; if the engine is in C i If d i The value of is 0. Then the complete observation data of the ith engine can be expressed as i = {Y i ,T i ,d i}.

[0050] 2) Constructing a Bayesian mixed effects model to model multivariate degradation signals:

[0051] 21) Use the Bayesian mixed effect model to characterize the engine degradation process and denoise the training set data:

[0052] y i,j,t =X i,j,t β i,j +ε t (Formula 3) Where y i,j,t represents the jth signal value of the i-th engine at time t (j=1,…,J), X i,j,t Yes (q j +1)-dimensional polynomial basis function vector (qj represents the highest order term of the signal and time function,), denoted as β i,j Yes (q j +1)-dimensional regression parameter vector, which obeys the multivariate normal distribution, ε t Represents Gaussian noise, which is normally independent and identically distributed. The order of polynomial regression of different sensors may be different, which can be determined using quantitative model selection methods. The present invention uses the Bayesian Information Criterion (BIC) to determine the optimal order of the model polynomial form.

[0053] For censored units, the limited information provided by the censored signal is often insufficient to accurately predict lifespan. Therefore, the present invention considers migrating features in the training data to the test data, which can be achieved through the Bayesian method. The method utilizes the similarity and group behavior between units in the data, can effectively achieve information migration, and can model the characteristics of the data. Specifically, the training data provides prior information for the model parameters, and this information can be used to update the posterior distribution of the test data.

[0054] 22) A two-stage approach is used to estimate model parameters.

[0055] In order to perform Bayesian updating in the online stage, the present invention designs a joint conjugate prior, i.e., the regression parameter in and are the mean vector and covariance matrix respectively, ε t represents Gaussian noise, which is normally independent and identically distributed and is expressed as Its mean is 0 and its variance is Model parameters These parameters will be utilized in the online phase via Bayesian updating. The maximum likelihood estimation (MLE) method is to estimate ψ (j) A natural way to obtain , specifically, can be obtained by maximizing the following marginal likelihood:

[0056]

[0057] in, represents the signal vector of the engine, is the parameter vector β i,j and variance The joint prior distribution of . However, the above marginal likelihood is very complex and not suitable for direct calculation, which makes parameter estimation by maximum likelihood complicated. In order to improve parameter estimation, the present invention adopts a two-stage method to estimate the hyperparameter ψ (j) This is a simpler and more efficient method. In this method, the first step is to use the MLE method to fit the degradation signal of each unit and estimate the parameters The second step is to use Estimate the hyperparameter ψ (j) Although the two-stage method may introduce some bias, this bias is usually negligible in the Bayesian online updating process. Its maximum likelihood estimate is expressed as formula 5:

[0058]

[0059] Among them, n i represents the number of signals collected by the i-th engine, Xi,j is the design matrix of sensor j for engine i, which can be expressed as

[0060] Formula 6:

[0061]

[0062] Among them, q j Represents the highest order term of the signal and time function. Its maximum likelihood estimate is expressed as Equation 7:

[0063]

[0064] Where I represents the number of engines, and are the parameter vector and the estimated variance.

[0065] 3) Constructing engine remaining life prediction model and model training based on deep survival analysis:

[0066] The engine remaining life prediction model based on deep survival analysis includes an input layer, a deep survival analysis network layer and an output layer. Figure 2 The model framework diagram is as follows:

[0067] 31) Model input: Multi-sensor degradation signals are used as the input of the model, and the sliding window technique is used to process the time series. In this process, the time series batches are normalized to a preset fixed length L to ensure that the multi-sensor degradation signal data maintains the same length L;

[0068] 32) Deep survival analysis network layer: This layer combines the long short-term memory network layer, the random dropout layer and two fully connected layers to learn the degradation information represented by the multi-sensor degradation signals. Among them, the long short-term memory network layer can process time series data, extract effective features from the signals of multiple sensors, and capture the time dependence in the degradation process; the present invention also designs a new loss function based on the survival analysis theory, so that the long short-term memory network layer can fully learn the degradation information contained in the complete data and the missing data; the random dropout layer prevents overfitting by randomly discarding neurons, and the fully connected layer is used to further fuse and process the extracted features;

[0069] 33) Model output: Assuming that the life of all units follows the Weibull distribution, the output of the model is set to the Weibull distribution parameters λ and k that the engine life follows. The details are as follows:

[0070] In the survival analysis of the engine, the survival function S of the engine i (t) represents the probability that unit i survives after a given time t, defined as:

[0071] S i (t) = P(L i >t)=1-F i (t) (Equation 8)

[0072] Among them, S i (0) = 1, represents the failure probability, and f(t) represents the probability density function (PDF) of unit i. Assume that engine i has survived to time t 0 , conditional survival function S i (t|t 0 ) is expressed as:

[0073]

[0074] The remaining life can be predicted as follows:

[0075]

[0076] in, The present invention uses Weibull distribution to characterize the probability density function f(t) of life. Weibull distribution is often used in life modeling and is an effective tool for analyzing system reliability. It has many advantages, including flexibility, wide applicability, intuitive parameter interpretation and adaptability. In survival analysis, Weibull distribution is widely used due to its effectiveness in modeling survival time and failure rate distribution.

[0077] Failure time L i It refers to the specific time from when the engine starts running to when the failure occurs, which usually represents the life of the engine. Its probability density function is:

[0078]

[0079] Where λ>0 and k>0 represent the scale parameter and shape parameter of the Weibull distribution, which control the location and scale of the distribution and determine its shape. The survival function S(t) can be calculated through the probability density function f(t) as follows:

[0080]

[0081] Among them, S(L i ) represents the survival function; L i is the time when the fault occurred;

[0082] According to (Equation 9) and (Equation 10), the remaining life can be calculated as:

[0083]

[0084] Among them, RUL i (t 0 ) indicates that engine i is at t 0 The remaining life at time t 0 Represents the current time point, i.e., the starting time point for predicting the remaining useful life. Obviously, predicting the distribution of life or remaining useful life is equivalent to predicting the two parameters λ and k of the Weibull distribution.

[0085] Neural network methods are widely used in data fusion due to their flexible structure and ability to effectively approximate arbitrary nonlinear functions. Among them, the Long Short-Term Memory (LSTM) network can effectively capture important information in sequence data and solve long-term dependency problems, which makes it perform well in processing various complex sequence tasks.

[0086] The present invention uses the denoised multi-sensor degradation signal as the input of the Bayesian LSTM-SURV model, and uses the sliding window technique to process the length of the time series. In this process, the time series batch is normalized to a preset fixed length L to ensure that the multi-sensor degradation signal data maintains the same length L and has S feature dimensions. The goal of the Bayesian LSTM-SURV model is to approximate g to achieve data fusion and engine remaining life prediction, expressed as [λ i ,k i ]=g(y i,1 ,y i,2,…, y i,L ), where the slice signal y i,L =[y i,1,L ,y i,2,L ,…,y i,J,L The BayesianLSTM-SURV model combines long short-term memory network layers, random dropout layers, and fully connected layers to fuse the degradation signals of multiple sensors and output two parameters λ i and k i Used for life span distribution prediction. Figure 3 The structure of LSTM is shown, which contains three special gating mechanisms designed to control the flow of information in the LSTM network, including the forget gate, input gate and output gate. The formula is as follows:

[0087]

[0088] Among them, y i is a vector of multidimensional signals, i t is the input gate, f t It's the forget gate, t is the output gate, g tis a candidate state, W i , W f , W o and W g are the weights of the input gate, forget gate, output gate and candidate state, respectively, and b i 、b f 、b o and b g are the biases of the input gate, forget gate, output gate and candidate state, σ is the sigmoid activation function, f is the activation function tanh, c t is the cell state, c t-1 is the unit state at the previous moment, h t is the hidden state output at the final time t. The output of the neural network is processed by the Softplus activation function to ensure that the output is non-negative. The output engine i is at the predicted point t i There are two distribution parameters: λ i and k i , respectively represent the scale parameter and shape parameter of the Weibull distribution. To prevent overfitting, the present invention applies random dropout in the output layer.

[0089] 34) Offline model training:

[0090] In the offline stage, the Bayesian LSTM-SURV model constructed in step 3) is trained using the denoised training set data in step 2) to fully learn the degradation information contained in the complete data and the deleted data, achieve accurate prediction of RUL, and update the model parameters of the neural network. The details are as follows:

[0091] Let all model parameters be Where J = 0, 1, 2, …, J, among these parameters, {θ, α} represents the model parameters of the neural network, which are the same for all units; parameters It captures the variability of each sensor between different units and can be estimated according to (Equation 5) and (Equation 6).

[0092] During model training, the loss function is designed based on the maximum likelihood estimation of the Weibull distribution, and the distribution parameter k is constrained by joint optimization. i .

[0093] The first part of the loss function is the likelihood function based on survival analysis, which takes into account the probability of the event occurring, including the missing data at time point C i The survival probability and complete data at time point L i probability of failure.

[0094] ①The first part of the loss function is as follows:

[0095] For complete data, we obtain complete degradation data and information on the exact time when the failure occurred. Among them C R represents the right-censored set, and uses f(t i ) to describe the probability distribution of failure. For censored data, T i ∈C R , using S(t i ) describes the probability distribution of the missing event. The joint likelihood function of the complete data and the missing data is obtained by substituting f(t i ) and S(t i ) to estimate the probability distribution of the entire sample, so the likelihood function can be expressed as:

[0096]

[0097] Among them, θ represents the unknown parameters of the neural network, n represents the number of engines, and L i represents the failure time of complete data, C i represents the end point of censored data, d i is the survival state, that is, if engine i is in L i If the fault occurs, then d i The value is 1; if the engine is in C i If d i The value of is 0. According to (Equation 11) and (Equation 12), (Equation 17) can be further expressed as:

[0098]

[0099] Among them, λ i and k i They represent the scale parameter and shape parameter of the Weibull distribution output by the neural network respectively.

[0100] Taking the negative logarithm of (Equation 18), we get the first part of the loss function ζ 1 :

[0101]

[0102] ②The second part of the loss function is the shape parameter k i The lower bound of k i Greater than 1, that is:

[0103] Among them, α is a negative constant, which represents the penalty term coefficient. According to the previous introduction to Weibull distribution, in the degradation process, it is restricted by k>1. The linear combination of the two loss functions ζ 1 and2 Get the final loss function L net :

[0104]

[0105] Where n is the number of engines; L i represents the failure time of complete data, C i represents the end point of missing data, d i is the survival state, that is, if engine i is in L i If the fault occurs, then d i The value is 1; if the engine is in C i If d i The value of is 0, i and k i are the scale parameter and shape parameter of Weibull distribution, and α is the penalty term coefficient. ;

[0106] 4) Online parameter update and remaining service life prediction:

[0107] After the training parameters are obtained by grid search in the offline stage, the remaining service life of the in-service engine can be predicted. Assuming that the in-service unit is represented by i, its available degradation signal is recorded as where j = 1, 2, 3, ..., J, t * Indicates the time point of prediction. In order to achieve accurate RUL prediction, by combining historical data with individual information, that is, using the Bayesian mixed effect model learned from the training data set, the model fitting parameters are used as prior information, combined with the sensor degradation signal of the in-service engine as the observation information, and the posterior distribution of the parameters is calculated by the Bayesian formula to update the Bayesian mixed effect model parameters, thereby realizing the information transfer from training data to test data. In the proposed signal update framework based on the Bayesian method, for sensor j of engine i, its posterior distribution is a mean and variance The multivariate normal distribution of is calculated as follows:

[0108]

[0109] in, is sensor j at time t * The previous design matrix, is the estimated variance of the measurement error of sensor j, and is the β obtained based on the degradation signal data of the training engine i,j The mean and covariance matrix of the prior distribution, is the degradation signal of the in-service engine. The updated degradation signal of engine i can be calculated using the mean of the posterior distribution according to (Equation 3). After obtaining the updated degradation signal, it is then input into the Bayesian LSTM-SURV model to obtain the Weibull distribution estimation parameters obeyed by the in-service engine i life and Thus, RUL is calculated i Estimated value of The specific formula is as follows:

[0110]

[0111] Among them, t * Indicates the time point at which the prediction is made.

[0112] 5) Construct model evaluation indicators for model evaluation:

[0113] In order to evaluate the performance of the Bayesian LSTM-SU RV model proposed in the present invention, we use the root mean square error (RMSE) as the model evaluation index. RMSE can effectively evaluate the prediction accuracy of the model by quantifying the deviation between the predicted value and the actual value. In addition, the present invention compares the proposed model with the existing time-dependent survival neural network (TSNN) and DeepSurv model to compare the prediction performance of the three models on the same data set. The present invention is further illustrated by examples below.

[0114] Example 1: Simulation Verification

[0115] For the simulation data, the present invention assumes that the degradation signal is collected from two sensors, and the real degradation signal path of unit i has a polynomial trajectory with personalized random effects. Specifically:

[0116]

[0117] Among them, X i,j,t =[1,t,t 2 ], j∈{1,2}, regression parameters in and are the mean vector and covariance matrix respectively, ε j represents Gaussian noise, which is normally independent and identically distributed and is expressed as Its mean is 0 and its variance is The parameters used in (Equation 25) are listed in Table 1. The present invention assumes that the actual life of the engine follows a Weibull distribution, with a scale parameter λ=20 and a shape parameter k=80, and then generates the failure occurrence time Li of each unit by uniform sampling.

[0118] Table 1. Parameter settings for degradation signals

[0119]

[0120] In order to explore and compare the effect of the proportion of missing data on the accuracy of remaining life prediction, the present invention defines two parameters: the proportion of complete data p 1 and the censoring time proportion p 2 , by setting different p 1 and p 2 The present invention can construct training sets with different degrees of missing data. The two parameters are defined as follows:

[0121]

[0122]

[0123] Among them, N F is the number of complete life units (i.e., complete data), N is the total number of training data, and C i is the end point of the censored data, L i is the time of failure before censoring. In practical applications, p 2 may be different for different units, however, for simplicity, we apply the same p to all units 2 In the present invention, p 1 Set to 0.3, 0.5, and 0.7; p 2 It is set to 0.2, 0.4, 0.6, 0.8, and 0 to compare the impact of different proportions of missing data on prediction accuracy. In particular, p 2 = 0 means no missing data. Based on this, we can generate training sets with different missing levels.

[0124] The data generation process is described in detail in Table 2. The training dataset consists of 500 units, i.e., N = 500, and the test dataset consists of 100 units, i.e., N s =100, each unit corresponds to a time period. According to the steps provided in Table 2, the training data set O of the present invention can be generated. i , where i=1,2,…,500.

[0125] Table 2. Description of data generation process

[0126]

[0127] In order to determine the optimal structure of the Bayesian LSTM-SURV model and prevent the model from overfitting, the present invention adopts a five-fold cross validation on the training data set, retains 20% of the units for validation, and finally selects the model with the smallest loss function. In the cross validation process, the present invention uses grid search to determine the hyperparameters of the neural network, including the number of hidden nodes in the long short-term memory network layer, the number of hidden layers, the number of training rounds, the learning rate, and the penalty term coefficient α of the loss function. These hyperparameters increase with the complete data ratio p. 1 and the censoring time proportion p 2 In addition, for the random dropout layer and fully connected layers (FC) of the Bayesian LSTM-SURV model, specific values ​​are set: the number of hidden nodes in the FC1 layer is 64, the number of hidden nodes in the FC2 layer is 32, the dropout rate of the random dropout layer is 0.2, and for the activation function, both FC1 and FC2 use the Sigmoid activation function. The output layer uses the Softplus function to ensure that the output value is positive, which meets the constraint that the parameters λ and k of the Weibull distribution must be positive.

[0128] During the training process, all input data need to be normalized and denoised, and the one with the smallest loss function is selected as the training result and the neural network structure and parameters are saved, that is, the trained neural network is obtained. After the model training is completed, the present invention evaluates the prediction accuracy of the model by calculating the root mean square error (RMSE), which is defined as:

[0129]

[0130] Where N is the number of engines, and i are the predicted and actual RUL of engine i respectively.

[0131] Figure 4 Shown in different p 1 = 0.7, 0.5, 0.3 (proportion of complete data) and p 2 =0, 0.2, 0.4, 0.6, 0.8 (proportion of censoring time), the RMSE comparison results of the training data. By observing the RMSE in the figure with p 2 It can be clearly seen that the RMSE of the model increases with p 2 This phenomenon shows that at a fixed complete data ratio p 1 Under this condition, the increase of degradation signal in the censored data improves the accuracy of model prediction. 1The impact of the value is expected to be 1 The larger the value, the lower the RMSE, which is consistent with Figure 4 This is because the higher proportion of complete data p 1 It provides more complete degradation signals, that is, more degradation status and remaining service life prediction information. As the equipment runs longer, more degradation signals are collected and the degradation trend becomes more significant. 1 and p 2 The corresponding RMSE of model training is shown in Table 3.

[0132] Table 3. Different p 1 and p 2 The corresponding root mean square error of model training

[0133]

[0134] During the training process, the model takes multi-sensor degradation signals as input and predicts the lifetime distribution of each unit by outputting parameters λ and k, which follows a Weibull distribution. Compared with the traditional point estimation method, the model proposed in this paper predicts the distribution of the remaining useful life by outputting the parameters of the lifetime distribution, thereby capturing the uncertainty of the lifetime more comprehensively. This distribution-based approach improves the accuracy and reliability of the remaining useful life prediction by providing detailed information about the lifetime, including probability density functions, confidence intervals, survival functions, etc. Next, the prediction results are presented from the perspective of distribution prediction.

[0135] Figure 5 The conditional probability density function (PDF) of the prediction of six randomly selected units (#42, #21, #57, #17, #81, #95) in the test data set is shown. The PDF is drawn based on the λ (scale) and k (shape) parameters of the Weibull distribution predicted by the Bayesian LSTM-SURV model for each unit. As can be seen from the figure, the predicted life (solid line) is very close to the actual life (dashed line), mainly concentrated near the peak of the PDF, where the probability of failure is the highest. In addition, all actual life falls within the gray shaded area, which represents the 90% confidence interval (CI) of the unit life calculated based on the Weibull distribution, which shows that the model effectively captures the variability in the degradation process of different units and provides a life distribution prediction with high confidence.

[0136] Figure 6 The cell c is shown at four different prediction time points (t 0 =40,45,50,55). At the early prediction time t 0When t = 40, the gap between the predicted value and the true value is the largest. However, as more degradation signal information is collected, t 0 = 55, the predicted survival curve is getting closer and closer to the true curve. The difference in early predictions is because they are highly dependent on the accuracy of the prior information obtained from historical data, and there are fewer observations available for unit c. Therefore, the conditional survival curve in the early stage mainly reflects the trend at the overall level rather than the individual unique characteristics of unit c. As more degradation signal data are collected, the estimated survival curve is increasingly able to capture the individual degradation trend, resulting in more accurate predictions of the survival curve and more accurate predictions of the remaining useful life.

[0137] In order to further evaluate the accuracy of the confidence interval (CI) predicted by the model, the present invention defines P CI As an indicator, P CI Indicates actual life span L i falls within the 90% CI of the prediction [A i ,B i ], which is the prediction parameter λ of the Weibull distribution. i and k i Calculated. The formula is as follows:

[0138]

[0139] Among them, N s is the total number of test units, L i is the actual life of the ith unit, A i and B i Represent the lower and upper bounds of the 90% confidence interval of the prediction; is an indicator function, if the actual life span L i If it falls within the 90% confidence interval, it is equal to 1, otherwise it is equal to 0.

[0140] Figure 7 It shows the complete data proportion p at different remaining life levels. 1 =0.7 (corresponding to p 2 =0.2, 0.4, 0.6, 0.8), and the proportion of censoring time p 2 =0.6 (corresponding to p 1 =0.7,0.5,0.3) in the case of actual life L i The proportion P that falls within the 90% confidence interval of the prediction CIIn the figure, "All" represents all engines to be predicted, and "TL" represents engines with a remaining life less than or equal to L. Overall, an upward trend is observed in both figures, because as the engine approaches failure, more observations are collected, making the model's inference of the degradation process more accurate, resulting in more actual life falling within the predicted 90% confidence interval. In the left figure, when p 1 = 0.7, the model is in p 2 = 0.6, maintaining strong prediction performance, which is comparable to p 2 = 0.8; in the right figure, when p 2 =0.6, the solid line (p 1 =0.7) is significantly better than the dotted line (p 1 =0.5). Therefore, p 1 = 0.7 and p 2 =0.6 is a relatively cost-effective combination that can achieve satisfactory prediction results with relatively less data.

[0141] In order to verify the effectiveness of the model proposed in this paper, it is compared with the Time-dependent Survival Neural Network (TSNN) and DeepSurv model. TSNN is a non-parametric model with a single hidden layer. This method uses multi-sensor data as input and discretizes the future time interval into a series of different time points β 1 <β 2 …<β k , each time interval is marked with 1 or 0 to indicate the k Whether a failure occurs before, this method effectively structures TSNN into a combination of multiple binary classifiers. In order to combine historical data, an exponential decay ratio is used to enhance the failure risk estimation based on degradation data. The DeepSurv model combines the Cox proportional hazard (Cox PH) model with a deep multilayer perceptron. Although it does not explicitly consider the temporal dependence of covariates on the occurrence of events, it shows a high degree of adaptability when combined with a neural network and is an effective Cox PH model. The present invention uses these two methods as benchmark methods for model comparison. In the experimental parameter setting, the number of hidden layer neurons of TSNN is set to 20, and the DeepSurv model has 2 hidden layers, each containing 64 neurons.

[0142] Table 4 shows the RMSE comparison results of the Bayesian LSTM-SURV model, TSNN, and DeepSurv. The results show that as p 1 and p 2As p increases, the RMSE of the three methods all improve, which is what we expected because the dataset is growing. However, it is obvious that the proposed model (Bayesian LSTM-SURV) always outperforms the baseline method at all scales. 1 =0.7 and p 2 = 0.8, the Bayesian LSTM-SURV method achieved the best prediction results with an RMSE of 5.500. In contrast, the performance of the DeepSurv model is always lower than that of other models, mainly because it relies on the latest observations and the overall level obtained during training, and fails to effectively utilize historical time series data.

[0143] Table 4. Comparison of root mean square error of different models

[0144]

[0145] Example 2: C-MAPSS turbofan engine dataset example

[0146] Specifically, the engine degradation simulation data generated by the C-MAPSS software developed by the National Aeronautics and Space Administration (NASA) is used. This software is widely used to simulate the engine health monitoring of real large commercial turbofan engines. Users can obtain signal data under different flight conditions by adjusting different flight parameters. The engine simulated by C-MAPSS has a total of 21 output signals, as shown in Table 5.

[0147] Table 5.21 Output signal introduction

[0148]

[0149]

[0150] Table 5.21 Output signal introduction

[0151]

[0152] This dataset includes 100 training units with 20,631 observation periods and 100 test units with 13,096 observation periods. In the training dataset, the engine decays until failure. To simulate censored data, we set the complete data ratio p 1 = 0.7, 0.5, 0.3 and the censoring time proportion p 2=0, 0.2, 0.4, 0.6, 0.8, these proportions are consistent with the simulation verification. In the test data set, the data is terminated at a certain time before failure, but its remaining life is known. The present invention uses training data to estimate the prior distribution parameters of the signal and train the neural network, and uses test data to predict and verify the model prediction effect. The specific implementation method is as follows:

[0153] (1) Variable screening and data preprocessing:

[0154] In the data set, it was observed that the operating parameters and sensor variables numbered S1, S5, S6, S10, S16, S18 and S19 had no significant changes, so they were removed during the modeling process. In order to maintain the consistency and quality of the data, min-max normalization was used to improve the prediction performance. Subsequently, the present invention used the Bayesian method to denoise the signal. The core idea of ​​the Bayesian method is to use prior knowledge and observation data to estimate the true signal, thereby reducing the impact of noise. That is, the training data is fitted by a Bayesian mixed effects model to model the potential degradation trend; for the test data, the signal is corrected by applying Bayesian updates in combination with prior knowledge, thereby effectively reducing noise and improving prediction accuracy. In the Bayesian method, the specification of the prior usually depends on the availability of relevant historical data or expert knowledge. In the present invention, since the test unit i has similarities with the training data, we choose a prior distribution based on the training data to transfer information from the training data to the test data. Figure 8 The Bayesian update process of some degradation signals of randomly selected unit i is shown, including the original measured value (fluctuating solid line), the directly fitted signal (smooth solid line) and the signal after Bayesian update (dashed line). Obviously, the dashed line combined with prior information captures the degradation trend more accurately than the smooth solid line that only fits the observed value. Therefore, the Bayesian method achieves a more accurate and effective denoising effect.

[0155] (2) Display of lifespan prediction results:

[0156] During the training process, the present invention adopts five-fold cross validation and uses a grid search method to estimate the hyperparameters of the BayesianLSTM-SURV model, such as the number of hidden layers, the number of neurons, and the number of iterations. The parameters of the model are as follows: the first fully connected layer (FC1) has 64 hidden nodes, the second fully connected layer (FC2) has 32 hidden nodes, and a dropout rate of 0.2 is used to reduce overfitting. Both FC1 and FC2 layers use the ReLU activation function, while the output layer uses the Softplus activation function. Finally, the model with the smallest loss is selected. After training, the model is applied to 100 test units for performance evaluation.

[0157] Fig. 9 Indicates that in p 1= 0.7, 0.5, 0.3 (proportion of complete data) and p 2 = 0, 0.2, 0.4, 0.6, 0.8 (the proportion of the missing time) of the training data, showing the performance of the model under different proportions of missing data, as shown in Table 6. The results are consistent with the simulation verification. 1 and p 2 As the p increases, the RMSE of the model decreases accordingly, which is obvious because the larger the p 1 and p 2 Indicates that the proportion of complete data and the proportion of missing time are higher, and more degradation data provide the model with richer information, enabling it to more accurately monitor the degradation trajectory of the engine and improve the accuracy of RUL prediction.

[0158] Table 6. Different p 1 and p 2 The corresponding root mean square error of model training

[0159]

[0160] Fig.10 It shows the proportion of actual life falling within the predicted 90% confidence interval under different remaining service life levels, that is, P CI The left figure shows the 1 =0.7, different censoring time proportions p 2 P of (0.2, 0.4, 0.6, 0.8) CI , while the right figure shows the 2 =0.6, different p 1 In general, an upward trend is observed in both figures. As the device approaches failure, the accumulation of more degradation data makes the model estimate the degradation trend more accurately, resulting in more actual life falling within the predicted 90% confidence interval.

[0161] In the case part, the performance of the Bayesian LSTM-SURV model proposed in the present invention is verified by further comparison with the benchmark methods (especially TSNN and DeepSurv). The results are shown in Table 7. The results show that the prediction effect of the model (Bayesian LSTM-SURV) proposed in the present invention exceeds the benchmark method in all tested proportions, especially when p 1 = 0.7 and p 2= 0.8, the Bayesian LSTM-SURV method achieved the best prediction performance with an RMSE of 7.696. In addition, the DeepSurv model consistently outperformed the other models because it only relied on the latest observations and the overall level obtained during training without considering historical time series data.

[0162] Table 7. Comparison of root mean square error of different models

[0163]

[0164] It should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A method for predicting remaining engine life based on deep survival analysis, characterized in that: Construct an engine remaining service life prediction model based on deep survival analysis, model the nonlinear relationship between multi-source sensor signals and the remaining service life distribution of the engine, and use the survival analysis method to achieve effective online prediction of the remaining service life of the engine for signal deletion or lack of remaining service life RUL labels; The steps include: 1) Collect complete observation data of the engine, including: engine degradation signal data, engine failure occurrence time or loss time, and engine failure or loss indicator variables; 2) Model the multi-sensor degradation signals, establish a Bayesian mixed effect model to characterize the engine degradation process, and estimate the model parameters; 3) Construct an engine remaining life prediction model based on deep survival analysis and perform model training; By constructing an engine remaining life prediction model based on deep survival analysis, namely the Bayesian LSTM-SURV model, the relationship between degradation signals and remaining service life is established; the engine remaining life prediction model includes an input layer, a deep survival analysis network layer, namely the LSTM-SURV layer, and an output layer; Taking multi-sensor degradation signals as input to the model; The LSTM-SURV layer is used to learn the degradation information represented by the multi-sensor degradation signals; the structure of the LSTM in the LSTM-SURV layer includes a forget gate, an input gate, and an output gate, which are used to control the flow of information in the LSTM; The parameters of the distribution of engine life are used as the output of the model; Model training is performed in the offline stage: the denoised multi-sensor degradation signal is input into the engine remaining service life prediction model based on deep survival analysis constructed in step 3). Based on the survival analysis, the loss function of the model is designed to include the first part and the second part, which is used to fully learn the degradation information contained in the complete data and the missing data to predict the remaining service life; The first part of the loss function includes the survival probability of the censored data at the censoring time and the failure probability of the complete data at the failure time; the second part of the loss function is the lower bound of the shape parameter, ensuring that it is greater than 1; 4) Preprocess the engine data to be predicted based on the prior distribution of the training data, migrate the processed information from the training data to the test data, and dynamically update the model parameters using the test set data; input the updated data into the trained engine remaining life prediction model based on deep survival analysis to perform online parameter estimation and remaining service life prediction; 5) Construct model evaluation indicators and conduct model evaluation on the constructed model; Through the above steps, the online prediction of the remaining service life of the engine based on the Bayesian deep survival analysis network is realized.

2. The engine remaining life prediction method based on deep survival analysis as claimed in claim 1, characterized in that: In step 4), Weibull distribution is specifically used to characterize the probability density function of the remaining life of the engine; Survival function S(L i ) is calculated by the probability density function f(t), expressed as: Where λ>0 and k>0 represent the scale parameter and shape parameter of the Weibull distribution respectively; S(L i ) represents the survival function; L i is the time when the fault occurred; The calculation of remaining life is expressed as: Among them, RUL i (t0) represents the remaining life of engine i at time t0; t0 represents the current time point, that is, the starting time point for predicting the remaining service life; the distribution of the engine life or the remaining service life is predicted, that is, the two parameters of the predicted distribution.

3. The engine remaining life prediction method based on deep survival analysis as claimed in claim 2, characterized in that: The loss function of model training is expressed as: Where ζ1 is the first part of the loss function; ζ2 is the second part of the loss function; n is the number of engines; L i represents the failure time of complete data, C i represents the end point of missing data, d i is the survival state, that is, if engine i is in L i If the fault occurs, then d i The value is 1; if the engine is in C i If d i The value of is 0, i and k i are the scale parameter and shape parameter of Weibull distribution respectively, and α is the penalty coefficient.

4. The engine remaining life prediction method based on deep survival analysis as claimed in claim 3 is characterized in that: Step 2) specifically uses a Bayesian mixed effects model to characterize the engine degradation process.

5. The engine remaining life prediction method based on deep survival analysis as claimed in claim 4 is characterized in that: Specifically, a two-stage method is used to estimate model parameters.

6. The engine remaining life prediction method based on deep survival analysis as claimed in claim 5, characterized in that: Calculate the remaining service life RUL of the engine i Estimated value of The method is expressed as: in, and are the estimated parameters of the Weibull distribution of the service life of the engine i; t * Indicates the time point at which the prediction is made.

7. The engine remaining life prediction method based on deep survival analysis as claimed in claim 1, characterized in that: The output of the engine remaining life prediction model based on deep survival analysis constructed in step 3) is processed by the Softplus activation function.

8. The engine remaining life prediction method based on deep survival analysis as claimed in claim 1, characterized in that: Step 1) Collect the complete observation data of the engine and express it as: i = {Y i ,T i ,d i }, where Y i Data representing the J-dimensional degradation signal of the engine, T i represents the engine failure or loss time, d i Indicates engine failure or deletion, and the distribution value is 1 or 0.

9. The engine remaining life prediction method based on deep survival analysis as claimed in claim 1, characterized in that: In step 5), the root mean square error (RMSE) is used as the model evaluation indicator to effectively evaluate the prediction accuracy of the model by quantifying the deviation between the predicted value and the actual value.

Images