Method for predicting residual life under small sample based on task increment element learning

Abstract

The invention relates to a task increment element learning-based residual life prediction method under a small sample, which comprises the following steps of: acquiring a multi-element sensor failure signal and an actual residual life of an aircraft engine, and preprocessing and selecting the signal; a dual network is constructed, the dual network comprises a meta network and an auxiliary network, the meta network regards mapping of sensor data of each device to residual life as a regression task, and a task specific model of each device is automatically learned through meta learning; the auxiliary network generates a pseudo label for the label-free sample of the non-failure equipment; training a dual network by a task increment training algorithm based on the labeled data of the failed equipment and the unlabeled data of the non-failed equipment; and utilizing the auxiliary network to predict a pseudo label of the test equipment sample, updating the element network to the task specific model based on the pseudo label support set, and utilizing the task specific model to predict the residual life of the test equipment. Compared with the prior art, the method can achieve the precise prediction of the residual life under a small number of failure samples.

Description

Technical Field

[0001] The present invention relates to the field of equipment remaining life prediction, and in particular to a method for predicting remaining life under small samples based on task incremental meta-learning. Background Art

[0002] With the development of sensor technology, multi-sensor signals have been widely used to monitor the degradation status of equipment and predict the remaining service life of equipment. Based on the diagnosis results, operation and maintenance personnel can reasonably arrange maintenance plans to avoid production delays, safety hazards and other problems caused by sudden equipment failure, while reducing maintenance costs and improving maintenance efficiency.

[0003] A deep learning model is a neural network composed of multiple layers of interconnected nodes. It can automatically learn the characteristics of data, and use the characteristics as input to output the corresponding degradation status, such as health index or remaining useful lifetime (RUL). Remaining useful lifetime prediction methods based on deep learning usually require a large amount of labeled data to train the model in order to achieve satisfactory generalization effects. In actual application scenarios, labeled data can only be obtained when the equipment fails, and the equipment is usually repaired or replaced before failure. Therefore, usually only a small number of failed devices generate a small amount of labeled data, and a large number of non-failed devices generate a large amount of unlabeled data. Therefore, training a deep learning model with a small amount of labeled data can be regarded as a small sample problem.

[0004] Another problem with deep learning models is that only local sensor data is used as input to the model. The model extracts local features from the input and outputs the predicted RUL value, while ignoring the global features of the equipment. Typically, deep learning-based methods use moving windows to divide sensor signals into multiple fixed-length time series. Each fixed-length time series is used as the input of a sample, which contains local degradation information of a device within a specific time period, such as sensor measurements, degradation rate, degradation curvature, etc. However, the RUL value corresponding to a time series also depends on the unique characteristics of the equipment, such as initial wear state, assembly error, failure mode, and full life length. These unique characteristics vary on different devices and play a decisive role in the remaining service life of the equipment. Summary of the invention

[0005] The purpose of the present invention is to provide a method for predicting the remaining useful life under small samples based on task incremental meta-learning, which uses labeled data of a small number of failed devices and unlabeled data of a large number of non-failed devices to train the model, and perform accurate RUL prediction on the running test equipment.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a method for predicting the remaining useful life under small samples based on task incremental meta-learning, comprising: (1) proposing a dual network structure: the dual network is composed of a meta-network and an auxiliary network. The meta-network regards the mapping of sensor data of each device to RUL as a regression task, and uses meta-learning to automatically learn the task-specific model of each device. The parameters of these models contain the global characteristics of the device and can help to achieve accurate RUL prediction. The auxiliary network generates pseudo-labels for unlabeled samples of non-failed devices, so that the meta-model can update the task-specific model of non-failed devices based on the pseudo-label samples. (2) Proposing a task incremental training algorithm: the entire training process is designed as a series of iterative training processes with task additions to generate accurate pseudo-labels for unlabeled samples. In each iteration, the dual network generates pseudo-labels for unlabeled samples of non-failed devices. The pseudo-label samples of a non-failed device constitute a pseudo-label task. The algorithm selects a certain number of tasks with reliable pseudo-labels from the pseudo-label tasks and adds them to the training task set. The model will be retrained based on the expanded training task set. The model trained based on the expanded training task set has better generalization ability and can provide more reliable pseudo labels for the remaining unlabeled samples. When all the non-failed devices are added to the training task set, the iteration of the algorithm stops. (3) A pseudo-label calibration algorithm is proposed: the input sensor data usually contains noise caused by changing working conditions and sensor measurement errors, so that the RUL prediction values ​​output by the meta-network and the auxiliary network contain prediction errors and deviate from the true values. Using these erroneous RUL prediction values ​​as pseudo-label samples for training the model will lead to error accumulation, so the network output needs to be calibrated. The designed calibration algorithm uses the potential linear correlation between the RULs of multiple time steps to calibrate the output of the model, and then uses the calibrated prediction values ​​as pseudo-labels. (4) A pseudo-label task selection algorithm is proposed: the meta-learning and auxiliary networks in the dual network structure learn different degradation characteristics. Therefore, it is assumed that for a non-failed device, the closer the RUL output obtained by the two networks inputting all unlabeled samples of the same non-failed device is, the more reliable the pseudo-label of the task can be considered. Finally, the dual network is trained with a limited number of labeled samples of failed devices and a sufficient number of pseudo-label samples of non-failed devices. As the number of tasks in the training task set increases, the generalization performance of the dual network on different devices continues to improve. For the test device, the meta network uses the pseudo-labels generated by the auxiliary network to update the task-specific model, and then predicts its RUL. The specific steps are as follows:

[0008] S1, data acquisition: obtaining the failure signals and actual remaining life of the aircraft engine's multi-sensor, and preprocessing and selecting the signals;

[0009] S2, network construction: construct a dual network, the dual network includes a meta-network and an auxiliary network, the input of the meta-network and the auxiliary network are sensor failure signals of multiple time steps of fixed length, and the output is the time interval between the last time step of the input and the failure moment, that is, the remaining life; wherein the meta-network regards the mapping of sensor data of each device to the remaining life as a regression task, and uses meta-learning to automatically learn the task-specific model of each device, the parameters of the task-specific model include the global feature information of the device; the auxiliary network generates pseudo-labels for the unlabeled samples of non-failed devices, so that the meta-network can update the task-specific model of non-failed devices based on the pseudo-label samples;

[0010] S3, network training: train the dual network using the task increment training algorithm based on the labeled data of failed devices and the unlabeled data of non-failed devices;

[0011] S4, Remaining Life Prediction: The auxiliary network is used to predict the pseudo-labels of the test equipment samples, and the meta-network is updated to the task-specific model based on the pseudo-label support set, and the task-specific model is used to predict the remaining life of the test equipment.

[0012] The step S1 is specifically as follows: using S sensors to monitor the operating status of the equipment at time intervals m, a sample at time t contains sensor measurements of c consecutive time steps, that is, from time step j to j+c-1 and c>0, then the input of the meta-network and the auxiliary network is the matrix in is a column vector of sensor measurements at time step j, and the corresponding label y i，j is the regularized remaining life value corresponding to time step j, where the non-failed equipment samples are unlabeled;

[0013] The sample set of failed equipment l is defined as The sample set of non-failed equipment u is The sample set of the test device q is where n l , n u and n q The purpose of the present invention is to train a model using labeled data of L failed devices and unlabeled data of U non-failed devices to predict the RUL of Q test devices.

[0014] The meta-network regards the mapping of sensor data of each device to RUL as a regression task. Since different devices have similar degradation processes and common degradation characteristics, the mapping of sensor data of different devices to RUL is a related task. On the other hand, different devices have different initial wear states, assembly errors, failure modes, and full life lengths, which make these tasks different. The meta-learning model can learn the model initialization parameters shared by multiple related tasks, and can update the initialization parameters to task-specific parameters through gradient descent based on the labeled samples of each task. The initialization parameters shared by the tasks contain common feature information of multiple devices, and the task-specific parameters contain global feature information specific to each device. Therefore, compared with the machine learning model, meta-learning can use common features and unique information to achieve accurate RUL prediction.

[0015] The meta-network consists of a task-shared feature extractor gθ(·) that learns common degenerate features and a task-specific regression layer. The remaining life prediction value is output using the equipment-specific global feature information, expressed as:

[0016]

[0017] Where X i，j is the input sensor data, y i，j is the output remaining life prediction value, g θ (·) is a task-shared feature extractor with parameter θ, is a task-specific regression layer for task i, with parameters w i , f i (·) represents the task-specific model of the meta-network.

[0018] The purpose of the meta-network is to learn meta-knowledge shared by the associated tasks, including the initialization parameters w0 of the task-specific regression layer and the parameters θ of the task-shared feature extractor; by gradient descent based on the support set, the initialization parameters w0 of the task-specific regression layer are updated to the task-specific parameters w i ; Meta-knowledge is learned by external updates based on query sets of multiple tasks, so that the learned feature extractor and initialization parameter w0 have good generalization ability, that is, the model f obtained by internal updates for different tasks i (·) have good prediction effects; the support set and query set are two non-overlapping sub-datasets, which are composed of n of failed devices / non-failed devices / test devices. spt and n qry The sample consists of labeled / pseudo-labeled samples.

[0019] The training process of the meta-network is as follows:

[0020] Task sampling: Sample T tasks from the training task set. From the dataset D i Sample n spt and n qry The samples constitute the support set and the query set. i is the sample set D of failed device l l , the sample set D of the non-failed equipment u u Or the sample set D of the test device q q ;

[0021] Internal update: task-specific parameter w i Using the initialization parameter w0 as the initial value, perform n gradient descent updates based on the support set to obtain f i (·);

[0022] External update: Update the distribution p(w0) of the initialization parameter w0 of the task-specific regression layer and the task-shared feature extractor parameters θ by gradient descent, the gradient of which is based on the updated f of all sampled tasks i (·) The loss function is calculated on the query set.

[0023] The present invention further considers cognitive uncertainty and learns the distribution of initialization parameters w0 through external update and the distribution of parameters w1 through internal update. i The distribution of p(w i ). Next, we will introduce the internal update and external update processes in detail.

[0024] The purpose of the internal update is to update the initialization parameter distribution p(w0) to make it close to the posterior distribution of task-specific parameters It is implemented using Stein variational gradient descent (SVGD), where is the task-specific parameter w i The posterior distribution of To support the set SVGD is a powerful nonparametric variational inference method that approximates the target distribution using a series of particles.

[0025] Example of keeping M model parameters based on Stein variational gradient descent Each instance is a particle. In the internal update, the update rule of the mth particle is as follows:

[0026]

[0027] Where, ∈ is the size of the update step; K(·, ·) is a positive definite kernel, and the square Euclidean distance function is selected in the present invention; The first term makes the particles approach the target distribution, and the second term prevents the particles from overlapping to one point; For particles The Del operator.

[0028] The outer update optimizes the shared feature extractor parameters θ and the regression layer initialization parameter distribution p(w0) so that for the sampling task, the task-specific model f after the inner update i (·) has a good prediction effect on its query set. The present invention uses the following formula to evaluate the sampling task Prediction results on the query set:

[0029]

[0030] In the formula, is the conditional likelihood function of the remaining life prediction, since Stein’s variational gradient descent approximates the posterior distribution by keeping M examples The Monte Carlo approximation of the above formula for evaluating prediction performance is:

[0031]

[0032] In the formula

[0033] Then, the loss function of the external update is:

[0034]

[0035] The external updated loss function minimizes the log-likelihood value of T sampling tasks. By optimizing the loss function, all the initialization parameter particles and feature extractor parameters are jointly optimized to achieve a good generalization effect, so that for any associated task Its updated task-specific model f i (·) Accurate RUL prediction can be achieved.

[0036] In the present invention, a small sample scenario is defined as having only a limited number of failed devices. Since the training task set consists of a limited number of failed devices, the generalization ability of the meta-network obtained through training is limited. Therefore, the purpose of the present invention is to predict accurate pseudo-labels and select reliable pseudo-label tasks, add them to the training task set, and increase the number of training tasks to improve the generalization performance of the model. However, when using a meta-network to predict the RUL of samples of non-failed devices, the initialization parameters of the regression layer cannot be updated to the task-specific parameters through gradient descent. In order to solve this problem, the present invention designs a dual network structure, in which an auxiliary network is designed to generate pseudo-labels for non-failed devices, so that the meta-network can be internally updated based on the generated pseudo-labels and generate another set of pseudo-labels. In the present invention, the auxiliary network can be any machine learning model that can realize the output of RUL prediction values ​​based on input sensor data, which is expressed as Where ρα(·) is an auxiliary network with parameter α. Then, the present invention calibrates the two sets of pseudo labels and selects pseudo label tasks with high reliability to add to the training task set. The increased number of tasks will improve the generalization effect of the model.

[0037] The task increment training algorithm is specifically as follows: the entire training process of the dual network is designed as an iterative training process with a series of tasks added, so as to generate accurate pseudo labels for unlabeled samples. In each iteration, the dual network generates pseudo labels for unlabeled samples of non-failed devices. The pseudo-label samples of a non-failed device constitute a pseudo-label task. The task increment training algorithm selects a preset number of pseudo-label reliable tasks from the pseudo-label tasks and adds them to the training task set. The dual network is retrained based on the expanded training task set. When all non-failed devices are added to the training task set, the iteration of the algorithm stops.

[0038] The kth iteration of the task incremental training algorithm includes the following steps:

[0039] Based on the training task set Train the auxiliary network parameters α and predict the unlabeled task set Pseudo labels of all task samples in , where, for the first iteration, the training task set The unlabeled task set consists of L labeled samples of failed devices. It consists of U unlabeled samples of non-failed devices;

[0040] Based on the training task set Train the meta-network to obtain the shared feature extractor parameters θ k and regression layer initialization parameters w k，0 , then for the unlabeled task set For all tasks in , the meta-network is based on the pseudo-labeled support set Update to get the task-specific model f i (·) and predict another set of pseudo labels Where X i，j ∈D i , Pseudo labels in the pseudo label support set;

[0041] The pseudo-label calibration algorithm is used to calibrate the two sets of pseudo-labels. and Perform calibration;

[0042] Select reliable pseudo-label tasks to form a selected task set based on pseudo-label task selection algorithm And update the training task set and unlabeled task set like Then update k = k + 1, if Then the algorithm ends.

[0043] Task-incremental learning improves the generalization ability of the model by adding tasks with reliable pseudo-labels to the training dataset and retraining the model in each iteration, while generating more accurate pseudo-labels for the remaining unlabeled tasks for subsequent iterations. Therefore, the task-incremental model can continuously improve the prediction accuracy of the model as the number of iterations increases.

[0044] Due to changing working conditions and measurement noise, the outputs of the meta-network and auxiliary network usually contain prediction errors and deviate from the true RUL. The meta-network is updated based on the support set of pseudo-labels and updated based on the pseudo-label tasks outside. Therefore, inaccurate pseudo-labels will lead to error accumulation and increase the prediction error of the network. Therefore, the present invention proposes a pseudo-label calibration algorithm to calibrate the outputs of the auxiliary network and the meta-network. The algorithm uses the linear relationship between the RUL of the same device at different time steps to calibrate the pseudo-label of each non-failed device, and uses the calibrated prediction value as the pseudo-label. The design of the algorithm mainly relies on two ideas:

[0045] First, there is a theoretical linear relationship between samples of the same device at time steps jk and j:

[0046] yi,jk=yi,j + km

[0047] Where m is the time interval.

[0048] Second, the RUL prediction value of the corresponding sample when the equipment is close to failure is more reliable, because the sensor data close to failure has a clearer degradation trend and smaller measurement noise. iThe relationship between the samples of and the samples of time step j is globally calibrated, that is:

[0049]

[0050] In the formula, is the global calibration result, is the time step n i 's pseudo-labels.

[0051] On the other hand, there is also a linear relationship between the RUL prediction values ​​of adjacent time steps. The present invention uses SG (Savizkg-Golag) filtering for local calibration. SG filtering is a data smoothing method based on local least squares polynomial approximation. Local calibration uses 2n c +1 adjacent time step, i.e. The remaining life prediction value is fitted with a linear model, and the result of local calibration is:

[0052]

[0053] In the formula, is the local calibration result, β0 and β are the model coefficients of SG filtering.

[0054] The final calibration result of the pseudo-label calibration algorithm is:

[0055]

[0056] Where λ is a hyperparameter that balances global calibration and local calibration. is the pseudo label after calibration.

[0057] Global calibration can filter out prediction errors caused by sensor measurement errors, but the results of global calibration lack robustness because they only rely on the prediction results of the last time step. Local calibration linearly fuses the prediction results of adjacent time steps, saves the local prediction results, and can filter out prediction errors caused by changing working conditions.

[0058] The pseudo-label calibration algorithm is used to calibrate the outputs of the auxiliary network and the meta-network to obtain the calibrated remaining life prediction values. and Although the output of the network has been calibrated, in the first few iterations, there is very little labeled data used to train the dual network, which limits the generalization ability of the model and the predicted pseudo-labels still contain errors. Directly adding all of these pseudo-labeled non-failed devices to the training task set will lead to the accumulation of model errors. Therefore, the present invention adopts a pseudo-label task selection algorithm to select only a small number of pseudo-label tasks with reliable pseudo-labels to be added to the training data in each iteration. Considering that the auxiliary network and the meta-network have learned different degradation characteristics, if the predicted RUL values ​​of all samples of the same non-failed device by the two networks are very close, then the non-failed device can be regarded as a pseudo-label task with reliable pseudo-labels. Therefore, in the kth iteration, the N pseudo-label tasks with the highest consistency of the two prediction results are selected to form the selected task set. Among them, the consistency is evaluated by cosine similarity, that is:

[0059]

[0060] In the formula, and are the calibrated pseudo-label vectors of all samples of device i output by the auxiliary network and the meta-network, respectively. When α is smaller, the consistency of the two sets of outputs is higher;

[0061] For the selected task, the final pseudo-label is the calibrated pseudo-label output by the meta-network, that is,

[0062] The remaining life prediction is specifically:

[0063] When the test equipment q runs to a certain moment before failure, its observable sample set is

[0064] Based on the observable sample set D q , use the auxiliary network to obtain its pseudo label, that is, ρ α (·) is the auxiliary network with parameter α;

[0065] Based on pseudo-label support set Obtain task-specific parameter sets through internal updates This results in a task-specific model, where n spt is the number of samples in the pseudo-label support set, and M is the number of parameters of the model;

[0066] Take the last observation sample As input, the output of the meta-network’s task-specific model is used as the final predicted remaining lifespan value, i.e.

[0067]

[0068] in, is the output RUL prediction value, gθ(·) is the task-shared feature extractor with parameter θ, is a task-specific regression layer for task q, and its parameters are

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] 1) The present invention designs a dual network structure so that meta-learning can be applied to learn task-specific global feature information and perform RUL prediction, that is, for non-failed devices, the auxiliary network generates pseudo labels so that the meta-network can be updated to the task-specific model. The meta-network can automatically learn the unique information of the device and achieve more accurate prediction.

[0071] 2) Designing the semi-supervised learning process as an iterative training process with a series of tasks added helps to generate reliable pseudo-labels. In each iteration, the outputs of the two networks are fused through the consistency criterion to select reliable pseudo-label tasks, which are added to the training task set. Then, the dual network is retrained based on the expanded training task set to improve the generalization performance of the network. The iteration stops when the pseudo-label tasks corresponding to all non-failed devices are added to the training task set.

[0072] 3) By adopting the technical solution provided by the present invention, it is helpful to use a small number of unlabeled data of failed devices and a sufficient number of unlabeled data of non-failed devices to train the model under a small sample, accurately predict the remaining life of the equipment, help operation and maintenance personnel make maintenance decisions, and avoid economic losses and safety hazards caused by sudden failure of equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a flow chart of the method of the present invention;

[0074] Figure 2 This is a framework diagram of incremental meta-learning of tasks in the present invention;

[0075] Figure 3 It is a framework diagram of the training process of the meta-network in the present invention;

[0076] Figure 4 The sensor data of five failed devices in a single operating condition and a single failure mode in one embodiment;

[0077] Figure 5 The sensor data of five failed devices in a multi-operating condition and multi-failure mode in one embodiment;

[0078] Figure 6The RUL prediction error of the method of the present invention and the comparative method under different proportions of the number of failed devices in the FD001 data set in an embodiment;

[0079] Figure 7 It is the RUL prediction error of the method of the present invention and the comparative method under different proportions of the number of failed devices in the FD004 data set in an embodiment. DETAILED DESCRIPTION

[0080] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0081] This embodiment takes the failure data set of aircraft turbofan engines as an example to provide a method for predicting the remaining life under small samples based on task incremental meta-learning. Figure 1 As shown, the following steps are included:

[0082] S1, data acquisition: obtaining the failure signals and actual remaining life of the aircraft engine's multi-sensor, and preprocessing and selecting the signals;

[0083] S2, network construction: construct a dual network, such as Figure 2 As shown, the dual network includes a meta-network and an auxiliary network. The inputs of the meta-network and the auxiliary network are sensor failure signals of multiple time steps of fixed length, and the output is the time interval between the last time step of the input and the failure moment, that is, the remaining life. The meta-network regards the mapping of the sensor data of each device to the remaining life as a regression task, and uses meta-learning to automatically learn the task-specific model of each device. The parameters of the task-specific model include the global characteristics of the device. The auxiliary network generates pseudo-labels for the unlabeled samples of the non-failed devices, so that the meta-network can update the task-specific model of the non-failed devices based on the pseudo-label samples.

[0084] S3, network training: Figure 3 As shown, the dual network is trained using a task increment training algorithm based on the labeled data of failed devices and the unlabeled data of non-failed devices;

[0085] S4, Remaining Life Prediction: The auxiliary network is used to predict the pseudo-labels of the test equipment samples, and the meta-network is updated to the task-specific model based on the pseudo-label support set, and the task-specific model is used to predict the remaining life of the test equipment.

[0086] The technical solution provided by the present invention can help solve the problem of poor generalization performance of the remaining life prediction deep learning model due to limited labeled data caused by the limited number of failed equipment, and realize the remaining life prediction of equipment, which can effectively improve the economy of maintenance and the stability and safety of flight.

[0087] 1. Data acquisition

[0088] Using S sensors to monitor the operating status of the equipment at time intervals m, a sample at time t contains c The sensor measurements of consecutive time steps, that is, from time step j to j+c-1 and c>0, then the input of the meta-network and auxiliary network is the matrix in is a column vector of sensor measurements at time step j, and the corresponding label y i，j is the regularized remaining life value corresponding to time step j, where for non-failed equipment, its failure time cannot be obtained, so its sample is unlabeled.

[0089] The sample set of failed equipment l is defined as The sample set of non-failed equipment u is The sample set of the test device q is where n l , n u and n q The purpose of the present invention is to train a model using labeled data of L failed devices and unlabeled data of U non-failed devices to predict the RUL of Q test devices.

[0090] This embodiment takes the failure data set of an aircraft gas turbine engine as an example. The data set is generated on the Commercial Modular Aviation Propulsion System Simulator (C-MAPSS) developed by NASA, and simulates the degradation process of a turbofan engine. The data set has four sub-data sets, and this embodiment uses the FD001 and FD004 sub-data sets for case studies. In FD001, the engine works under a single operating condition and has only one failure mode. In FD004, the engine works under six operating conditions and has two failure modes, namely fan damage and high-pressure compressor rupture. The degradation state of the aircraft engine is monitored by 21 sensor signals. The FD001 data set includes data from 100 training engines and data from 100 test engines. The FD004 data set includes data from 248 training engines and data from 249 test engines. The data of each training engine includes multi-sensor monitoring data collected from startup to failure, while the data of the test engine includes multi-sensor data and real RUL collected from the engine startup to a certain moment before failure.

[0091] In order to simulate the limited number of failed devices and sufficient number of non-failed devices in actual application scenarios, this embodiment extracts a certain proportion of engines from the training engines, retains the data collected from the start-up to the failure process, and uses it to simulate the failed engines that run to failure. For the remaining engines, their data are randomly intercepted, and only the process data from the start-up to a certain moment before failure is retained to simulate the non-failed engines that are repaired before failure. This embodiment uses the labeled data of the failed engines and the unlabeled data of the non-failed engines for model training, and uses all test engines to evaluate the prediction effect of the model.

[0092] In this embodiment, 14 sensor signals with obvious monotonic trends are selected from 21 sensor signals as inputs of the model. Figure 4 Data from startup to failure of the 14 sensors selected for the 5 training engines in the FD001 dataset. Figure 5 Data from startup to failure of the 14 sensors selected for the 5 training engines in the FD004 dataset.

[0093] 2. Hierarchical Meta-Network

[0094] The meta-network regards the mapping of sensor data of each device to RUL as a regression task. Since different devices have similar degradation processes and common degradation characteristics, the mapping of sensor data of different devices to RUL is a related task. On the other hand, different devices have different initial wear states, assembly errors, failure modes, and full life lengths, which make these tasks different. The meta-learning model can learn the model initialization parameters shared by multiple related tasks, and can update the initialization parameters to task-specific parameters through gradient descent based on the labeled samples of each task. The initialization parameters shared by the tasks contain common feature information of multiple devices, and the task-specific parameters contain global feature information specific to each device. Therefore, compared with the machine learning model, meta-learning can use common features and unique information to achieve accurate RUL prediction.

[0095] The meta-network consists of a task-shared feature extractor gθ(·) that learns common degenerate features and a task-specific regression layer. The remaining life prediction value is output using the equipment-specific global feature information, expressed as:

[0096]

[0097] Where X i，j is the input sensor data, y i，j is the output remaining life prediction value, g θ(·) is a task-shared feature extractor with parameter θ, is a task-specific regression layer for task i, with parameters w i To simplify the description, the task-specific model of the meta-network is denoted as f i (·).

[0098] The purpose of the meta-network is to learn meta-knowledge shared by the associated tasks, including the initialization parameters w0 of the task-specific regression layer and the parameters θ of the task-shared feature extractor; by gradient descent based on the support set, the initialization parameters w0 of the task-specific regression layer are updated to the task-specific parameters w i ; Meta-knowledge is learned by external updates based on query sets of multiple tasks, so that the learned feature extractor and initialization parameter w0 have good generalization ability, that is, the model f obtained by internal updates for different tasks i (·) have good prediction results. The support set and query set are two non-overlapping sub-datasets, which are composed of n of failed devices / non-failed devices / test devices. spt and n qry The sample consists of labeled / pseudo-labeled samples.

[0099] The training process of the meta-network is as follows:

[0100] (1) Task sampling: Sample T tasks from the training task set. From the dataset D i Sample n spt and n qry The samples constitute the support set and the query set. i is the sample set D of failed device l l , the sample set D of the non-failed equipment u u Or the sample set D of the test device q q ;

[0101] (2) Internal update: task-specific parameter w i Using the initialization parameter w0 as the initial value, perform n gradient descent updates based on the support set to obtain f i (·);

[0102] (3) External update: Update the distribution p(w0) of the initialization parameter w0 of the task-specific regression layer and the task-shared feature extractor parameters θ by gradient descent. The gradient is based on the updated f of all sampled tasks. i (·) The loss function is calculated on the query set.

[0103] The present invention further considers cognitive uncertainty and learns the distribution of initialization parameters w0 through external update and the distribution of parameters w1 through internal update. i The distribution of p(w i). Next, we will introduce the internal update and external update processes in detail.

[0104] i. Internal update

[0105] The purpose of the internal update is to update the initialization parameter distribution p(w0) to make it close to the posterior distribution of the task-specific parameters. This embodiment is implemented by Stein variational gradient descent (SVGD), where: is the task-specific parameter w i The posterior distribution of To support the set SVGD is a powerful nonparametric variational inference method that approximates the target distribution using a series of particles.

[0106] Example of keeping M model parameters based on Stein variational gradient descent Each instance is a particle. In the internal update, the update rule of the mth particle is as follows:

[0107]

[0108] Where, ∈ is the size of the update step; K(·, ·) is a positive definite kernel, and the square Euclidean distance function is selected in the present invention; The first term makes the particles approach the target distribution, and the second term prevents the particles from overlapping to one point; For particles The Del operator.

[0109] ii. External Update

[0110] The outer update optimizes the shared feature extractor parameters θ and the regression layer initialization parameter distribution p(w0) so that for the sampling task, the task-specific model f after the inner update i (·) has a good prediction effect on its query set. The present invention uses the following formula to evaluate the sampling task Prediction results on the query set:

[0111]

[0112] In the formula, is the conditional likelihood function of the remaining life prediction, since Stein’s variational gradient descent approximates the posterior distribution by keeping M examples The Monte Carlo approximation of formula (4) is:

[0113]

[0114] In the formula

[0115] Then, the loss function of the external update is:

[0116]

[0117] The external updated loss function minimizes the log-likelihood value of T sampling tasks. By optimizing this loss function, all the initialization parameter particles and feature extractor parameters are jointly optimized to achieve good generalization effect, so that for any associated task Its updated task-specific model f i (·) Accurate RUL prediction can be achieved.

[0118] Table 1 Hierarchical meta-network training algorithm

[0119]

[0120] 3. Task Incremental Training

[0121] In the present invention, a small sample scenario is defined as having only a limited number of failed devices. Since the training task set consists of a limited number of failed devices, the generalization ability of the meta-network obtained through training is limited. Therefore, the purpose of the present invention is to predict accurate pseudo-labels and select reliable pseudo-label tasks, add them to the training task set, and increase the number of training tasks to improve the generalization performance of the model. However, when using a meta-network to predict the RUL of samples of non-failed devices, the initialization parameters of the regression layer cannot be updated to the task-specific parameters through gradient descent. In order to solve this problem, the present invention designs a dual network structure, in which an auxiliary network is designed to generate pseudo-labels for non-failed devices, so that the meta-network can be internally updated based on the generated pseudo-labels and generate another set of pseudo-labels. In the present invention, the auxiliary network can be any machine learning model that can realize the output of RUL prediction values ​​based on input sensor data, which is expressed as Where ρα(·) is an auxiliary network with parameter α. Then, the present invention calibrates the two sets of pseudo labels and selects pseudo label tasks with high reliability to add to the training task set. The increased number of tasks will improve the generalization effect of the model.

[0122] The task increment training algorithm is specifically as follows: the entire training process of the dual network is designed as an iterative training process with a series of tasks added to generate accurate pseudo labels for unlabeled samples. In each iteration, the dual network generates pseudo labels for unlabeled samples of non-failed devices. The pseudo-label samples of a non-failed device constitute a pseudo-label task. The task increment training algorithm selects a preset number of pseudo-label reliable tasks from the pseudo-label tasks and adds them to the training task set. The dual network is retrained based on the expanded training task set. When all non-failed devices are added to the training task set, the iteration of the algorithm stops.

[0123] The kth iteration of the task incremental training algorithm includes the following steps:

[0124] (1) Based on the training task set Train the auxiliary network parameters α and predict the unlabeled task set Pseudo labels of all task samples in , where, for the first iteration, the training task set The unlabeled task set consists of L labeled samples of failed devices. It consists of U unlabeled samples of non-failed devices;

[0125] (2) Based on the training task set Train the meta-network to obtain the shared feature extractor parameters θ k and regression layer initialization parameters w k，0 , then for the unlabeled task set For all tasks in , the meta-network is based on the pseudo-labeled support set Update to get the task-specific model f i (·) and predict another set of pseudo labels Where X i，j ∈D i , Pseudo labels in the pseudo label support set;

[0126] (3) Use the pseudo-label calibration algorithm to calibrate the two sets of pseudo-labels and Perform calibration;

[0127] (4) Select reliable pseudo-label tasks based on the pseudo-label task selection algorithm to form the selected task set And update the training task set and unlabeled task set like Then update k = k + 1, if Then the algorithm ends.

[0128] Task-incremental learning improves the generalization ability of the model by adding tasks with reliable pseudo-labels to the training dataset and retraining the model in each iteration, while generating more accurate pseudo-labels for the remaining unlabeled tasks for subsequent iterations. Therefore, the task-incremental model can continuously improve the prediction accuracy of the model as the number of iterations increases.

[0129] Table 2 Task incremental training algorithm

[0130]

[0131]

[0132] 4. Pseudo-label calibration

[0133] Due to changing working conditions and measurement noise, the outputs of the meta-network and auxiliary network usually contain prediction errors and deviate from the true RUL. The meta-network is updated based on the support set of pseudo-labels and updated based on the pseudo-label tasks outside. Therefore, inaccurate pseudo-labels will lead to error accumulation and increase the prediction error of the network. Therefore, the present invention proposes a pseudo-label calibration algorithm to calibrate the outputs of the auxiliary network and the meta-network. The algorithm uses the linear relationship between the RUL of the same device at different time steps to calibrate the pseudo-label of each non-failed device, and uses the calibrated prediction value as the pseudo-label. The design of the algorithm mainly relies on two ideas:

[0134] First, there is a theoretical linear relationship between samples of the same device at time steps jk and j:

[0135] yi,jk=yi,j + km (7)

[0136] Where m is the time interval.

[0137] Second, the RUL prediction value of the corresponding sample when the equipment is close to failure is more reliable, because the sensor data close to failure has a clearer degradation trend and smaller measurement noise. i The relationship between the samples of and the samples of time step j is globally calibrated, that is:

[0138]

[0139] In the formula, is the global calibration result, is the time step n i 's pseudo-labels.

[0140] On the other hand, there is also a linear relationship between the RUL prediction values ​​of adjacent time steps. The present invention uses SG (Savizkg-Golag) filtering for local calibration. SG filtering is a data smoothing method based on local least squares polynomial approximation. Local calibration uses 2n c +1 adjacent time step, i.e. The remaining life prediction value is fitted with a linear model, and the result of local calibration is:

[0141]

[0142] In the formula, is the local calibration result, β0 and β are the model coefficients of SG filtering.

[0143] The final calibration result of the pseudo-label calibration algorithm is:

[0144]

[0145] Where λ is a hyperparameter that balances global calibration and local calibration. is the pseudo label after calibration.

[0146] Global calibration can filter out prediction errors caused by sensor measurement errors, but the results of global calibration lack robustness because they only rely on the prediction results of the last time step. Local calibration linearly fuses the prediction results of adjacent time steps, saves the local prediction results, and can filter out prediction errors caused by changing working conditions.

[0147] 5. Pseudo-label task selection

[0148] The pseudo-label calibration algorithm (10) is used to calibrate the outputs of the auxiliary network and the meta-network to obtain the calibrated remaining life prediction values. and Although the output of the network has been calibrated, in the first few iterations, there is very little labeled data used to train the dual network, which limits the generalization ability of the model and the predicted pseudo-labels still contain errors. Directly adding all of these pseudo-labeled non-failed devices to the training task set will lead to the accumulation of model errors. Therefore, the present invention adopts a pseudo-label task selection algorithm to select only a small number of pseudo-label tasks with reliable pseudo-labels to be added to the training data in each iteration. Considering that the auxiliary network and the meta-network have learned different degradation characteristics, if the predicted RUL values ​​of all samples of the same non-failed device by the two networks are very close, then the non-failed device can be regarded as a pseudo-label task with reliable pseudo-labels. Therefore, in the kth iteration, the N pseudo-label tasks with the highest consistency of the two prediction results are selected to form the selected task set. Among them, the consistency is evaluated by cosine similarity, that is:

[0149]

[0150] In the formula, and are the calibrated pseudo-label vectors of all samples of device i output by the auxiliary network and the meta-network, respectively. When α is smaller, the consistency of the two sets of outputs is higher;

[0151] For the selected task, the final pseudo-label is the calibrated pseudo-label output by the meta-network, that is,

[0152] 6. Remaining life prediction

[0153] When the test equipment q runs to a certain moment before failure, its observable sample set is

[0154] Based on the observable sample set D q, use the auxiliary network to obtain pseudo labels, that is, ρ α (·) is the auxiliary network with parameter α;

[0155] Based on pseudo-label support set The task-specific parameter set is obtained by formula (2): This results in a task-specific model, where n spt is the number of samples in the pseudo-label support set, and M is the number of parameters of the model;

[0156] Take the last observation sample As input, the output of the meta-network’s task-specific model is used as the final predicted remaining lifespan value, i.e.

[0157]

[0158] in, is the output RUL prediction value, gθ(·) is the task-shared feature extractor with parameter θ, is a task-specific regression layer for task q, and its parameters are

[0159] In order to evaluate the effectiveness of the proposed model, this paper uses the root mean square error as the indicator of RUL prediction. The true RUL value of the test aircraft engine q is y q , the predicted value is Then the RMSE is:

[0160]

[0161] The smaller the RMSE, the better the model effect.

[0162] Figure 6 The RMSE of the model of the proposed method and five comparative methods when the proportion of failed devices in the FD001 dataset is different is shown. Figure 7 The RMSE of the model and five comparison methods for different proportions of failed devices in the FD004 dataset are shown. The five comparison methods include: net al. in 2019, Self-supervision proposed by Krokotsch et al. of the Technical University of Berlin in 2021, Concurrent proposed by Wang et al. of Dalian University of Technology in 2022, Denoising proposed by Wang et al. of Xi'an University of Posts and Telecommunications in 2023, and Pseudo labeling proposed by Takayama et al. of the University of Tokyo in 2023. The research objectives of these methods are the same as those of the present invention, which is to solve the remaining life prediction when there are only a small number of labeled data of failed devices and a large number of unlabeled data of non-failed devices. Compared with the comparative methods (i.e., RBM, Self-supervision, Concurrent, Denoising and Pseudo labeling), the proposed model has a smaller RMSE when using the FD001 and FD004 datasets, and when the proportion of the number of failed devices is different, indicating that the proposed model has better diagnostic performance in small sample scenarios.

[0163] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for predicting remaining useful life under small samples based on task incremental meta-learning, characterized in that: The following steps are involved: S1, data acquisition: obtaining the failure signals and actual remaining life of the aircraft engine's multi-sensor, and preprocessing and selecting the signals; S2, network construction: construct a dual network, the dual network includes a meta-network and an auxiliary network, the input of the meta-network and the auxiliary network are sensor failure signals of multiple time steps of fixed length, and the output is the time interval between the last time step of the input and the failure moment, that is, the remaining life; wherein the meta-network regards the mapping of sensor data of each device to the remaining life as a regression task, and uses meta-learning to automatically learn the task-specific model of each device, the parameters of the task-specific model include the global feature information of the device; the auxiliary network generates pseudo-labels for the unlabeled samples of non-failed devices, so that the meta-network can update the task-specific model of non-failed devices based on the pseudo-label samples; S3, network training: train the dual network using the task increment training algorithm based on the labeled data of failed devices and the unlabeled data of non-failed devices; S4, Remaining Life Prediction: The auxiliary network is used to predict the pseudo-labels of the test equipment samples, and the meta-network is updated to the task-specific model based on the pseudo-label support set, and the task-specific model is used to predict the remaining life of the test equipment.

2. According to claim 1, a method for predicting remaining useful life under small samples based on task incremental meta-learning is characterized in that: The step S1 is specifically as follows: using S sensors to monitor the operating status of the equipment at time intervals m, a sample at time t contains sensor measurements of c consecutive time steps, that is, from time step j to j+c-1 and c>0, then the input of the meta-network and the auxiliary network is the matrix in is a column vector of sensor measurements at time step j, and the corresponding label y i,j is the regularized remaining life value corresponding to time step j, where the non-failed equipment samples are unlabeled; The sample set of failed equipment l is defined as The sample set of non-failed equipment u is The sample set of the test device q is Where n l , b u and n q are the sample numbers of failed equipment l, non-failed equipment u and tested equipment q respectively.

3. According to claim 1, a method for predicting remaining useful life under small samples based on task incremental meta-learning is characterized in that: The meta-network consists of a task-shared feature extractor g and a task-specific regression layer. θ (·) Learn common degenerate features, task-specific regression layers The remaining life prediction value is output using the equipment-specific global feature information, expressed as: Where X i,j is the input sensor data, y i,j is the output remaining life prediction value, g θ (·) is a task-shared feature extractor with parameter θ, is a task-specific regression layer for task i, with parameters w i , f i (·) represents the task-specific model of the meta-network.

4. The method for predicting remaining life under small sample based on task incremental meta-learning according to claim 3 is characterized in that: The purpose of the meta-network is to learn meta-knowledge shared by the associated tasks, including the initialization parameters w0 of the task-specific regression layer and the parameters θ of the task-shared feature extractor; by gradient descent based on the support set, the initialization parameters w0 of the task-specific regression layer are updated to the task-specific parameters w i ;Meta-knowledge is learned by external updates based on query sets of multiple tasks; The support set and query set are two non-overlapping sub-datasets, which are composed of n of failed devices / non-failed devices / test devices. spt and n qry It consists of samples with labels / pseudo labels; The training process of the meta-network is as follows: Task sampling: Sample T tasks from the training task set. From the dataset D i Sample n apt and n qrt The samples constitute the support set and the query set. i is the sample set D of failed device l l , the sample set D of the non-failed equipment u u Or the sample set D of the test device q q ; Internal update: task-specific parameter w i Using the initialization parameter w0 as the initial value, perform n gradient descent updates based on the support set to obtain f i (·); External update: Update the distribution p(w0) of the initialization parameter w0 of the task-specific regression layer and the task-shared feature extractor parameters θ by gradient descent, the gradient of which is based on the updated f of all sampled tasks i (·) The loss function is calculated on the query set.

5. The method for predicting remaining useful life under small samples based on task incremental meta-learning according to claim 4 is characterized in that: The purpose of the internal update is to update the initialization parameter distribution p(w0) to make it close to the posterior distribution of task-specific parameters It is implemented using Stein's variational gradient descent, where is the task-specific parameter w i The posterior distribution of To support the set The likelihood distribution of ; Example of keeping M model parameters based on Stein variational gradient descent Each instance is a particle. In the internal update, the update rule of the mth particle is as follows: Where ∈ is the size of the update step, K(·,·) is the positive definite kernel, For particles The Del operator.

6. The method for predicting remaining useful life under small samples based on task incremental meta-learning according to claim 5, characterized in that: The external update uses the following evaluation sampling task Prediction results on the query set: In the formula, is the conditional likelihood function of the remaining life prediction, since Stein’s variational gradient descent approximates the posterior distribution by keeping M examples The Monte Carlo approximation of the above formula for evaluating prediction performance is: In the formula Then, the loss function of the external update is: The externally updated loss function minimizes the log-likelihood value of the T sampling tasks. By optimizing the loss function, all the initialization parameter particles and feature extractor parameters are jointly optimized.

7. The method for predicting remaining useful life under small samples based on task incremental meta-learning according to claim 1, characterized in that: The task increment training algorithm is specifically as follows: the entire training process of the dual network is designed as an iterative training process with a series of tasks added, so as to generate accurate pseudo labels for unlabeled samples. In each iteration, the dual network generates pseudo labels for unlabeled samples of non-failed devices. The pseudo-label samples of a non-failed device constitute a pseudo-label task. The task increment training algorithm selects a preset number of tasks with reliable pseudo labels from the pseudo-label tasks and adds them to the training task set. The dual network is retrained based on the expanded training task set. When all non-failed devices are added to the training task set, the iteration of the algorithm stops. The kth iteration of the task incremental training algorithm includes the following steps: Based on the training task set Train the auxiliary network parameters α and predict the unlabeled task set Pseudo labels of all task samples in , where, for the first iteration, the training task set The unlabeled task set consists of L labeled samples of failed devices. It consists of U unlabeled samples of non-failed devices; Based on the training task set Train the meta-network to obtain the shared feature extractor parameters θ k and regression layer initialization parameters w k,0 , then for the unlabeled task set For all tasks in , the meta-network is based on the pseudo-labeled support set Update to get the task-specific model f i (·) and predict another set of pseudo labels Where X i,j ∈D i , Pseudo labels in the pseudo label support set; The pseudo-label calibration algorithm is used to calibrate the two sets of pseudo-labels. and Perform calibration; Select reliable pseudo-label tasks to form a selected task set based on pseudo-label task selection algorithm And update the training task set and unlabeled task set like Then update k = k + 1, if Then the algorithm ends.

8. The method for predicting remaining useful life under small samples based on task incremental meta-learning according to claim 7, characterized in that: The pseudo-label calibration algorithm is specifically as follows: the output of the model is calibrated using the potential linear correlation between the remaining life of the same device at multiple time steps, and the calibrated prediction value is used as the pseudo-label. The calibration process includes global calibration and local calibration; wherein, Based on the last time step n i The relationship between the samples of and the samples of time step j is globally calibrated, that is: In the formula, is the global calibration result, m is the time interval, is the time step n i Pseudo labels of Use SG filtering for local calibration, using 2n c +1 adjacent time step, i.e. The remaining life prediction value is fitted with a linear model, and the result of local calibration is: In the formula, is the local calibration result, β0 and β are the model coefficients of SG filtering; The final calibration result of the pseudo-label calibration algorithm is: Where λ is a hyperparameter that balances global calibration and local calibration. is the pseudo label after calibration.

9. The method for predicting remaining useful life under small sample based on task incremental meta-learning according to claim 8, characterized in that: The pseudo-label task selection algorithm is specifically as follows: using the pseudo-label calibration algorithm to calibrate the outputs of the auxiliary network and the meta-network to obtain the calibrated remaining life prediction values. and In the kth iteration, select the N pseudo-label tasks with the highest consistency between the two prediction results to form the selected task set Among them, the degree of consistency is evaluated by cosine similarity, that is: In the formula, and are the calibrated pseudo-label vectors of all samples of device i output by the auxiliary network and the meta-network, respectively. When α is smaller, the consistency of the two sets of outputs is higher; For the selected task, the final pseudo-label is the calibrated pseudo-label output by the meta-network, that is, 10. The method for predicting remaining useful life under small samples based on task incremental meta-learning according to claim 1, characterized in that: The remaining life prediction is specifically: When the test equipment q runs to a certain moment before failure, its observable sample set is Based on the observable sample set D q , use the auxiliary network to obtain its pseudo label, that is, ρ α (·) is the auxiliary network with parameter α; Based on pseudo-label support set Obtain task-specific parameter sets through internal updates This results in a task-specific model, where n spt is the number of samples in the pseudo-label support set, and M is the number of parameters of the model; Take the last observation sample As input, the output of the meta-network’s task-specific model is used as the final predicted remaining lifetime value, i.e. in, is the output remaining life prediction value, g θ (·) is a task-shared feature extractor with parameter θ, is a task-specific regression layer for task q, and its parameters are

Images