A small sample residual life prediction method based on task incremental meta-learning

Abstract

The application relates to a small-sample residual life prediction method based on task incremental meta-learning, which comprises the following steps: acquiring a multi-element sensor failure signal and an actual residual life of an airplane engine, and pre-processing and selecting the signal; constructing a dual network, including a meta network and an auxiliary network, wherein the meta network regards the mapping of sensor data of each device to the residual life as a regression task, and automatically learns a task-specific model of each device by using meta learning; the auxiliary network generates pseudo labels for unlabeled samples of non-failed devices; the dual network is trained based on labeled data of failed devices and unlabeled data of non-failed devices by using a task incremental training algorithm; the pseudo labels of test device samples are predicted by using the auxiliary network, the meta network is updated to a task-specific model based on a pseudo label support set, and the residual life of the test device is predicted by using the task-specific model. Compared with the prior art, the application can realize accurate residual life prediction under a small number of failure samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of equipment remaining life prediction, and in particular to a remaining life prediction method under small sample based on task incremental meta-learning. BACKGROUND

[0002] With the development of sensor technology, multi-sensor signals have been widely used to monitor the degradation state of equipment and predict the remaining useful life of equipment. Based on the diagnosis results, the maintenance personnel can reasonably arrange the maintenance plan to avoid the problems such as production delay, safety hazard and so on caused by sudden failure of equipment, and reduce the maintenance cost and improve the maintenance efficiency.

[0003] A deep learning model is a neural network composed of multiple interconnected nodes, which can automatically learn the characteristics of data and take the characteristics as input, and output the corresponding degradation state, such as health index or remaining useful life (RUL). The remaining useful life prediction method based on deep learning usually needs a large amount of labeled data to train the model to achieve satisfactory generalization effect. However, in actual application scenarios, labeled data can only be obtained when the equipment fails, and the equipment is usually repaired or replaced before failure. Therefore, under normal circumstances, only a small amount of labeled data is generated from a small number of failed equipment, and a large amount of unlabeled data is generated from a large number of non-failed equipment. Therefore, training a deep learning model using a small amount of labeled data can be regarded as a small sample problem.

[0004] For a deep learning model, another problem is that only local sensor data is used as input of the model, and the model extracts local features from the input and outputs the predicted RUL value, while ignoring the global features of the equipment. Generally, the deep learning-based method divides the sensor signal into multiple fixed-length time series using a moving window. Each fixed-length time series is used as input of a sample, which contains local degradation information of a device in a specific time period, such as sensor measurement value, degradation rate, and degradation curvature. However, the RUL value corresponding to a time series also depends on the unique characteristics of the equipment, such as initial wear state, assembly error, failure mode and total life length. These unique characteristics differ on different equipment and play a decisive role in the remaining useful life of the equipment. SUMMARY

[0005] The purpose of the present application is to provide a remaining life prediction method under small sample based on task incremental meta-learning, which trains the model using a small amount of labeled data from failed equipment and a large amount of unlabeled data from non-failed equipment, and accurately predicts the RUL of the running test equipment.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] The present invention provides a method for predicting remaining useful life under small samples based on task incremental meta-learning, comprising: (1) proposing a dual network structure: the dual network consists of a meta-network and an auxiliary network. The meta-network regards the mapping of sensor data of each device to RUL as a regression task, and uses meta-learning to automatically learn the task-specific model of each device. The parameters of these models contain the global characteristics of the device and can help to achieve accurate RUL prediction. The auxiliary network generates pseudo-labels for the unlabeled samples of the non-failed devices, so that the meta-model can update the task-specific model of the non-failed devices based on the pseudo-label samples. (2) proposing a task incremental training algorithm: the entire training process is designed as a series of iterative training processes with task additions to generate accurate pseudo-labels for unlabeled samples. In each iteration, the dual network generates pseudo-labels for the unlabeled samples of the non-failed devices. The pseudo-label samples of one non-failed device constitute a pseudo-label task. The algorithm selects a certain number of tasks with reliable pseudo-labels from the pseudo-label tasks and adds them to the training task set. The model will be retrained based on the expanded training task set. The model trained based on the expanded training task set has better generalization ability and can provide more reliable pseudo labels for the remaining unlabeled samples. When all non-failed devices are added to the training task set, the iteration of the algorithm stops. (3) A pseudo label calibration algorithm is proposed: the input sensor data usually contains noise caused by changing working conditions and sensor measurement errors, which makes the RUL prediction values ​​output by the meta-network and auxiliary network contain prediction errors and deviate from the true values. Using these erroneous RUL prediction values ​​as pseudo label samples for training the model will lead to error accumulation, so the network output needs to be calibrated. The designed calibration algorithm uses the potential linear correlation between the RULs of multiple time steps to calibrate the output of the model, and then uses the calibrated prediction values ​​as pseudo labels. (4) A pseudo label task selection algorithm is proposed: the meta-learning and auxiliary networks in the dual network structure learn different degradation characteristics. Therefore, it is assumed that for a certain non-failed device, the closer the RUL outputs of all unlabeled samples of the same non-failed device are to the two networks, the more reliable the pseudo label of the task can be. Ultimately, the dual network is trained using a limited number of labeled samples from failed devices and a sufficient number of pseudo-labeled samples from non-failed devices. As the number of tasks in the training set increases, the dual network's generalization performance across different devices continues to improve. For the test device, the meta-network uses the pseudo-labels generated by the auxiliary network to update the task-specific model and then predict its RUL. The specific steps are as follows:

[0008] S1, data acquisition: obtain the failure signal and actual remaining life of the aircraft engine's multi-sensor, and pre-process and select the signal;

[0009] S2, network construction: a dual network is constructed, the dual network includes a meta network and an auxiliary network, the inputs of the meta network and the auxiliary network are sensor failure signals of a plurality of time steps of fixed length, and the output is the time interval between the last time step of the input and the failure time, that is, the remaining useful life; wherein the meta network regards the mapping of the sensor data of each device to the remaining useful life as a regression task, and automatically learns the task-specific model of each device by using meta learning, parameters of the task-specific model contain global feature information of the device; the auxiliary network generates pseudo labels for unlabeled samples of non-failed devices, so that the meta network can update the task-specific model of the non-failed devices based on the pseudo label samples;

[0010] S3, network training: the dual network is trained based on the labeled data of the failed devices and the unlabeled data of the non-failed devices by using a task incremental training algorithm;

[0011] S4, remaining useful life prediction: the pseudo labels of the test device samples are predicted by using the auxiliary network, the meta network is updated to the task-specific model based on the pseudo label support set, and the remaining useful life of the test device is predicted by using the task-specific model.

[0012] The step S1 is specifically: the running state of the device is monitored by using S sensors at a time interval m, a sample at time t includes c consecutive time step sensor measurement values, that is, from time step j to j+c-1 and c>0, and the input of the meta network and the auxiliary network is a matrix Wherein is a column vector composed of the sensor measurement value at time step j, and the corresponding label y i，j is the regularized remaining useful life value corresponding to time step j, wherein the sample of the non-failed device is unlabeled;

[0013] The sample set of the failed device l is defined as The sample set of the non-failed device u is The sample set of the test device q is Wherein n l , n u and n q are the number of samples of the failed device l, the non-failed device u and the test device q respectively. The purpose of the present application is to train the model by using the labeled data of L failed devices and the unlabeled data of U non-failed devices, and to predict the RUL of Q test devices.

[0014] The meta-network takes the mapping of sensor data of each device to RUL as a regression task. Since different devices have similar degradation processes and common degradation features, the mapping of sensor data of different devices to RUL is a related task. On the other hand, different devices have different initial wear states, assembly errors, failure modes and total life lengths, etc., so these tasks have differences. The meta-learning model can learn the model initialization parameters shared by multiple related tasks, and can update the initialization parameters to task-specific parameters based on the labeled samples of each task through gradient descent. The initialization parameters shared by the task contain common feature information of multiple devices, and the task-specific parameters contain global feature information specific to each device. Therefore, compared with the machine learning model, the meta-learning can use common features and specific information to achieve accurate RUL prediction.

[0015] The meta-network is composed of a task-shared feature extractor and a task-specific regression layer. The task-shared feature extractor gθ(·) learns common degradation features, and the task-specific regression layer The device-specific global feature information is used to output the remaining life prediction value, which is represented as:

[0016]

[0017] where X i，j is the input sensor data, y i，j is the output remaining life prediction value, g θ (·) is a task-shared feature extractor with parameters θ, is the task-specific regression layer of task i, whose parameters are w i , and f i (·) represents the task-specific model of the meta-network.

[0018] The purpose of the meta-network is to learn the meta-knowledge shared by related tasks, including the initialization parameters w0 of the task-specific regression layer and the parameters θ of the task-shared feature extractor. Through gradient descent based on the support set, the initialization parameters w0 of the task-specific regression layer are updated to the task-specific parameters w i through inner update; the meta-knowledge is learned by outer update based on the query set of multiple tasks, so that the learned feature extractor and the initialization parameters w0 have good generalization ability, that is, the models f i (·) obtained by inner update of different tasks all have good prediction effect; the support set and the query set are two non-overlapping sub-data sets, which are composed of n spt and n qry labeled / pseudo-labeled samples of failed devices / non-failed devices / test devices, respectively.

[0019] The training process of the meta-network is as follows:

[0020] Task sampling: sample T tasks from the training task set, for each task From the dataset D i , sample n spt and n qry samples to form the support set and query set, respectively, where the dataset D i is the sample set D l of failed devices l, the sample set D u of non-failed devices u, or the sample set D q of test devices q;

[0021] Inner update: update the task-specific parameter w i based on the support set with n gradient descent updates with the initial parameter w0 as the initial value to obtain f i (·);

[0022] Outer update: update the distribution p(w0) of the initial parameter w0 of the task-specific regression layer and the task-shared feature extractor parameter θ by gradient descent, which is based on the loss function calculated on the query set by the updated f i (·) of all sampled tasks.

[0023] The present application further considers the cognitive uncertainty and learns the distribution of the initial parameter w0 through outer update and the distribution p(w i ) of the parameter w i through inner update. Next, the inner update and outer update processes will be specifically introduced.

[0024] The purpose of the inner update is to update the initial parameter distribution p(w0) to approximate the posterior distribution of the task-specific parameter It is realized by Stein variational gradient descent (SVGD), wherein, is the posterior distribution of the task-specific parameter w i , and is the likelihood distribution of the support set , and SVGD is a powerful non-parametric variational inference method that uses a series of particles to approximate the target distribution.

[0025] Based on Stein variational gradient descent, M instances of model parameters are maintained Each instance is a particle, and in the inner update, the update rule of the mth particle is as follows:

[0026]

[0027] In the formula, ∈ is the size of the update step; K(·, ·) is a positive definite kernel, and in the present application, the squared Euclidean distance function is selected; The first term makes the particles approach the target distribution, and the second term makes the particles not overlap to a point; For the particles of the Del operator.

[0028] The outer update optimizes the shared feature extractor parameters θ and the regression layer initialization parameter distribution p(w0), so that for the sampled task, the task-specific model f i (·) has good prediction effect on its query set, the present application uses the following formula to evaluate the sampled task The prediction effect on the query set:

[0029]

[0030] In the formula, is the conditional likelihood function of the remaining useful life prediction, since in the Stein variational gradient descent, M examples are kept to approximate the posterior distribution The Monte Carlo approximation of the above formula for evaluating the prediction effect is:

[0031]

[0032] In the formula

[0033] Then, the loss function of the outer update is:

[0034]

[0035] The loss function of the outer update minimizes the log-likelihood value of the T sampled tasks, by optimizing the loss function, all initialization parameter particles and feature extractor parameters are jointly optimized to achieve good generalization effect, so that for any related task Its updated task-specific model f i (·) can achieve accurate RUL prediction.

[0036] In the present application, a small sample scenario is defined as only a limited number of failed devices. Since the training task set is composed of a limited number of failed devices, the generalization ability of the meta-network trained is limited. Therefore, the purpose of the present application is to predict accurate pseudo-labels and select reliable pseudo-label tasks to add to the training task set, thereby increasing the number of training tasks to improve the generalization performance of the model. However, when predicting the RUL of non-failed device samples with the meta-network, the initialization parameters of the regression layer cannot be updated to task-specific parameters through gradient descent. To solve this problem, the present application designs a dual network structure, in which an auxiliary network is designed to generate pseudo-labels for non-failed devices, so that the meta-network can perform inner updates based on the generated pseudo-labels and generate another set of pseudo-labels. In the present application, the auxiliary network can be any machine learning model that can input sensor data and output RUL prediction values, which is represented as where is an auxiliary network with parameter. Then, the present application calibrates the two sets of pseudo-labels and selects pseudo-label tasks with high reliability to add to the training task set, and the increased number of tasks will improve the generalization effect of the model.

[0037] The task incremental training algorithm is specifically: the entire training process of the dual network is designed as a series of task-incremental iterative training processes, which realizes generating accurate pseudo-labels for unlabeled samples. In each iteration, the dual network generates pseudo-labels for unlabeled samples of non-failed devices, and the pseudo-label samples of a non-failed device constitute a pseudo-label task. The task incremental training algorithm selects a preset number of pseudo-label reliable tasks from the pseudo-label tasks to add to the training task set, and re-trains the dual network based on the expanded training task set. When all non-failed devices are added to the training task set, the iteration of the algorithm stops.

[0038] The kth iteration of the task incremental training algorithm includes the following steps:

[0039] Based on the training task set Train the auxiliary network parameter and predict the unlabeled task set The pseudo-labels of all task samples in, where, for the first iteration, the training task set is composed of labeled samples of L failed devices, and the unlabeled task set is composed of unlabeled samples of U non-failed devices;

[0040] Based on the training task set Train the meta-network to obtain the shared feature extractor parameter and the regression layer initialization parameter k and k，0 , and then for all tasks in the unlabeled task set , the meta-network generates another set of pseudo-labels based on the pseudo-label support set updating the task-specific model f i (·), and predicting another set of pseudo-labels where X i，j ∈D i , is a pseudo-label in the pseudo-label support set;

[0041] calibrating the two sets of pseudo-labels and using a pseudo-label calibration algorithm;

[0042] selecting reliable pseudo-label tasks to form a selected task set based on a pseudo-label task selection algorithm and updating the training task set and the unlabeled task set If then updating k = k + 1, if the algorithm ends.

[0043] Task incremental learning improves the generalization ability of the model by adding pseudo-label reliable tasks to the training data set in each iteration and retraining the model, and generates more accurate pseudo-labels for the remaining unlabeled tasks for subsequent iterations. Therefore, the task incremental model can continuously improve the prediction accuracy of the model as the number of iterations increases.

[0044] Due to changing operating conditions and measurement noise, the outputs of the main network and the auxiliary network usually contain prediction errors, deviating from the true RUL. The main network is updated based on the support set of the pseudo-label, and is updated based on the pseudo-label task, so inaccurate pseudo-labels will lead to error accumulation and increase the prediction error of the network. Therefore, the present application proposes a pseudo-label calibration algorithm to calibrate the outputs of the auxiliary network and the main network, which calibrates the pseudo-labels of each non-failed device using the linear relationship between the RULs of the same device at different time steps, and uses the calibrated prediction value as the pseudo-label. The design of this algorithm mainly relies on two ideas:

[0045] First, there is a theoretical linear relationship between the samples of the same device at time steps j-k and j as follows:

[0046] yi,j-k=yi,j + km

[0047] In the formula, m is the time interval.

[0048] Second, the RUL prediction value of the corresponding sample is more reliable when the device is close to failure, because the sensor data at the time close to failure has a clearer degradation trend and smaller measurement noise. Therefore, based on the last time step n iThe relationship between the sample of the time step j and the sample of the time step n is globally calibrated, that is:

[0049]

[0050] In the formula, is the global calibration result, is the pseudo label of the time step n i .

[0051] In another aspect, the RUL prediction values of adjacent time steps also have a linear relationship, and the present application uses SG (Savitzkg-Golag) filtering for local calibration. SG filtering is a data smoothing method based on local least square polynomial approximation. Local calibration uses 2n c +1 adjacent time steps, that is, the remaining life prediction value of the time step n

[0052]

[0053] In the formula, is the local calibration result, and β0 and β are the model coefficients of SG filtering.

[0054] The final calibration result of the pseudo label calibration algorithm is:

[0055]

[0056] In the formula, λ is a hyperparameter for balancing global calibration and local calibration, is the calibrated pseudo label.

[0057] Global calibration can filter the prediction error caused by sensor measurement error, but since it only depends on the prediction result of the last time step, the result of global calibration lacks robustness. Local calibration linearly integrates the prediction results of adjacent time steps, preserves the local prediction result, and can filter the prediction error caused by changing working conditions.

[0058] The output of the auxiliary network and the meta network calibrated by the pseudo label calibration algorithm respectively obtains the calibrated remaining life prediction value and Although the output of the network is calibrated, in the first few iterations, the label data used to train the dual network is very small, which limits the generalization ability of the model, and the predicted pseudo-labels still contain errors. Directly adding all the non-failed devices of these pseudo-labels to the training task set will cause the accumulation of model errors. Therefore, the pseudo-label task selection algorithm is adopted in the present application, and only a small number of pseudo-label reliable pseudo-label tasks are selected in each iteration to join the training data. Considering that the auxiliary network and the meta network learn different degradation features, therefore, if the predicted RUL values of all samples of the same non-failed device by the two networks are very close, the non-failed device can be regarded as a pseudo-label reliable pseudo-label task. Therefore, in the kth iteration, the N pseudo-label tasks with the highest consistency of the two prediction results are selected to form the selected task set wherein the consistency degree is evaluated by the cosine similarity, that is:

[0059]

[0060] wherein, and are the calibrated pseudo-label vectors of all samples of device i output by the auxiliary network and the meta network respectively, and the smaller the alpha is, the higher the consistency of the two sets of outputs is;

[0061] For the selected task, the final pseudo-label is the calibrated pseudo-label output by the meta network, that is

[0062] The remaining life prediction is specifically:

[0063] At a certain time before the failure of the test device q, the observable sample set is

[0064] Based on the observable sample set D q , the pseudo-label of the test device q is obtained by using the auxiliary network, that is ρ α (·) is the auxiliary network with the parameter alpha;

[0065] Based on the pseudo-label support set The task-specific parameter set is obtained by inner update to obtain the task-specific model, wherein n spt is the number of samples in the pseudo-label support set, and M is the number of parameters of the model;

[0066] The last observed sample is taken as the input, and the output of the task-specific model of the meta network is taken as the final predicted remaining life value, that is

[0067]

[0068] wherein, is the output residual life prediction value, gθ(·) is a task-shared feature extractor with parameters θ, is a task-specific regression layer of task q, and its parameters are

[0069] Compared with the prior art, the present application has the following beneficial effects:

[0070] 1) The present application designs a dual network structure so that meta-learning can be applied to learn task-specific global feature information and make RUL prediction, i.e. for non-failed devices, the auxiliary network generates pseudo labels so that the meta network can be updated to a task-specific model. The meta network can automatically learn the specific information of the device to achieve more accurate prediction.

[0071] 2) The semi-supervised learning process is designed as an iterative training process of a series of tasks, which helps to generate reliable pseudo labels. In each iteration process, the outputs of the two networks are fused by a consistency criterion to select reliable pseudo label tasks, which are added to the training task set, and then the dual network is retrained based on the expanded training task set to improve the generalization performance of the network. The iteration stops when all the pseudo label tasks corresponding to the non-failed devices are added to the training task set.

[0072] 3) By using the technical solution provided by the present application, it is helpful to train the model using a small amount of unlabeled data of failed devices and a sufficient amount of unlabeled data of non-failed devices under small sample conditions, accurately predict the residual life of the device, help maintenance personnel make maintenance decisions, and avoid economic losses and safety hazards caused by sudden failure of the device. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1 is the flow chart of the method of the present application;

[0074] Figure 2 is the framework diagram of task incremental meta-learning in the present application;

[0075] Figure 3 is the training process framework diagram of the meta network in the present application;

[0076] Figure 4 is the sensor data of 5 failed devices under a single working condition and a single failure mode in an embodiment;

[0077] Figure 5 is the sensor data of 5 failed devices under multiple working conditions and multiple failure modes in an embodiment;

[0078] Figure 6RUL prediction error of the method of the present application and the comparative method in the embodiment under different proportions of the number of failed devices in the FD001 data set;

[0079] Figure 7 RUL prediction error of the method of the present application and the comparative method in the embodiment under different proportions of the number of failed devices in the FD004 data set. DETAILED DESCRIPTION

[0080] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present application, and gives a detailed implementation and specific operation process, but the protection scope of the present application is not limited to the following embodiments.

[0081] The present embodiment takes the failure data set of an aircraft turbofan engine as an example, and provides a small sample residual life prediction method based on task incremental meta-learning, as shown in Figure 1 The method comprises the following steps:

[0082] S1, data acquisition: acquiring the multi-element sensor failure signals and the actual residual life of the aircraft engine, and pre-processing and selecting the signals;

[0083] S2, network construction: constructing a dual network, as shown in Figure 2 The dual network comprises a meta network and an auxiliary network, and the inputs of the meta network and the auxiliary network are fixed-length multi-time-step sensor failure signals, and the output is the time interval between the last time step of the input and the failure time, i.e. the residual life. Wherein, the meta network regards the mapping of the sensor data of each device to the residual life as a regression task, and uses meta-learning to automatically learn the task-specific model of each device, and the parameters of the task-specific model contain the global features of the device; the auxiliary network generates pseudo labels for unlabeled samples of non-failed devices, so that the meta network can update the task-specific model of non-failed devices based on the pseudo label samples;

[0084] S3, network training: as shown in Figure 3 The dual network is trained based on the labeled data of the failed devices and the unlabeled data of the non-failed devices using a task incremental training algorithm;

[0085] S4, residual life prediction: the auxiliary network is used to predict the pseudo labels of the test device samples, the meta network is updated to the task-specific model based on the pseudo label support set, and the task-specific model is used to predict the residual life of the test device.

[0086] By adopting the technical solutions provided by the application, the problem that the generalization performance of the remaining useful life prediction deep learning model is poor due to the limited number of invalid devices and the limited labeled data, and the remaining useful life prediction of the device is realized, and the economy of maintenance and the stability and safety of flight can be effectively improved.

[0087] 1. Data acquisition

[0088] The running state of the device is monitored by using S sensors at a time interval m, and one sample at time t includes c sensor measurement values of c consecutive time steps, that is, from time step j to j+c-1 and c>0, and the input of the primary network and the auxiliary network is a matrix wherein is a column vector composed of the sensor measurement values at time step j, and the corresponding label y i，j is the regularized remaining useful life value corresponding to time step j, wherein, for the device that has not failed, the failure time cannot be obtained, and therefore the sample is unlabeled.

[0089] The sample set of the failed device l is defined as The sample set of the non-failed device u is The sample set of the test device q is wherein n l , n u and n q are the number of samples of the failed device l, the non-failed device u and the test device q respectively. The purpose of the application is to train the model by using the labeled data of L failed devices and the unlabeled data of U non-failed devices to predict the RUL of Q test devices.

[0090] This embodiment takes the failure data set of an aircraft gas turbine engine as an example, which is generated on the commercial modular aircraft propulsion system simulator (C-MAPSS) developed by NASA, simulating the degradation process of a turbofan engine. The data set has four sub-data sets, and this embodiment uses the FD001 and FD004 sub-data sets for case study. In FD001, the engine works in a single operating condition and has only one failure mode. In FD004, the engine works in six operating conditions and has two failure modes, i.e. fan damage and high-pressure compressor rupture. The degradation state of the aircraft engine is monitored by 21 sensor signals. The FD001 data set includes data of 100 training engines and data of 100 test engines. The FD004 data set includes data of 248 training engines and data of 249 test engines. The data of each training engine includes multi-sensor monitoring data collected during the startup to failure process, and the data of the test engine includes multi-sensor data collected during the startup to a certain time before failure and the true RUL.

[0091] To simulate the actual application scenario where the number of failed devices is limited and the number of non-failed devices is sufficient, the embodiment extracts a certain proportion of engines from the training engines, retains the data collected from the start to the failure process, and uses it to simulate the failed engines running to failure. For the remaining engines, the data is randomly truncated to retain only the process data from the start to a certain time before failure, which is used to simulate non-failed engines repaired before failure. The embodiment will use the labeled data of failed engines and the unlabeled data of non-failed engines for model training, and use all test engines to evaluate the prediction effect of the model.

[0092] The embodiment selects 14 sensor signals with obvious monotonic trends from the 21 sensor signals as inputs of the model. Figure 4 Data from the start to the failure process of the selected 14 sensors of the 5 training engines in the FD001 data set. Figure 5 Data from the start to the failure process of the selected 14 sensors of the 5 training engines in the FD004 data set.

[0093] 2, Hierarchical Meta-network

[0094] The meta-network regards the mapping of sensor data of each device to RUL as a regression task. Since different devices have similar degradation processes and common degradation features, the mapping of sensor data of different devices to RUL is a related task. On the other hand, different devices have different initial wear states, assembly errors, failure modes, and total life lengths, etc., which make these tasks different. The meta-learning model can learn the model initialization parameters shared by multiple related tasks, and can update the initialization parameters to task-specific parameters based on the labeled samples of each task through gradient descent. The initialization parameters shared by the task contain common feature information of multiple devices, and the task-specific parameters contain global feature information specific to each device, so compared with machine learning models, meta-learning can use common features and specific information to achieve accurate RUL prediction.

[0095] The meta-network consists of a task-shared feature extractor and a task-specific regression layer, the task-shared feature extractor gθ(·) learns common degradation features, and the task-specific regression layer Outputs the remaining life prediction value using the global feature information specific to the device, denoted as:

[0096]

[0097] Where X i，j is the input sensor data, y i，j is the output remaining life prediction value, g θis a task-shared feature extractor with parameter is a task-specific regression layer of task i with parameter i For simplicity, the task-specific model of the meta-network is denoted as f i (·) with parameter

[0098] The purpose of the meta-network is to learn the meta-knowledge associated with the task sharing, including the initialization parameter w0 of the task-specific regression layer and the parameter of the task-shared feature extractor; through gradient descent based on the support set, the initialization parameter w0 of the task-specific regression layer is updated to the task-specific parameter w i through inner update; the meta-knowledge is learned by outer update based on the query set of multiple tasks, so that the learned feature extractor and the initialization parameter w0 have good generalization ability, that is, the models f i (·) obtained by inner update of different tasks all have good prediction effect. The support set and the query set are two non-overlapping sub-datasets, respectively composed of n spt and n qry labeled / pseudo-labeled samples of failed devices / non-failed devices / test devices.

[0099] The training process of the meta-network is as follows:

[0100] (1) Task sampling: sampling T tasks from the training task set, for the task from the data set D i , respectively sampling n spt and n qry samples to form the support set and the query set, wherein the data set D i is the sample set D l of the failed device l, the sample set D u of the non-failed device u or the sample set D q of the test device q;

[0101] (2) Inner update: the task-specific parameter w i is updated to f i (·) based on n times of gradient descent of the initialization parameter w0 of the support set;

[0102] (3) Outer update: updating the distribution p(w0) of the initialization parameter w0 of the task-specific regression layer and the parameter of the task-shared feature extractor through gradient descent, which is based on the loss function calculated on the query set of the updated f i (·) of all sampled tasks.

[0103] The present application further considers the cognitive uncertainty and learns the distribution of the initialization parameter w0 through outer update and the distribution p(w i of the parameter w i). Next, the inner update and the outer update processes will be introduced in detail.

[0104] i. Inner update

[0105] The purpose of the inner update is to update the initialization parameter distribution p(w0) to approximate the posterior distribution of the task-specific parameter This embodiment is implemented by Stein variational gradient descent (SVGD) descent, in which, is the posterior distribution of the task-specific parameter w i . is the likelihood distribution of the support set . SVGD is a powerful non-parametric variational inference method, which uses a series of particles to approximate the target distribution.

[0106] Based on Stein variational gradient descent, M instances of model parameters are maintained Each instance is a particle, and in the inner update, the update rule of the mth particle is as follows:

[0107]

[0108] In the formula, ∈ is the size of the update step; K(·, ·) is a positive definite kernel, and in the present application, the squared Euclidean distance function is selected; The first term of the formula makes the particle approximate the target distribution, and the second term makes the particle not overlap to a point; is the Del operator about the particle .

[0109] ii. Outer update

[0110] The outer update optimizes the shared feature extractor parameter θ and the regression layer initialization parameter distribution p(w0), so that for the sampled task, the task-specific model f i (·) has good prediction effect on its query set, and the present application uses the following formula to evaluate the prediction effect of the sampled task on the query set:

[0111]

[0112] In the formula, is the conditional likelihood function of the remaining life prediction, and since M examples are maintained in the Stein variational gradient descent to approximate the posterior distribution The Monte Carlo approximation of formula (4) is:

[0113]

[0114] In the formula

[0115] ​Then, the loss function of the outer update is:

[0116]

[0117] The loss function of the outer update minimizes the log-likelihood of T sampled tasks, and by optimizing the loss function, all the initialization parameter particles and feature extractor parameters are jointly optimized to achieve good generalization, so that for any related task The updated task-specific model f i (·) can achieve accurate RUL prediction.

[0118] Table 1 hierarchical meta-network training algorithm

[0119]

[0120] 3. Task incremental training

[0121] In the present application, the small sample scenario is defined as only a limited number of failed devices. Since the training task set is composed of a limited number of failed devices, the generalization ability of the meta-network trained is limited. Therefore, the purpose of the present application is to predict accurate pseudo-labels and select reliable pseudo-label tasks to add to the training task set, increase the number of training tasks to improve the generalization performance of the model. However, when the meta-network is used to predict the RUL of non-failed device samples, the initialization parameters of the regression layer cannot be updated to task-specific parameters through gradient descent. In order to solve this problem, the present application designs a dual network structure, in which an auxiliary network is designed to generate pseudo-labels for non-failed devices, so that the meta-network can perform inner update based on the generated pseudo-labels and generate another set of pseudo-labels. In the present application, the auxiliary network can be any machine learning model that can output RUL prediction values from input sensor data, which is represented as where is an auxiliary network with parameter. Then, the present application calibrates the two sets of pseudo-labels and selects pseudo-label tasks with high reliability to add to the training task set, and the increased number of tasks will improve the generalization effect of the model.

[0122] The task incremental training algorithm is as follows: the entire training process of the dual network is designed as a series of task-incremental iterative training processes, which realizes the generation of accurate pseudo-labels for unlabeled samples. In each iteration, the dual network generates pseudo-labels for unlabeled samples of non-failed devices, and the pseudo-label samples of a non-failed device constitute a pseudo-label task. The task incremental training algorithm selects a preset number of pseudo-label reliable tasks from the pseudo-label tasks to add to the training task set, and re-trains the dual network based on the expanded training task set. When all non-failed devices are added to the training task set, the iteration of the algorithm stops.

[0123] where the k-th iteration of the task incremental training algorithm comprises the following steps:

[0124] (1) Based on the training task set Train the auxiliary network parameter a and predict the pseudo-labels for the unlabeled task set where for the first iteration, the training task set consists of labeled samples from L failed devices, and the unlabeled task set consists of unlabeled samples from U non-failed devices.

[0125] (2) Based on the training task set Train the meta-network to obtain the shared feature extractor parameter k and the regression layer initialization parameter w k，0 , and then for all tasks in the unlabeled task set , the meta-network updates the task-specific model f i (·) based on the pseudo-label support set and predicts another set of pseudo-labels where X i，j ∈ D i , is the pseudo-label in the pseudo-label support set.

[0126] (3) Calibrate the two sets of pseudo-labels and using the pseudo-label calibration algorithm.

[0127] (4) Select reliable pseudo-label tasks to form the selected task set based on the pseudo-label task selection algorithm, and update the training task set and the unlabeled task set If , update k = k + 1, and if , the algorithm ends.

[0128] Task incremental learning improves the generalization ability of the model by adding pseudo-label reliable tasks to the training data set in each iteration and retraining the model, and generates more accurate pseudo-labels for the remaining unlabeled tasks for subsequent iterations. Therefore, the task incremental model can continuously improve the prediction accuracy of the model as the number of iterations increases.

[0129] Table 2 Task incremental training algorithm

[0130]

[0131]

[0132] 4. Pseudo-label calibration

[0133] Due to varying working conditions and measurement noise, the outputs of the meta-network and the auxiliary network usually contain prediction errors, deviating from the true RUL. The meta-network updates based on the support set of pseudo-labels, and updates outside the task based on pseudo-labels, so inaccurate pseudo-labels will lead to error accumulation, increasing the prediction error of the network. Therefore, the present application proposes a pseudo-label calibration algorithm to calibrate the outputs of the auxiliary network and the meta-network, which calibrates the pseudo-labels of each non-failed device using the linear relationship between the RULs of the same device at different time steps, and uses the calibrated prediction value as the pseudo-label. The design of this algorithm mainly relies on two ideas:

[0134] First, there is a theoretical linear relationship between the samples of the same device at time steps j-k and j as follows:

[0135] yi,j-k=yi,j + km (7)

[0136] In the formula, m is the time interval.

[0137] Second, the RUL prediction value of the corresponding sample is more reliable when the device is close to failure, because the sensor data at the time of failure has a clearer degradation trend and smaller measurement noise. Therefore, based on the relationship between the sample at the last time step n i and the sample at time step j, global calibration is performed, that is:

[0138]

[0139] In the formula, is the global calibration result, is the pseudo-label at time step n i .

[0140] On the other hand, the RUL prediction values of adjacent time steps also have a linear relationship, and the present application uses SG (Savizkg-Golag) filtering for local calibration. SG filtering is a data smoothing method based on local least squares polynomial approximation. Local calibration uses the remaining life prediction values of 2n c +1 adjacent time steps, that is, to fit a linear model, and the result of local calibration is:

[0141]

[0142] In the formula, is the local calibration result, and β0 and β are the model coefficients of SG filtering.

[0143] The final calibration result of the pseudo-label calibration algorithm is:

[0144]

[0145] where λ is a hyperparameter balancing global calibration and local calibration, is the calibrated pseudo-label.

[0146] Global calibration can filter the prediction error caused by sensor measurement error, but due to the dependence on the prediction result of the last time step, the result of global calibration lacks robustness. Local calibration linearly fuses the prediction results of adjacent time steps, preserves the local prediction results, and can filter the prediction error caused by changing working conditions.

[0147] 5. Pseudo-label task selection

[0148] The output of the auxiliary network and the meta network is calibrated by the pseudo-label calibration algorithm (10) to obtain the remaining useful life prediction value after calibration and Although the output of the network is calibrated, in the initial several iterations, the label data used to train the dual network is very small, which limits the generalization ability of the model, and the predicted pseudo-label still contains errors. Therefore, the present application adopts a pseudo-label task selection algorithm, which selects only a small amount of pseudo-label reliable pseudo-label tasks to add to the training data in each iteration. Considering that the auxiliary network and the meta network learn different degradation features, therefore, if the predicted RUL values of all samples of the same non-failed device by the two networks are very close, the non-failed device can be regarded as a pseudo-label reliable pseudo-label task. Therefore, in the kth iteration, the N pseudo-label tasks with the highest consistency of the two prediction results are selected to form the selected task set where the consistency degree is evaluated by the cosine similarity, that is:

[0149]

[0150] where and are the calibrated pseudo-label vectors of all samples of device i output by the auxiliary network and the meta network respectively, and the smaller the α is, the higher the consistency of the two sets of outputs is;

[0151] For the selected task, the final pseudo-label is the calibrated pseudo-label output by the meta network, that is

[0152] 6. Remaining useful life prediction

[0153] Test device q runs to a certain time before failure, and its observable sample set is

[0154] Based on the observable sample set D q, the pseudo-label is obtained by the auxiliary network, i.e. ρ α (·) is the auxiliary network with parameters α;

[0155] Based on the pseudo-label support set The task-specific parameter set is obtained by formula (2) The task-specific model is obtained, wherein n spt is the number of samples in the pseudo-label support set, and M is the number of parameters of the model;

[0156] The last observation sample is taken as input, and the output of the task-specific model of the meta-network is taken as the final predicted remaining life value, i.e.

[0157]

[0158] wherein, is the output remaining life prediction value, gθ(·) is the task-shared feature extractor with parameters θ, is the task-specific regression layer of task q, and the parameters thereof are

[0159] In order to evaluate the effectiveness of the proposed model, the root mean square error is used as the index of RUL prediction. The true RUL value of the test aircraft engine q is y q , and the predicted value is The RMSE is:

[0160]

[0161] The smaller the RMSE is, the better the model is.

[0162] Figure 6 The RMSE of the proposed method and five comparison methods when the proportion of different failure equipment in the FD001 data set is shown. Figure 7 The RMSE of the proposed method and five comparison methods when the proportion of different failure equipment in the FD004 data set is shown. The five comparison methods include: Ellefse nRBM proposed by Krokotsch et al. in 2019, Self-supervision proposed by Wang et al. in 2022, Concurrent proposed by Wang et al. in 2022, Denoising proposed by Wang et al. in 2023, and Pseudo labeling proposed by Takayama et al. in 2023. The research purposes of these methods are the same as the research purpose of the present application, which are to solve the remaining useful life prediction when there are only a small amount of labeled data of failed devices and a large amount of unlabeled data of non-failed devices. Compared with the comparative methods (i.e., RBM, Self-supervision, Concurrent, Denoising and Pseudo labeling), the proposed model has smaller RMSE when using FD001 and FD004 data sets and when the proportion of different failed devices is different, indicating that the proposed model has better diagnosis performance in the small sample scene.

[0163] The preferred embodiments of the present application are described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solutions obtained by logical analysis, reasoning or limited experiments based on the prior art according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A method for predicting remaining useful life under small sample conditions based on task incremental meta-learning, characterized in that: The following steps are involved: S1, data acquisition: obtain the failure signal and actual remaining life of the aircraft engine's multi-sensor, and pre-process and select the signal; S2, Network Construction: Construct a dual network, which includes a meta-network and an auxiliary network. The inputs of the meta-network and the auxiliary network are both sensor failure signals of multiple time steps of fixed length, and the output is the time interval between the last input time step and the failure moment, i.e., the remaining life. The meta-network regards the mapping of sensor data for each device to the remaining life as a regression task and uses meta-learning to automatically learn a task-specific model for each device. The parameters of the task-specific model include the global feature information of the device. The auxiliary network generates pseudo-labels for unlabeled samples of non-failed devices, so that the meta-network can update the task-specific model of non-failed devices based on the pseudo-label samples. S3, network training: The dual network is trained using the task-incremental training algorithm based on the labeled data of the failed device and the unlabeled data of the remaining devices; S4, Remaining Life Prediction: The auxiliary network is used to predict the pseudo labels of the test equipment samples. The meta-network is updated to the task-specific model based on the pseudo-label support set, and the task-specific model is used to predict the remaining life of the test equipment; The purpose of the meta-network is to learn meta-knowledge shared by related tasks, including the initialization parameters of the task-specific regression layer Share the parameters of the feature extractor with the task ; Initialization parameters of the task-specific regression layer by gradient descent based on the support set By internally updating to task-specific parameters ; Meta-knowledge is learned by external updating of query sets based on multiple tasks; the support set and query set are two non-overlapping sub-datasets, which are respectively composed of failed devices / non-failed devices / test devices and It consists of labeled / pseudo-labeled samples; The training process of the meta-network is as follows: Task sampling: sampling from the training task set tasks, for which , from the dataset Sampling and The samples constitute the support set and the query set. For expired devices Sample set , non-failed equipment Sample set or test equipment Sample set ; Internal update: task-specific parameters To initialize the parameters The initial value is based on the support set Sub-gradient descent update ; External update: Update the initialization parameters of the task-specific regression layer through gradient descent Distribution Share feature extractor parameters with tasks , the gradient is based on the updated The loss function is calculated on the query set; The purpose of the internal update is to update the initialization parameter distribution Make it close to the posterior distribution of task-specific parameters , implemented using Stein's variational gradient descent, where Task-specific parameters The posterior distribution of To support the set The likelihood distribution of Based on Stan's variational gradient descent, keep An instance of a model parameter , each instance is a particle, in the internal update, The update rule for each particle is as follows: Where, is the update step size, is a positive definite nucleus, For particles Del operator; The external update uses the following formula to evaluate the sampling task Prediction results on the query set: Where, is the conditional likelihood function of the remaining life prediction, since Stein’s variational gradient descent is performed by keeping Approximating the posterior distribution with examples , the Monte Carlo approximation of the above formula for evaluating the prediction effect is: In the formula ; Then, the loss function of the external update is: The outer updated loss function is minimized The log-likelihood value of the sampling task, by optimizing the loss function, all the initialization parameter particles and feature extractor parameters are jointly optimized; The task-increment training algorithm specifically comprises the following steps: designing the entire training process of the dual network as a series of iterative training processes with added tasks to achieve accurate pseudo-label generation for unlabeled samples. In each iteration, the dual network generates pseudo-labels for unlabeled samples of non-failed devices. Pseudo-label samples of a non-failed device constitute a pseudo-label task. The task-increment training algorithm selects a preset number of tasks with reliable pseudo-labels from the pseudo-label tasks and adds them to the training task set. The dual network is retrained based on the expanded training task set. When all non-failed devices are added to the training task set, the algorithm iteration stops. Among them, the task incremental training algorithm The iteration consists of the following steps: Based on the training task set Training auxiliary network parameters , and predict the unlabeled task set Pseudo labels of all task samples in , where, for the first iteration, the training task set Depend on The unlabeled task set consists of labeled samples of failed devices. Depend on The unlabeled samples consist of 100 non-failed devices; Based on the training task set Train the meta-network to obtain shared feature extractor parameters and regression layer initialization parameters , then for the unlabeled task set For all tasks in the meta-network, the pseudo-label support set Updated mission-specific models , and predict and generate another set of pseudo labels ,in , , Pseudo labels in the pseudo label support set; The two sets of pseudo labels are calibrated using the pseudo label calibration algorithm. and Perform calibration; Select reliable pseudo-label tasks to form the selected task set based on the pseudo-label task selection algorithm , and update the training task set and unlabeled task sets ,like Update ,like Then the algorithm ends; The pseudo-label calibration algorithm specifically comprises the following steps: utilizing the potential linear correlation between the remaining lifetimes of the same device at multiple time steps to calibrate the model output, and using the calibrated predicted values ​​as pseudo-labels. The calibration process includes global calibration and local calibration. Based on the last time step Samples and time steps The relationship between samples is globally calibrated, namely: Where, is the global calibration result, is the time interval, is the time step Pseudo labels of Use SG filtering for local calibration, using adjacent time steps, i.e. , the remaining life prediction value is fitted with a linear model, and the result of local calibration is: Where, is the local calibration result, and is the model coefficient of SG filtering; The final calibration result of the pseudo-label calibration algorithm is: Where, To balance the hyperparameters of global and local calibration, is the pseudo label after calibration.

2. The method for predicting remaining useful life under small sample conditions based on task incremental meta-learning according to claim 1, characterized in that: The step S1 is specifically as follows: Sensors at time intervals Monitor the operating status of the equipment, one The sample at the moment contains The sensor measurements of consecutive time steps, i.e., from time step arrive and , then the input of the meta-network and auxiliary network is the matrix ,in is the time step The sensor measurements are composed of a column vector, and the corresponding labels is the time step The corresponding regularized remaining life value, where the non-failed equipment samples are unlabeled; Defining Deactivated Devices The sample set is , non-failed equipment The sample set is , test equipment The sample set is ,in , and Deactivated devices , non-failed equipment and test equipment The number of samples.

3. The method for predicting remaining useful life under small sample conditions based on task incremental meta-learning according to claim 1, characterized in that: The meta-network consists of a task-shared feature extractor and a task-specific regression layer. Learning common degradation features, task-specific regression layers The remaining life prediction value is output using the equipment's unique global feature information, which is expressed as: in is the input sensor data, is the output remaining life prediction value, The parameter is The task-shared feature extractor, It's a task The task-specific regression layer is parameterized by , Representing task-specific models of meta-networks.

4. The method for predicting remaining useful life under small sample conditions based on task incremental meta-learning according to claim 1, characterized in that: The pseudo-label task selection algorithm is specifically as follows: using the pseudo-label calibration algorithm to calibrate the outputs of the auxiliary network and the meta-network to obtain the calibrated remaining life prediction values. and , in In the iteration, the two prediction results with the highest consistency are selected. pseudo-label tasks to form a selection task set , where the degree of consistency is evaluated by cosine similarity, namely: Where, and Devices for auxiliary network and meta-network output respectively The calibrated pseudo-label vectors of all samples are The smaller it is, the higher the consistency between the two sets of outputs; For the selected task, the final pseudo-label is the calibrated pseudo-label output by the meta-network, that is, , , .

5. The method for predicting remaining useful life under small sample conditions based on task incremental meta-learning according to claim 1, characterized in that: The remaining life prediction is specifically as follows: Test equipment Running to a certain moment before failure, its observable sample set is ; Based on observable sample sets , use the auxiliary network to obtain its pseudo label, that is, , , The parameters are Auxiliary network; Based on pseudo-label support set , obtain the task-specific parameter set through internal update , thus obtaining a task-specific model, where is the number of samples in the pseudo-label support set, is the number of parameters of the model; Take the last observation sample As input, the output of the meta-network’s task-specific model is used as the final predicted remaining lifetime value, i.e. in, is the output remaining life prediction value, The parameter is The task-shared feature extractor, It's a task The task-specific regression layer is parameterized by .

Images