A DIformer-based method for predicting the remaining useful life of aircraft engines

Abstract

The present invention discloses a method for predicting the remaining useful life of an aircraft engine based on a dual-channel Transformer with dimensionally independent patch embedding. The method comprises the following steps: data preprocessing, constructing a DIformer neural network model, constructing a training and validation method and performing model training, and using the model to predict the remaining useful life of an aircraft engine. The DIformer model proposed in the present invention adds feature extraction of multivariate sensor dimensions to the traditional Transformer model, so that the DIformer model achieves lower prediction errors in typical multivariate time series prediction problems for aircraft engine remaining useful life prediction, strengthens the feature extraction capability of aircraft engine multivariate sensors, and effectively improves the accuracy of the model in predicting the remaining useful life of an aircraft engine.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and aircraft engine prognostics and health management (PHM), and in particular to a method for predicting the remaining useful life of an aircraft engine based on DIformer. Background Art

[0002] Aircraft engine Remaining Useful Life (RUL) prediction has long been considered a core technology for prediction and health management. Advanced aircraft engine RUL prediction methods can eliminate potential problems before failures occur, further reducing aircraft accident rates, and support maintenance decisions throughout the engine's life cycle, greatly reducing operating and maintenance costs. It is a key technology urgently needed for development in the aviation industry and a significant indicator of advanced intelligent engines.

[0003] Remaining useful life (RUL) refers to the lifespan of a randomly degraded device from its current state until failure. Based on the device's past operating state and historical data from similar devices under the same operating conditions, the RUL prediction is used to predict the device's useful life from its current state until failure. Therefore, the prediction method developed in this context is applicable not only to aircraft engine RUL prediction, but also to mechanical equipment with similar random degradation properties.

[0004] The Transformer architecture is a neural network architecture proposed by the Google team in 2017 for natural language processing. After years of development, the Transformer architecture has evolved into multiple variants, applied to fields beyond natural language processing, such as computer vision, speech signal processing, and multimodal applications. In the field of multivariate time series prediction, many Transformer variants have also been developed to solve various practical problems. In the area of aircraft engine remaining useful life prediction, researchers have extracted time-step features from multivariate sensor data to predict engine life. However, research on extracting both time-step features and sensor features from multivariate sensor data is currently lacking.

[0005] From the above description of the aircraft engine RUL prediction problem, it can be seen that the current RUL value of the engine is mainly determined by the signals of different sensors at different previous time steps. Therefore, making full use of the information of different time steps and various sensors plays a vital role in RUL prediction. In addition, the multi-source sensor information at different time steps contains important features in two dimensions: the time step dimension and the sensor dimension. That is to say, different time steps and different sensors have different degrees of influence on the engine RUL. In order to better extract the features of the engine CBM data in the time step dimension and the sensor dimension at the same time, the present invention provides a dual-channel Transformer (DIformer) with dimension-independent patch embedding, and introduces a new dimension-independent patch embedding method to effectively capture the dependency of the embedded segment in the two dimensions of sensor and time step. Summary of the Invention

[0006] The present invention provides a DIformer-based aircraft engine RUL prediction method, which provides a new technical solution for current aircraft engine RUL prediction.

[0007] The technical solution of the present invention is as follows:

[0008] A method for predicting the remaining useful life of an aircraft engine based on DIformer, characterized by comprising the following steps:

[0009] S1: Collect time series monitoring data from various sensors of the aircraft engine and label the collected time series data as the original data set for training the model;

[0010] S2: Preprocess the labeled multivariate time series data of aircraft engines and convert them into standardized time series feature vectors that meet the requirements of model training;

[0011] S3: Use sliding windows to process continuous standardized time series feature vectors and convert them into small batch samples required for model training;

[0012] S4: Build the DIformer model, set the model hyperparameter optimization algorithm, and train the model using small batch training samples as input;

[0013] S5: The optimization algorithm automatically selects and retains the model weight file with the smallest prediction error to obtain the optimized model;

[0014] S6: Input the validation set into the model for functional verification. If the validation result is close to the test result, the model training is completed, and the DIformer-based aircraft engine remaining useful life prediction model is obtained.

[0015] The present invention according to the above technical solution is characterized in that: in step S1, the collected time series is marked, and the data to be marked includes the engine ID and the number of engine working cycles. It should be noted that if the collected aircraft engine sensor data is not the full life cycle data of the aircraft engine, the remaining service life of the engine in the last working cycle needs to be additionally marked.

[0016] The present invention according to the above technical solution is characterized in that: in step S2, the labeled multivariate time series data of the aircraft engine needs to be preprocessed, and the specific steps of the preprocessing are as follows:

[0017] S2.1: Data cleaning. Assume that the complete engine degradation process includes N moments, and obtain the engine operation data collected by m sensors:

[0018]

[0019] The degradation sequence of the jth (j=1, 2,…, m) sensor is expressed as:

[0020]

[0021] Since the prediction of the remaining useful life of an aircraft engine requires the extraction of degradation features, the engine ID and engine operating cycle that are not related to the degradation features, as well as the sensor data whose parameters remain basically unchanged throughout the entire degradation cycle of the engine, are eliminated. These data have little contribution to the extraction of degradation features, and D sensor degradation data are finally retained:

[0022]

[0023] S2.2: Normalization. Perform Min-Max normalization on the input D sensor degradation data according to the formula:

[0024]

[0025] Normalize the data of each sensor, where x is the degraded data before normalization, x min is the minimum value of the sequence where x is located, x max is the maximum value of the sequence where x is located, and the normalized sensor data is:

[0026]

[0027] The normalized data of the jth sensor is:

[0028]

[0029] The present invention according to the above technical solution is characterized in that: in step S3, a sliding window is used to process the continuous normalized time series feature vectors, and sample features are pre-extracted to convert them into small batch samples required for model training. The specific steps are as follows:

[0030] S3.1: RUL generation. Assuming the original dataset is the engine full life cycle dataset, the number of engine working cycles is extracted from the original engine dataset, and the corresponding remaining useful life label is calculated using the following function:

[0031]

[0032] where cycle max The maximum value of the cycle in the sequence of the current engine ID is set. The maximum value of the remaining service life is set to 125, which is just a typical value. When solving different engine remaining service life prediction problems, it should be set according to the actual situation.

[0033] S3.2: Generate small batches of samples using a sliding window. Use the sliding window method to generate small batches of samples. Let the sliding window size be n. The small batch of samples can be expressed as:

[0034] X=[X 1 ,X 2 ,...,X D ]

[0035] where X i (i=1,2,...,D) is expressed as:

[0036]

[0037] Select the remaining useful life Y at the last moment of each sliding window RUL Serves as a label for the remaining useful life of the sample.

[0038] S3.3: Sample feature pre-extraction. Use the linear regression method to pre-extract features from the small batch of samples, and append the extracted linear coefficients coef and mean mean to the end of the small batch of samples:

[0039] coef i =LinearRegression(X i ).coef_

[0040] mean i =mean(X i )

[0041] Use X new Represents a small batch of samples after feature pre-extraction:

[0042]

[0043] in Expressed as:

[0044]

[0045] The present invention according to the above technical solution is characterized in that: in step S4, a DIformer model is built, model hyperparameters and its optimization algorithm are set, and small batch training samples are used as model input for training. The specific steps are as follows:

[0046] S4.1: Build DIformer model:

[0047] S4.1.1: Dimensionality-Independent Embedding (DI-Embedding):

[0048] Using a small batch sample generation method with independent embedding vectors for each dimension can capture cross-dimensional and cross-time dependencies at the same time. seg The segment size is the time window sample X of length T (s) To split, here we assume that T can be L seg Divisibility:

[0049]

[0050] in represents the i-th segment of the d-th sensor in the time window of length T, and inputs each segment into the embedding layer:

[0051]

[0052] in is the learnable matrix parameter;

[0053] S4.1.2: Multi-head Attention (MA):

[0054] Define a set of two-dimensional time series data with time dimension T and space dimension D: When the attention mechanism is applied to the time dimension, data of different spatial dimensions share weights and perform the same matrix operations. Take the time series data of the d-th dimension as an example:

[0055] The d-th dimension time series data after dimension-independent embedding is represented as d model To size the sequence dimension, first generate three matrices (Queries, Keys and Values) through the following calculations:

[0056] Q t =X t W t q

[0057] Where W t q , and are all learnable parameters, W t q and d k K t The vector dimension, Q t Need to be with K t The vector dimensions are consistent, d v V t The vector dimension of query Q t and key value Then calculate the scaled dot product of Q and K to get the probability of each time period in the dth dimension being worth noting

[0058]

[0059] Where softmax() is a normalized exponential function, K t The transpose of . t With V t Multiply and calculate the attention feature value of each time period

[0060] Attention t (Q t ,K t ,V t )=α t V t

[0061] MA t (.) is the multi-head attention mechanism function, which uses the multi-head attention mechanism MA (Multi-head Attention, MA) to allow the model to focus on information from different time periods:

[0062]

[0063] in, is a learnable parameter;

[0064] Similarly, we can apply the attention mechanism to the spatial dimension, share weights with different time dimensions, and express the attention mechanism calculation of the spatial dimension as: MA s ;

[0065] S4.1.3: Dual-channel attention DCA:

[0066] Embed the two-dimensional array obtained from the independent dimension As the input of DCA, L represents the number of segments obtained by dimensionally independent embedding in the time step dimension, D represents the total number of aircraft engine sensors used for prediction after preprocessing, and d model Indicates the vector dimension after dimension-independent embedding. For ease of reading, X :,d Represents the vector of all time steps of dimension d, 1≤d≤D, X l,: Represents the vector of all dimensions at the lth time step, 1≤l≤L. In DCA, X is the input vector received from the dimension-independent embedding; DCA consists of the temporal channel, the spatial channel, and the feature fusion layer, as follows:

[0067] 1) Temporal Channel

[0068] In the Temporal Channel, multi-head attention (MA) is applied to the time dimension and residually connected with the original input X to obtain Then it passes through a multi-layer perceptron (MLP) and finally with X t,1 Perform residual connection to get the output X of TemporalChannel t,out :

[0069]

[0070] x t,out =LayerNorm(X t,1 +MLP(X t，1 ))

[0071] Where 1≤d≤D, LayerNorm(·) represents the layer normalization widely used in time series forecasting, which can improve the convergence speed and generalization ability of the model;

[0072] 2) Spatial Channel

[0073] Similar to Temporal, Spatial Channel is specifically expressed as follows:

[0074]

[0075] X s,out =LayerNorm(X s，1 +MLP(X s，1 ))

[0076] 3) Feature Fusion

[0077] After dual-channel feature extraction, the features of the two dimensions are spliced to achieve feature fusion, and finally a linear layer is used to restore the dimension size:

[0078] X f，out =Linear(Concat(X t，out , X s,out ))

[0079] Concat means vector concatenation according to the last dimension, and Linear is a linear layer.

[0080] Y=DCA(X)

[0081] Y is DCA output;

[0082] S4.1.4: A dual-channel Transformer for dimension-independent patch embedding (DIformer):

[0083] DIformer consists of an encoder and a decoder, such as Figure 1 As shown, the encoder consists of dimension-independent embedding and DCA, and the structure of the decoder layer is as follows Figure 4 As shown;

[0084] 1) Encoder: The encoder includes dimension-independent embedding and multi-layer DCA. Dimension-independent embedding preserves the dimensional independence of the embedding vector. Multi-layer DCA is used to extract the deep features of the input vector in the time series and sensor dimensions. The encoder is represented as:

[0085]

[0086] Where Emb represents the dimension-independent patch embedding method, which is added to the position encoding vector PE to obtain the encoder's two-dimensional input vector X enc,0 or X enc,l , Linear represents a learnable linear layer used to merge the segmented vectors, and Concat represents vector concatenation;

[0087] 2) Decoder: The decoder takes as input the learnable positional encoding vector and the encoder vectors before and after DCA feature extraction. It then uses the sum of the outputs of the two decoding layers and passes through a linear layer to output the final prediction.

[0088] The decoder is represented as:

[0089]

[0090]

[0091] X3 dec，l =LayerNorm(X2 dec，l +X1 dec,l )

[0092] X dec，l =LayerNorm(X3 dec,l +MLP(X3 dec，l ))

[0093] in is the position encoding vector that the decoder can learn, τ is the length of the processed input sequence, if the input sequence length cannot be L seg If the length is not divisible by L, the sequence must be padded to the end until it can be divided by L. seg Divisible by X dec,l is the output of each layer of the decoder, X1 dec，l For DCA output, X2 dec，l X3 is the attention output after the decoder fuses the feature information from the encoder. dec，l X2 dec，l Residual connection of attention mechanism; d model is the vector dimension;

[0094] 3) Prediction layer (Predict): The prediction layer receives information from the decoder and uses the sum of layer-by-layer predictions as the final prediction value:

[0095]

[0096] Linear(.) represents a linear transformation function;

[0097] S4.2: Model performance evaluation criteria:

[0098] The root mean square error (RMSE) and the scoring function (Score) are used to evaluate the model performance, as follows:

[0099] 1)

[0100] 2)

[0101] Where n is the number of test samples, is the RUL predicted value of the i-th sample, yi is the true RUL value of the i-th sample, the lower the RMSE and Score values, the higher the prediction accuracy and the better the model performance.

[0102] S4.3: Set up model training and hyperparameter optimization methods:

[0103] S4.3.1: Set the training method:

[0104] The Adam optimizer was used to optimize the model parameters. The initial learning rate (Learning_rate) was set to 0.001, the mean squared error function (MSELoss) was used as the loss function, and the dynamic learning rate adjustment strategy (ReduceLROnPlateau) was used. During training, the loss function of the model on the validation set was monitored. Whenever the loss function did not decrease for eight consecutive epochs, the learning rate was reduced to better fine-tune the parameters.

[0105] S4.3.2: Model hyperparameter optimization methods:

[0106] The Optuna hyperparameter optimization framework is used to set the model hyperparameter search space. In order to reduce the uncertainty of test results caused by the randomness of model training, repeatable experiments are added to each optimization step. During the optimization process, the optimizer's monitoring indicator is the RMSE of the model on the test set.

[0107] The flow chart of model training and hyperparameter optimization method is as follows Figure 6 shown.

[0108] This patented method for predicting the remaining useful life of an aircraft engine uses a dimensionally independent embedding approach, which changes the way data is previously transmitted in the model. The dimensionally independent patch embedding allows the model to simultaneously focus on the characteristic information of the original monitoring data in any time period and on any sensor, thereby improving the spatiotemporal feature extraction capability of the model based on deep learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] The present invention will be further described below with reference to the accompanying drawings.

[0110] Figure 1 This is the overall flow chart of the remaining useful life prediction method based on DIformer;

[0111] Figure 2 Schematic diagram of DIformer dimension-independent embedding;

[0112] Figure 3 Schematic diagram of the DIformer dual-channel attention layer;

[0113] Figure 4 This is the internal expansion diagram of the DIformer decoder layer;

[0114] Figure 5 Schematic diagram of the DIformer multi-layer perceptron layer;

[0115] Figure 6 Flowchart of model training and hyperparameter optimization methods; DETAILED DESCRIPTION

[0116] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0117] See also Figure 1 —6. The present invention provides a method for predicting the remaining useful life of an aircraft engine based on DIformer, comprising the following steps:

[0118] S1: Collect time series monitoring data from various sensors of the aircraft engine and label the collected time series data as the original data set for training the model;

[0119] S2: Preprocess the labeled multivariate time series data of aircraft engines and convert them into standardized time series feature vectors that meet the requirements of model training;

[0120] S3: Use sliding windows to process continuous standardized time series feature vectors and convert them into small batch samples required for model training;

[0121] S4: Build the DIformer model, set the model hyperparameter optimization algorithm, and train the model using small batch training samples as input;

[0122] S5: The optimization algorithm automatically selects and retains the model weight file with the smallest prediction error to obtain the optimized model;

[0123] S6: Input the validation set into the model for functional verification. If the validation result is close to the test result, the model training is completed, and the DIformer-based aircraft engine remaining useful life prediction model is obtained.

[0124] In this embodiment, step S1 marks the collected time series. The data that need to be marked include the engine ID and the number of engine working cycles. It should be noted that if the collected aircraft engine sensor data is not the full life cycle data of the aircraft engine, the remaining service life of the engine in the last working cycle needs to be additionally marked.

[0125] In this embodiment, step S2 requires preprocessing the labeled multivariate time series data of the aircraft engine. The specific steps of the preprocessing are as follows:

[0126] S2.1: Data cleaning. Assume that the complete engine degradation process includes N moments, and obtain the engine operation data collected by m sensors:

[0127]

[0128] The degradation sequence of the jth (j=1, 2,…, m) sensor is expressed as:

[0129]

[0130] Since the prediction of the remaining useful life of an aircraft engine requires the extraction of degradation features, the engine ID and engine operating cycle that are not related to the degradation features, as well as the sensor data whose parameters remain basically unchanged throughout the entire degradation cycle of the engine, are eliminated. These data have little contribution to the extraction of degradation features, and D sensor degradation data are finally retained:

[0131]

[0132] S2.2: Normalization. Perform Min-Max normalization on the input D sensor degradation data according to the formula:

[0133]

[0134] Normalize the data of each sensor, where x is the degraded data before normalization, x min is the minimum value of the sequence where x is located, x max is the maximum value of the sequence where x is located, and the normalized sensor data is:

[0135]

[0136] The normalized data of the jth sensor is:

[0137]

[0138] In this embodiment, step S3 uses a sliding window to process the continuous normalized time series feature vectors, performs sample feature pre-extraction, and converts them into small batch samples required for model training. The specific steps are as follows:

[0139] S3.1: RUL generation. Assuming the original dataset is the engine full life cycle dataset, the number of engine working cycles is extracted from the original engine dataset, and the corresponding remaining useful life label is calculated using the following function:

[0140]

[0141] where cycle maxThe maximum value of the cycle in the sequence of the current engine ID is set. The maximum value of the remaining service life is set to 125, which is just a typical value. When solving different engine remaining service life prediction problems, it should be set according to the actual situation.

[0142] S3.2: Generate small batches of samples using a sliding window. Use the sliding window method to generate small batches of samples. Let the sliding window size be n. The small batch of samples can be expressed as:

[0143] X=[X 1 ,X 2 ,…,X D ]

[0144] where X i (i=1,2,…,D) is expressed as:

[0145]

[0146] Select the remaining useful life Y at the last moment of each sliding window RUL Serves as a label for the remaining useful life of the sample.

[0147] S3.3: Sample feature pre-extraction. Use the linear regression method to pre-extract features from the small batch of samples, and append the extracted linear coefficients coef and mean mean to the end of the small batch of samples:

[0148] coef i =LinearRegression(X i ).coef_

[0149] mean i =mean(X i )

[0150] Use X new Represents a small batch of samples after feature pre-extraction:

[0151]

[0152] in Expressed as:

[0153]

[0154] In this embodiment, step S4 is to build a DIformer model, set model hyperparameters and its optimization algorithm, and use small batch training samples as model input for training. The specific steps are as follows:

[0155] S4.1: Build DIformer model:

[0156] S4.1.1: Dimension-Independent Embedding (DI-Embedding):

[0157] Using a small batch sample generation method with independent embedding vectors for each dimension can capture cross-dimensional and cross-time dependencies at the same time. seg The segment size divides the time window sample X of length T, where it is assumed that T can be divided by L seg Divisibility:

[0158]

[0159] X i,d ={x t，d |(i-1)×L seg <t<i×L seg}

[0160] in represents the i-th segment of the d-th sensor in the time window of length T, and inputs each segment into the embedding layer:

[0161]

[0162] in are the learnable matrix parameters.

[0163] S4.1.2: Multi-head Attention (MA):

[0164] Define a set of two-dimensional time series data with time dimension T and space dimension D: When the attention mechanism is applied to the time dimension, data of different spatial dimensions share weights and perform the same matrix operations. Take the time series data of the d-th dimension as an example:

[0165] The d-th dimension time series data after DI-Embedding is expressed as d model To size the sequence dimension, first generate three matrices (queries, keys and values) by the following calculations:

[0166] Q t =X t W t q

[0167] Where W t q , and are all learnable parameters, W t q and d k K t The vector dimension, Q t Need to be with K t The vector dimensions are consistent, d v V t The vector dimension of . t and key value Then calculate the scaled dot product of Q and K to get the probability of each time period in the dth dimension being worth noting

[0168]

[0169] Where softmax() is a normalized exponential function, K t The transpose of . t With V t Multiply and calculate the attention feature value of each time period

[0170] Attention t (Q t ,K t ,V t )=α t V t

[0171] A multi-head attention mechanism (MA) is used to allow the model to focus on information from different time periods:

[0172]

[0173] Among them, MA t (.) represents the multi-head attention mechanism function, are learnable parameters.

[0174] Similarly, we can apply the attention mechanism to the spatial dimension, share weights with different time dimensions, and express the attention mechanism calculation of the spatial dimension as: MA s .

[0175] S4.1.2: Dual Channel Attention (DCA):

[0176] Get the two-dimensional array from DI-Embedding As the input of DCA, L represents the number of segments obtained by DI-Embedding in the time step dimension, D represents the total number of aircraft engine sensors used for prediction after preprocessing, and d model Indicates the vector dimension after DI-Embedding processing. For ease of reading, X is used. ：，d represents the vector of all time steps of dimension d (1≤d≤D), X l，： Represents the vector of all dimensions of the first time step (1≤l≤L). In DCA, X is the input vector received from DI-Embedding. DCA consists of Temporal Channel, Spatial Channel and feature fusion layer, as follows:

[0177] 1) Temporal Channel

[0178] In the Temporal Channel, multi-head attention (MA) is applied to the time dimension and residually connected with the original input X to obtain Then it passes through a multi-layer perceptron (MLP) and finally with X t，1 Perform residual connection to get the output X of TemporalChannel t，out :

[0179]

[0180] X t，out =LayerNorm(X t，1 +MLP(X t,1 ))

[0181] Where 1≤d≤D, LayerNorm(·) represents the layer normalization widely used in time series forecasting, which can improve the convergence speed and generalization ability of the model.

[0182] 2) Spatial Channel

[0183] Similar to Temporal, Spatial Channel is specifically expressed as follows:

[0184]

[0185] X s,out =LayerNorm(X s，1 +MLP(X s，1 ))

[0186] 3) Feature Fusion

[0187] After dual-channel feature extraction, the features of the two dimensions are spliced to achieve feature fusion, and finally a linear layer is used to restore the dimension size:

[0188] X f,out =Linear(Concat(X t，out , X s，out ))

[0189] Concat means concatenating vectors according to the last dimension, and Linear is a linear transformation function.

[0190] Y=DCA(X)

[0191] S4.1.3: DIformer:

[0192] DIformer consists of an encoder and a decoder, such as Figure 1 As shown, the encoder consists of DI-Embedding and DCA, and the structure of the decoder layer is as follows Figure 4 shown.

[0193] 4) Encoder: The encoder includes DI-Embedding and multi-layer DCA. DI-Embedding preserves the dimensionality independence of the embedding vector, while multi-layer DCA is used to extract the deep features of the input vector in the time series and sensor dimensions. The encoder can be expressed as:

[0194]

[0195] Where Emb represents the dimension-independent patch embedding method, which is added to the position encoding vector to obtain the encoder's two-dimensional input vector X enc,0 or X enc,l , Linear represents a learnable linear layer used to merge the segmented vectors, and Concat represents vector concatenation.

[0196] 5) Decoder: The decoder takes as input the learnable positional encoding vector and the encoder vectors before and after DCA feature extraction. It then uses the sum of the outputs of the two decoding layers and passes through a linear layer to output the final prediction value.

[0197] The decoder can be expressed as:

[0198]

[0199] X3 dec，l =LayerNorm(X2 dec,l +X1 dec,l )

[0200] X dec，l=LayerNorm(X3 tec，l +MLP(X3 dec，l ))

[0201] in is the position encoding vector that the decoder can learn, and τ is the length of the processed input sequence (if the input sequence length cannot be L seg If the length is divisible by L, the sequence must be padded to the end until it can be divided by L. seg divisible by), X dec,l is the output of each layer of the decoder, X1 dec，l For DCA output, X2 dec，l X3 is the attention output after the decoder fuses the feature information from the encoder. dec，l X2 dec，l Residual connections with attention mechanism.

[0202] 6) Prediction layer (Predict): The prediction layer receives information from the decoder and uses the sum of layer-by-layer predictions as the final prediction value:

[0203]

[0204] Linear() is a linear transformation function.

[0205] S4.2: Model performance evaluation criteria:

[0206] The root mean square error (RMSE) and score function (Score) are used to evaluate the model performance, as follows:

[0207] 3)

[0208] 4)

[0209] Where n is the number of test samples, is the RUL prediction value of the i-th sample, y i is the true value of RUL of the i-th sample. The lower the RMSE and Score values are, the higher the prediction accuracy is and the better the model performance is.

[0210] S4.3: Set up model training and hyperparameter optimization methods:

[0211] S4.3.1: Set the training method:

[0212] The Adam optimizer was used to optimize the model parameters. The initial learning rate Learning_rate was set to 0.001, the mean square error function MSELoss was used as the loss function, and the dynamic learning rate adjustment strategy ReduceLROnPlateau was adopted. During the training process, the loss function of the model on the validation set was monitored. Whenever the loss function did not decrease for 8 consecutive epochs, the learning rate was reduced to better fine-tune the parameters.

[0213] S4.3.2: Model hyperparameter optimization methods:

[0214] The Optuna hyperparameter optimization framework is used to set the model hyperparameter search space. In order to reduce the uncertainty of test results caused by the randomness of model training, repeatable experiments are added to each optimization step. During the optimization process, the optimizer's monitoring indicator is the RMSE of the model on the test set.

[0215] The flow chart of model training and hyperparameter optimization method is as follows Figure 6 shown.

[0216] To measure the performance of the proposed aircraft engine remaining useful life prediction model, the proposed model and training and optimization algorithms were applied to the C-MAPSS dataset, a widely used dataset in this field. Commonly used RMSE and Score evaluation functions were used to evaluate the model's prediction performance and compare it with currently published typical or advanced prediction models. Table 1 lists the hyperparameters of the DIformer model, and Tables 2 and 3 respectively show the RMSE and Score comparison results with other prediction models on four sub-datasets of the C-MAPSS dataset. The method proposed in this embodiment takes into account the spatiotemporal dependencies of multivariate time series and improves the dimensional independence shortcomings of previous methods by generating small batches of samples, further improving prediction performance and achieving SOTA results among publicly published models.

[0217] Table 1 DIformer model hyperparameters

[0218]

[0219] Table 2 RMSE comparison results with other SOTA methods

[0220]

[0221] The bold values are the best results among all methods, and the starred values are the second best results.

[0222] Table 3 Score comparison results with other SOTA methods

[0223]

[0224] The bold values are the best results among all methods, and the starred values are the second best results.

[0225] In summary, this embodiment provides a DIformer-based aircraft engine remaining useful life prediction method that employs dimensionally independent embedding. This approach changes the traditional method of data transfer within the model. Dimensionally independent patch embedding allows the model to simultaneously consider the characteristic information of the original monitoring data at any time period and from any sensor, improving the deep learning-based model's ability to extract spatiotemporal features. The proposed solution has broad application prospects in the field of advanced aircraft engine health management, where prediction is the core.

Claims

1. A method for predicting the remaining useful life of an aircraft engine based on DIformer, characterized in that: The following steps are involved: S1: Collect time series monitoring data from various sensors of the aircraft engine and label the collected time series data as the original data set for training the model; S2: Preprocess the labeled multivariate time series data of aircraft engines and convert them into standardized time series feature vectors that meet the requirements of model training; S3: Use sliding windows to process continuous standardized time series feature vectors and convert them into small batch samples required for model training; S3.1: RUL generation: Assuming that the original data set is the engine full life cycle data set, the engine working cycle number cycle is extracted from the engine original data set, and the corresponding remaining service life label is calculated using the following function: where cycle max The maximum value of the cycle in the sequence of the current engine ID. The maximum value of the remaining service life is set to 125 as a typical value. When solving different engine remaining service life prediction problems, it should be set according to the actual situation. S3.2: Generate small batches of samples using a sliding window. Use the sliding window method to generate small batches of samples. Let the sliding window size be n. The small batch of samples can be expressed as: X=[X 1 ,X 2 ,…,X D ] where X i Expressed as: i=1,2,…,D, select the remaining service life Y at the last moment of each sliding window RUL Serves as a label for the sample's remaining useful life; S3.3: Sample feature pre-extraction: Use linear regression to pre-extract features from small batches of samples, and append the extracted linear coefficients coef and mean mean to the end of the small batch of samples: coef i =LinearRegression(X i ).coef_ mean i =mean(X i ) Use X new Represents a small batch of samples after feature pre-extraction: in Expressed as: S4: Build the DIformer model, set the model hyperparameter optimization algorithm, and train the model using small batch training samples as input; S4.1: Build DIformer model: S4.1.1: Dimension-Independent Embeddings: It uses a mini-batch sample generation method with independent embedding vectors for each dimension to capture cross-dimensional and cross-temporal dependencies. Use length L seg The segment size is the time window sample X of length T (s) To split, here we assume that T can be L seg Divisibility: in represents the i-th segment of the d-th sensor in the time window of length T, and inputs each segment into the embedding layer: in is the learnable matrix parameter; S4.1.2: Multi-head Attention: Define a set of two-dimensional time series data with time dimension T and space dimension D: When the attention mechanism is applied to the time dimension, data of different spatial dimensions share weights and perform the same matrix operations. Take the time series data of the d-th dimension as an example: The d-th dimension time series data after dimension-independent embedding is represented as d model To size the sequence dimension, first generate three matrices by the following calculations: in and are all learnable parameters. and d k K t The vector dimension, Q t Need to be with K t The vector dimensions are consistent, d v V t The vector dimension of query Q t and key value Then calculate the scaled dot product of Q and K to get the probability of each time period in the dth dimension being worth noting Where softmax() is a normalized exponential function, K t The transpose of t With V t Multiply and calculate the attention feature value of each time period Attention t (Q t ,K t ,V t )=a t V t MA t (.) is the multi-head attention mechanism function, which uses the multi-head attention mechanism MA to allow the model to focus on information from different time periods: in, is a learnable parameter; Similarly, the attention mechanism is applied to the spatial dimension, and the weights are shared by different time dimensions. The attention mechanism calculation of the spatial dimension is expressed as: MA s ; S4.1.3: Dual-channel attention DCA: Embed the two-dimensional array obtained from the independent dimension As the input of DCA, L represents the number of segments obtained by dimensionally independent embedding in the time step dimension, D represents the total number of aircraft engine sensors used for prediction after preprocessing, and d model Indicates the vector dimension after dimension-independent embedding. For ease of reading, X :,d Represents the vector of all time steps of dimension d, 1≤d≤D, X l,: Represents the vector of all dimensions at the lth time step, 1≤l≤L. In DCA, X is the input vector received from the dimension-independent embedding; DCA consists of the temporal channel, the spatial channel, and the feature fusion layer, as follows: 1) Temporal Channel In the Temporal Channel, multi-head attention is applied to the time dimension and residual connection is performed with the original input X to obtain Then pass it through a multi-layer perceptron and finally combine it with X t,1 Perform residual connection to get the output X of Temporal Channel t,out : X t,out =LayerNorm(X t,1 +MLP(X t,1 )) Where 1≤d≤D, LayerNorm(·) represents the layer normalization widely used in time series forecasting to improve the convergence speed and generalization ability of the model; 2) Spatial Channel Similar to Temporal, Spatial Channel is specifically expressed as follows: X s,out =LayerNorm(X s,1 +MLP(X s,1 )) 3) Feature Fusion After dual-channel feature extraction, the features of the two dimensions are spliced to achieve feature fusion, and finally a linear layer is used to restore the dimension size: X f,out =Linear(Concat(X t,out ,X s,out )) Concat means vector concatenation according to the last dimension, and Linear means linear layer. Y=DCA(X) Y is DCA output; S4.1.4: A two-channel Transformer for dimension-independent patch embedding: DIformer consists of an encoder and a decoder. The encoder consists of dimension-independent embedding and DCA. 1) Encoder: The encoder includes dimension-independent embedding and multi-layer DCA. Dimension-independent embedding preserves the dimensional independence of the embedding vector, and multi-layer DCA is used to extract the deep features of the input vector in the time series and sensor dimensions. The encoder is represented as: Where Emb represents the dimension-independent patch embedding method, which is added to the position encoding vector PE to obtain the encoder's two-dimensional input vector X enc,0 or X enc,l , Linear represents a learnable linear layer used to merge the vector with the segmentation, Concat means vector concatenation; 2) Decoder: The decoder takes as input the learnable positional encoding vector and the encoder vectors before and after DCA feature extraction. It then uses the sum of the outputs of the two decoding layers and passes through a linear layer to output the final prediction. The decoder is represented as: X3 dec,l =LayerNorm(X2 dec,l +X1 dec,l ) X dec,l =LayerNorm(X3 dec,l +MLP(X3 dec,l )) in is the position encoding vector that the decoder can learn, τ is the length of the processed input sequence, if the input sequence length cannot be L seg If the length is not divisible by L, the sequence must be padded to the end until it can be divided by L. seg Divisible by X dec,l is the output of each layer of the decoder, X1 dec,l For DCA output, X2 dec,l X3 is the attention output after the decoder fuses the feature information from the encoder. dec,l X2 dec,l Residual connection of attention mechanism; d model is the vector dimension; 3) Prediction layer: The prediction layer receives information from the decoder and takes the sum of the layered predictions as the final prediction value: Linear(.) represents a linear transformation function; S4.2: Model performance evaluation; S5: The optimization algorithm automatically selects and retains the model weight file with the smallest prediction error to obtain the optimized model; S6: Input the validation set into the model for functional verification. If the validation result is close to the test result, the model training is completed, and the DIformer-based aircraft engine remaining useful life prediction model is obtained.

2. The method for predicting the remaining useful life of an aircraft engine based on DIformer according to claim 1, characterized in that: In step S1, the collected time series is marked. The data to be marked include the engine ID and the number of engine working cycles. It should be noted that if the collected aircraft engine sensor data is not the full life cycle data of the aircraft engine, the remaining service life of the engine in the last working cycle needs to be additionally marked.

3. The method for predicting the remaining useful life of an aircraft engine based on DIformer according to claim 2, characterized in that: In step S2, the labeled multivariate time series data of the aircraft engine needs to be preprocessed. The specific steps of the preprocessing are as follows: S2.1: Data cleaning: Assume that the complete engine degradation process includes N moments, and obtain engine operation data collected by m sensors: Among them, the degradation sequence of the jth sensor is expressed as: j = 1, 2, …, m. Since the prediction of the remaining useful life of an aircraft engine requires the extraction of degradation features, the engine ID and engine operating cycle that are irrelevant to the degradation features, as well as the sensor data whose parameters remain basically unchanged throughout the entire degradation cycle of the engine, are eliminated. These data have little contribution to the extraction of degradation features. Finally, D sensor degradation data are retained: S2.2: Normalization: Perform Min-Max normalization on the input D sensor degradation data according to the formula: Normalize the data of each sensor, where x is the degraded data before normalization, x min is the minimum value of the sequence where x is located, x max is the maximum value of the sequence where x is located, and the normalized sensor data is: The normalized data of the jth sensor is:

4. The method for predicting the remaining useful life of an aircraft engine based on DIformer according to claim 3, characterized in that: The specific method for evaluating model performance in S4.2 is: The root mean square error (RMSE) and the scoring function (Score) are used to evaluate the model performance, as follows: 1) 2) Where n is the number of test samples, is the RUL prediction value of the i-th sample, y i is the true value of RUL of the i-th sample. The lower the RMSE and Score values are, the higher the prediction accuracy is and the better the model performance is. S4.3: Set up model training and hyperparameter optimization methods: S4.3.1: Set the training method: The Adam optimizer was used to optimize the model parameters. The initial learning rate (Learning_rate) was set to 0.001, the mean squared error function (MSELoss) was used as the loss function, and the dynamic learning rate adjustment strategy (ReduceLROnPlateau) was used. During training, the loss function of the model on the validation set was monitored. Whenever the loss function did not decrease for eight consecutive epochs, the learning rate was reduced to better fine-tune the parameters. S4.3.2: Model hyperparameter optimization methods: The Optuna hyperparameter optimization framework is used to set the model hyperparameter search space. In order to reduce the uncertainty of test results caused by the randomness of model training, repeatable experiments are added to each optimization step. During the optimization process, the optimizer's monitoring indicator is the RMSE of the model on the test set.

Images