Aero-engine missing value filling method based on multi-scale time sequence and attribute feature extraction

Abstract

The invention discloses an aero-engine missing value filling method based on multi-scale time sequence and attribute feature extraction. Firstly, for data collected by each sensor of an aero-engine, a temporary replacement value of a missing value is generated by adopting linear interpolation; extracting a plurality of significant cycles of the aero-engine sequence data as a time scale, decomposing the sequence into trend terms and season terms according to the significant scale, and splicing all the trend terms and season terms in the time dimension; then, in-scale and inter-scale time sequence dependence is captured in a time dimension by using cross-scale graph convolution so as to carry out missing value filling in time sequence. In the attribute dimension, local neighborhood information is accumulated by adopting a cross-attribute graph volume, global attribute features are calculated by utilizing a relational network, node features are updated, and attribute association is supplemented on the basis of a time sequence filling result. And finally, aggregating the node features of a specific scale through a maximum pooling layer to represent the features, and calculating by adopting two linear layers and an activation function to obtain a final filling result.

Description

Technical Field

[0001] The invention belongs to the field of data mining and relates to a method for filling missing values ​​of an aero-engine based on multi-scale time series and attribute feature extraction. Background Art

[0002] The multivariate time series generated by multiple sensors of aircraft engines are an important resource for prediction and health management. However, due to the complex operating environment of aircraft engines, the large number of sensors involved, and the influence of factors such as the environment, hardware failures, and communication transmission failures, it is common for missing values ​​to appear in the aircraft engine time series data. Missing data of aircraft engines not only affects the comprehensive analysis of engine performance, but also may lead to a decrease in the accuracy of fault prediction models, which will limit subsequent health management work. At present, a common way to solve this problem is to fill in missing values ​​of aircraft engines.

[0003] Missing value filling technology uses the model estimated value to replace the missing data, thereby restoring incomplete time series data to complete data. Aircraft engine operation data usually has complex time series dependencies. Missing values ​​will not only lead to incomplete data, but also destroy this time series characteristic. Therefore, missing value filling is not only a basic step in data mining, but also an important technical means to ensure the safety of aircraft engines, optimize maintenance decisions, and improve the accuracy of fault diagnosis.

[0004] Existing filling methods include simple interpolation, regression models, and statistical methods. These methods analyze the patterns of observations to identify features in the data, but most of these methods are based on strict stationarity assumptions and often fail to mine the time dependency and nonlinear correlation information between variables in the data. With the development of machine learning and even deep learning technology, missing value filling methods based on neural network models have gradually emerged. In particular, Cini et al. proposed a method based on graph neural network (GNN) to fill missing values ​​in time series data (Cini A., Marisca I., Alippi C.. Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks. The Tenth International Conference on Learning Representations (ICLR), 2022). First, the graph neural network (GNN) can accurately capture the long-term and short-term dependencies between data points, and model the time series and attribute associations of data through graph structures, overcoming the limitation that traditional methods cannot effectively handle complex dependencies. Secondly, GNN can make full use of the correlation between data and combine the complementary information of multiple sensors or parameters to improve the accuracy of filling. In addition, graph neural networks balance local and global information through their information propagation mechanism, so that they can more comprehensively consider the overall trends and changes of time series when filling missing values, thereby improving the accuracy and reliability of filling.

[0005] In addition, there are some GNN-based variant methods that are widely used, such as Marisca et al. (Marisca I., Cini A., Alippi C.. Learning to reconstruct missing data from spatiotemporal graphs with sparse observations. Advances in Neural Information Processing Systems, 2022, 35: 32069-32082.) proposed combining the spatiotemporal attention mechanism with graph neural networks. Given a set of highly sparse discrete observations, the spatiotemporal propagation architecture is used to learn the representation of data points in time and space to reconstruct the observations of the corresponding sensors and their neighboring nodes. Shen et al. (Shen G., Zhou W., Zhang W., et al. Bidirectional spatial-temporal traffic data imputation via graph attention recurrent neural network. Neurocomputing, 2023, 531: 151-162.) proposed a data filling method based on graph attention recurrent neural network. This method combines the long short-term memory network (LSTM) with the graph attention mechanism (GAT) to model the temporal dependency and attribute correlation of the data respectively, and generates time and spatial estimates of missing values ​​in the bidirectional recursive process, making the filling results more reasonable. Chen et al. (Chen Y., Hu R., Li Z., et al. Exploring explicit and implicit graph learning for multivariate timeseries imputation. Engineering Applications of Artificial Intelligence, 2024, 127: 107217.) proposed an explicit and implicit graph recursive network to fill missing values. This method combines the recurrent neural network (RNN) with the graph convolutional network, uses adaptive graphs to mine implicit dependencies between attributes, and combines them with predefined graph structures to make up for the shortcomings of predefined graphs. However, most of these missing value filling methods based on graph neural networks focus on mining time series dependencies at a single scale, without considering the multi-scale information of time series, that is, various forms of long-term trends and cyclical changes.

[0006] The present invention provides a method for filling missing values ​​of aircraft engines based on multi-scale time series and attribute feature extraction. This method, combined with graph neural network technology, can effectively handle the problem of missing time series data generated during the operation of aircraft engines. This technology plays an important role in the health management and performance evaluation of aircraft engines. It can ensure the integrity and accuracy of various time series data obtained by sensors, thereby improving the quality and reliability of engine operation status evaluation, fault prediction and performance optimization, and can be deployed and applied in aircraft engine monitoring systems. Summary of the invention

[0007] The present invention aims to provide a method for filling missing values ​​of aircraft engines based on multi-scale time series and attribute feature extraction, referred to as MTAGIN. First, after recognizing the importance of periodicity in time series data, the fast Fourier transform method (FFT) is used to identify several significant time scales. Then, different time scales are used as moving average kernels to decompose the incomplete time series into trend components and seasonal components, and all trend components are spliced ​​into a continuous trend sequence, and all seasonal components are spliced ​​into a continuous season sequence. Secondly, the obtained trend sequence and season sequence are respectively input into the multi-scale time series feature extraction module to model the time series dependency within and between scales. In the modeling process, a cross-scale time series graph adjacency matrix is ​​constructed using a graph neural network, in which each time point is used as a node of the graph and the time series dependency is used as an edge to simulate the dependency relationship within and between scales. The neighborhood information is aggregated through the cross-scale graph convolution of the vertex domain to reveal the key attributes of the sequence such as trend and volatility, which is used to fill the missing data in the time series. Subsequently, the hidden states of the trend sequence and the seasonal sequence after feature extraction are added to obtain the time series dependency hidden information. Finally, the temporal hidden state is further input into the multi-scale attribute feature extraction module, and the graph convolution is used to aggregate the different nonlinear associations between attributes at each scale to more accurately capture the correlation between sequences unfolding at different time scales, thereby enhancing the filling effect. After the feature extraction process, that is, after the encoding is completed, the hidden state is restored to a single scale through the maximum pooling layer, and then the final filling result is output through two fully connected layers.

[0008] The present invention is designed for incomplete parameter data generated during the operation of aircraft engines. The data comes from the continuous monitoring and sampling of the engine, and contains time series of multiple parameters, forming a multivariate time series. In these time series, each parameter data will present different characteristics on different time scales. For example, parameters such as fan speed or turbine outlet temperature may experience rapid fluctuations on a shorter time scale, and may also show a slower upward or downward trend change on a longer time scale. At the same time, the relationship between the parameters also presents a correlation between different attributes. There is usually a positive correlation between the total engine inlet temperature and the total outlet temperature, and between the fan inlet pressure and the outlet pressure, and there may be a negative correlation between the fan speed and the turbine outlet temperature. Multi-scale time series modeling can accurately capture the complex time variation pattern of parameters, and attribute association modeling further improves the accuracy and rationality of the filling results.

[0009] The specific technical solutions of the present invention are as follows:

[0010] A missing value filling method for aircraft engines based on multi-scale time series and attribute feature extraction is proposed. Its architecture mainly consists of a multi-scale learning and decomposition module, a graph-based cross-scale time series modeling module, and a graph-based attribute association modeling module. The overall filling process includes the following steps:

[0011] (1) Multi-scale learning and time series decomposition

[0012] make Represents incomplete input data, L represents the number of time steps of the data, and N represents the number of attributes in the data set; the vector represents the observed sample at time step t, where t = 1, 2, ..., L; the mask matrix M is used to represent the missing positions in X; if If it is 1, it means exists; if If it is 0, it means Missing, where n = 1, 2, ..., N represents the index of the attribute; temporary replacement values ​​are generated for the missing values ​​in the incomplete input data using linear interpolation to form a new complete input Then, the fast Fourier transform is used to obtain multiple significant periods of the time series {p1, p2, …, p S}, and based on these significant cycle lengths, the moving average method is used to obtain the long-term trend term of the time series at each scale, and the rest is the seasonal term; for the input And the corresponding mask matrix M, the decomposition process is:

[0013]

[0014] In the formula, and represents the extracted trend term and seasonal term, ⊙ represents element-by-element multiplication, AvgPool(·) represents downsampled moving average, and Padding(·) represents extending the sequence length with 0; then, the trend terms and seasonal terms of each scale are concatenated in the time dimension to obtain the final multi-scale trend sequence. and multiscale seasonal series Where L ′ =S×L;

[0015] (2) Graph-based cross-scale temporal dynamic correlation modeling

[0016] After obtaining the trend component and seasonal components Afterwards, based on the graph neural network, and Each time point in the graph is considered as The nodes in the adjacency matrix are considered as edges. Then the weights between time nodes are allocated, and each element represents the associated weight of the state at different time points within or between scales; the adjacency matrix A is constructed by generating an adaptive graph T , calculated by the following formula:

[0017]

[0018] in, represents the learnable embedding matrix used to generate the adjacency matrix of the time series graph, D represents the graph embedding dimension, ReLu(·) represents the activation function that makes each element of the adjacency matrix positive, and Softmax(·) represents the normalization operation; The mask matrix represents the temporal adjacency matrix, which ensures that the time point at each scale is only related to the historical moment and is not affected by the future moment. Before performing the graph convolution operation, the decomposed input sequence is reconstructed into a dimension of d m The embedding operation makes the input expansion dimension become Then the node feature matrix is ​​obtained through graph convolution operation Calculated by the following formula:

[0019]

[0020] Among them, A T is the adjacency matrix, is the node feature matrix of the g-th graph convolution layer, G is the total number of graph convolution layers, and is a learnable weight matrix, b1 is a bias, GeLU(·) is an activation function, and is the mask matrix repeated L times; The calculation process of is the same as formula (4), and Add together to get the final output of the cross-scale temporal graph convolution module That is, the temporal feature encoding matrix used for filling;

[0021] (3) Graph-based attribute association modeling

[0022] In the attribute dimension, the time series feature encoding matrix H T Each attribute in is considered as an attribute graph The nodes in the , and the relationships between attributes are considered as edges; represents the attribute graph adjacency matrix of the kth scale, k = 1, 2, ..., S, and each element represents the association weight between attributes; the method is the same as formula (3), using the adaptive graph method to generate an attribute graph adjacency matrix at each scale Among them, each The diagonal elements of are kept to 0; then the feature encoding is obtained by using graph convolution operation at each scale It is expressed by the following formula:

[0023]

[0024] in, is the attribute graph adjacency matrix, is the node feature encoding of the g-th graph convolution layer at the k-th scale, G is the total number of graph convolution layers, W2 is the learnable weight matrix, and b2 is the bias; multi-scale attribute feature encoding is obtained through graph convolution operation

[0025] (4) Remodeling attribute relationships after graph convolution

[0026] A relational network module is added after the above graph convolution to further explore the potential complex relationships between global attributes. Specifically, the global relationship between attribute nodes is calculated through the relational network module, and the relationship is used to enhance the feature representation. After the graph convolution, each pair of attribute nodes is modeled with the following formula to determine the relationship between them:

[0027]

[0028] in, and is the feature embedding of node i and node j, f(·) is a nonlinear function, and a multi-layer perceptron MLP is selected, r ij Represents the relationship between node pairs; then, based on these relationships, the features of the nodes are further updated:

[0029]

[0030] Among them, W is the learnable weight matrix, It is the node feature updated after relationship modeling;

[0031] (5) Multi-scale feature fusion

[0032] All scale features represent Comprehensively reflect the temporal pattern and attribute association characteristics of each scale; pass these specific scale feature representations through a global maximum pooling layer, which can not only integrate scale features, but also extract the key features with the greatest influence between different scales; set the final output feature It is calculated by the following formula:

[0033]

[0034] Among them, MaxPool(·) represents the maximum pooling operation;

[0035] (6) Calculate the final filling result of the model

[0036] In the data encoding stage, the spliced ​​time series are encoded successively in the time dimension and the attribute dimension, and finally the hidden state H that integrates the time series and attribute feature information is obtained. ′ ; Then only one decoder is needed to decode the hidden state and fill in the missing values; specifically, two linear layers and activation functions are used as the output layer, where the first linear layer remaps the hidden state in the time dimension, and the second linear layer in the attribute dimension remaps the attribute embedding dimension d m The mapping is 1; the loss function of the MTAGIN model is calculated using the bi-norm of the model output and the existing value. The Adam algorithm is used to minimize the loss function during training. The model output obtained in the last round of training is the final filling result.

[0037] The beneficial effect of the present invention is that the missing parameter data generated during the operation of the aircraft engine can be filled through the designed aircraft engine missing value filling model, so that the incomplete data can be converted into complete data, so that the filled complete data can be used for subsequent analysis work such as aircraft engine operation status monitoring, performance evaluation and fault diagnosis, thereby improving data utilization and analysis accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a model framework diagram of the present invention.

[0039] Figure 2 It is a workflow diagram of the missing value filling method of the present invention. DETAILED DESCRIPTION

[0040] The specific implementation of the present invention is described in detail below in conjunction with the content of the invention and the accompanying drawings.

[0041] like Figure 1 As shown in the figure, first, the incomplete aircraft engine time series data X and the mask matrix M will go through the linear interpolation link, and a one-dimensional linear function will be constructed by finding the nearest neighbor value before and after the missing value. The estimated value at the current moment is calculated and used as the initialization value of the missing value to form a complete input By using fast Fourier transform to extract S significant cycles of the input sequence and use them as time scales, the aircraft engine data is decomposed into multiple forms of trend terms and seasonal terms using the moving average method based on these scales, and all trend components are spliced ​​into a continuous trend sequence. And stitch all seasonal components into a continuous seasonal sequence The cross-scale graph convolution module is used to mine the associations in time series and aggregate the historical information of time points at any scale at each scale. This module constructs the time series graph adjacency matrix A on the trend and season parts respectively. T , each element of the adjacency matrix represents the temporal dependency weight within or between scales. Before performing the graph convolution operation, the decomposed input sequence is reconstructed into a dimension of d m The embedding operation is used to obtain the node features that represent the temporal dependency through graph convolution operation. and Then, graph convolution is also used to mine the association between attributes. By constructing the attribute graph adjacency matrix A on each scale representation V , forcing the missing value of the current attribute to be reconstructed through the association between other attributes. Next, graph convolution is used at each scale to obtain the node features after mining attribute associations. However, the node features obtained after graph convolution may not fully capture all the relationships between attributes. Therefore, a relational network is added after graph convolution to calculate the global relationship between nodes to enhance the feature representation and obtain updated node features. Finally, these node features of specific scales are aggregated through a global maximum pooling layer to represent these features, and then two linear layers and activation functions are used to calculate the final filling result.

[0042] The present invention implements missing value filling of data according to the following steps (such as Figure 2 shown):

[0043] Step 1: Input incomplete data matrix And the corresponding mask matrix M;

[0044] Step 2: Use linear interpolation to initialize the missing values ​​in X and obtain the initialized complete data matrix

[0045] Step 3: Set the number of training iterations iter to 1;

[0046] Step 4: Calculate using Fast Fourier Transform Multiple significant periods {p1,p2,…,p S};

[0047] Step 5: Set the scale index k to 1;

[0048] Step 6: According to p k Use the moving average method to decompose the input into trend and seasonal terms;

[0049] Step 7: If the last scale has been traversed, go to step 8, otherwise set k to k+1 and repeat step 6;

[0050] Step 8: Concatenate the trend items and seasonal items of each scale in the time dimension to obtain the final multi-scale trend sequence. and multiscale seasonal series

[0051] Step 9: Reconstruct the decomposed input sequence into a matrix of dimension d m Embedding operation;

[0052] Step 10: Construct a cross-scale time series graph adjacency matrix and use graph convolution to obtain the trend representation after fusion of time series features and seasonal expressions

[0053] Step 11: Represent the trend and seasonal expressions Add to obtain the time series feature representation H T ;

[0054] Step 12: Construct the attribute graph adjacency matrix and use graph convolution to obtain node representations of fused attribute features at each scale

[0055] Step 13: Build an attribute relationship network, update node features, and obtain the final multi-scale node representation

[0056] Step 14: Represent nodes of a specific scale Through a maximum pooling layer, multi-scale feature fusion is performed to obtain the output feature

[0057] Step 15: Pass the output feature H through two linear layers and activation functions to obtain the final filled output;

[0058] Step 16: If it is the last round of training, go to step 17, otherwise set iter to iter+1 and return to step 4;

[0059] Step 17: Use the final imputed output to replace the missing values ​​in the incomplete data to complete the imputation.

Claims

1. A method for filling missing values ​​of aircraft engines based on multi-scale time series and attribute feature extraction, characterized in that: Here are the steps: (1) Multi-scale learning and time series decomposition make Represents incomplete input data, L represents the number of time steps of the data, and N represents the number of attributes in the data set; the vector represents the observed sample at time step t, where t = 1, 2, ..., L; the mask matrix M is used to represent the missing positions in X; if If it is 1, it means exists; if If it is 0, it means Missing, where n = 1, 2, ..., N represents the index of the attribute; temporary replacement values ​​are generated for the missing values ​​in the incomplete input data using linear interpolation to form a new complete input Then, the fast Fourier transform is used to obtain multiple significant periods of the time series {p1, p2, …, p S }, and based on these significant cycle lengths, the moving average method is used to obtain the long-term trend term of the time series at each scale, and the rest is the seasonal term; for the input And the corresponding mask matrix M, the decomposition process is: In the formula, and represents the extracted trend term and seasonal term, ⊙ represents element-by-element multiplication, AvgPool(·) represents downsampled moving average, and Padding(·) represents extending the sequence length with 0; then, the trend terms and seasonal terms of each scale are concatenated in the time dimension to obtain the final multi-scale trend sequence. and multiscale seasonal series Where L ′ =S×L; (2) Graph-based cross-scale temporal dynamic correlation modeling After obtaining the trend component and seasonal components Afterwards, based on the graph neural network, and Each time point in the graph is considered as The nodes in the adjacency matrix are considered as edges. Then the weights between time nodes are allocated, and each element represents the associated weight of the state at different time points within or between scales; the adjacency matrix A is constructed by generating an adaptive graph T , calculated by the following formula: in, represents the learnable embedding matrix used to generate the adjacency matrix of the time series graph, D represents the graph embedding dimension, ReLu(·) represents the activation function that makes each element of the adjacency matrix positive, and Softmax(·) represents the normalization operation; The mask matrix represents the temporal adjacency matrix, which ensures that the time point at each scale is only related to the historical moment and is not affected by the future moment. Before performing the graph convolution operation, the decomposed input sequence is reconstructed into a dimension of d m The embedding operation makes the input expansion dimension become Then the node feature matrix is ​​obtained through graph convolution operation Calculated by the following formula: Among them, A T is the adjacency matrix, is the node feature matrix of the g-th graph convolution layer, G is the total number of graph convolution layers, and is a learnable weight matrix, b1 is a bias, GeLU(·) is an activation function, and is the mask matrix repeated L times; The calculation process of is the same as formula (4), and Add together to get the final output of the cross-scale temporal graph convolution module That is, the temporal feature encoding matrix used for filling; (3) Graph-based attribute association modeling In the attribute dimension, the time series feature encoding matrix H T Each attribute in is considered as an attribute graph The nodes in the , and the relationships between attributes are considered as edges; represents the attribute graph adjacency matrix of the kth scale, k = 1, 2, ..., S, and each element represents the association weight between attributes; the method of generating an attribute graph adjacency matrix at each scale is the same as the method of formula (3) using the method of generating an adaptive graph Among them, each The diagonal elements of are kept to 0; then the feature encoding is obtained by using graph convolution operation at each scale It is expressed by the following formula: in, is the attribute graph adjacency matrix, is the node feature encoding of the g-th graph convolution layer at the k-th scale, G is the total number of graph convolution layers, W2 is the learnable weight matrix, and b2 is the bias; multi-scale attribute feature encoding is obtained through graph convolution operation (4) Remodeling attribute relationships after graph convolution A relational network module is added after the above graph convolution to further explore the potential complex relationships between global attributes. Specifically, the global relationship between attribute nodes is calculated through the relational network module, and the relationship is used to enhance the feature representation. After the graph convolution, each pair of attribute nodes is modeled with the following formula to determine the relationship between them: in, and is the feature embedding of node i and node j, f(·) is a nonlinear function, and a multi-layer perceptron MLP is selected, r ij Represents the relationship between node pairs; then, based on these relationships, the features of the nodes are further updated: Among them, W is the learnable weight matrix, It is the node feature updated after relationship modeling; (5) Multi-scale feature fusion All scale features represent Comprehensively reflect the temporal pattern and attribute association characteristics of each scale; pass these specific scale feature representations through a global maximum pooling layer, which can not only integrate scale features, but also extract the key features with the greatest influence between different scales; set the final output feature It is calculated by the following formula: Among them, MaxPool(·) represents the maximum pooling operation; (6) Calculate the final filling result of the model In the data encoding stage, the spliced ​​time series are encoded successively in the time dimension and the attribute dimension, and finally the hidden state H that integrates the time series and attribute feature information is obtained. ′ ; Then only one decoder is needed to decode the hidden state and fill in the missing values; specifically, two linear layers and activation functions are used as the output layer, where the first linear layer remaps the hidden state in the time dimension, and the second linear layer in the attribute dimension remaps the attribute embedding dimension d m The mapping is 1; the loss function of the MTAGIN model is calculated using the bi-norm of the model output and the existing value. The Adam algorithm is used to minimize the loss function during training. The model output obtained in the last round of training is the final filling result.

2. The method for filling missing values ​​of aircraft engines based on multi-scale time series and attribute feature extraction according to claim 1 is characterized in that: Here are the steps: Step 1: Input incomplete data matrix And the corresponding mask matrix M; Step 2: Use linear interpolation to initialize the missing values ​​in X and obtain the initialized complete data matrix Step 3: Set the number of training iterations iter to 1; Step 4: Calculate using Fast Fourier Transform Multiple significant periods {p1,p2,…,p S }; Step 5: Set the scale index k to 1; Step 6: According to p k Use the moving average method to decompose the input into trend and seasonal terms; Step 7: If the last scale has been traversed, go to step 8, otherwise set k to k+1 and repeat step 6; Step 8: Concatenate the trend items and seasonal items of each scale in the time dimension to obtain the final multi-scale trend sequence. and multiscale seasonal series Step 9: Reconstruct the decomposed input sequence into a matrix of dimension d m Embedding operation; Step 10: Construct a cross-scale time series graph adjacency matrix and use graph convolution to obtain the trend representation after fusion of time series features and seasonal expressions Step 11: Represent the trend and seasonal expressions Add to obtain the time series feature representation H T ; Step 12: Construct the attribute graph adjacency matrix and use graph convolution to obtain node representations of fused attribute features at each scale Step 13: Build an attribute relationship network, update node features, and obtain the final multi-scale node representation Step 14: Represent nodes of a specific scale Through a maximum pooling layer, multi-scale feature fusion is performed to obtain the output feature Step 15: Pass the output feature H through two linear layers and activation functions to obtain the final filled output; Step 16: If it is the last round of training, go to step 17, otherwise set iter to iter+1 and return to step 4; Step 17: Use the final imputed output to replace the missing values ​​in the incomplete data to complete the imputation.

Images