An aerospace engine anomaly detection method based on deep attention data enhancement

Abstract

This application relates to the technical field of aircraft engine anomaly detection, and in particular to an aircraft engine anomaly detection method based on deep attention data enhancement, which uses an encoder network to map a training set X into a feature space Z; and uses DA-SMOTE to adaptively generate anomaly samples Z in the learned feature space. new , balance the under-sampled training set; then use the Decoder network to convert the abnormal sample Z new Mapping back to the original space #imgabs0#, the abnormal samples #imgabs1# and the undersampled training set X form a balanced training set. Finally, a classification network is used to diagnose whether the engine has an abnormality. This application can simultaneously reduce the degree of data imbalance and aliasing. Using DA-SMOTE for data augmentation in this feature space can further reduce the risk of generating noisy data and effectively extract complex time series features from multidimensional time series.

Description

Technical Field

[0001] The present application belongs to the technical field of aircraft engine anomaly detection, and more specifically, relates to an aircraft engine anomaly detection method based on deep attention data enhancement. Background Art

[0002] Aircraft engines are core components of aircraft, and their health significantly impacts the stability and safety of aircraft flight. To prevent early engine anomalies from causing a chain reaction and leading to major failures during flight, the most effective approach is to accurately detect anomalies and conduct comprehensive health checks before takeoff. This allows necessary measures (such as engine repair or replacement) to be taken in advance, reducing costly unplanned maintenance and preventing serious flight accidents. Therefore, developing accurate engine anomaly detection methods is of great significance.

[0003] Currently, aeroengine anomaly detection methods fall into three main categories: model-based, traditional machine learning-based, and deep learning-based. Model-based methods offer good interpretability but require extensive domain knowledge. Especially for highly complex thermal machinery, constructing an accurate physical analytical model to analyze damage propagation and dynamic response is extremely difficult, limiting the practical application of these methods. Traditional machine learning-based methods can avoid the complex modeling process, but these shallow models struggle to learn useful and identifiable features from raw monitoring data and require the integration of manually designed statistical features. If the selected statistical features fail to characterize the health of the aeroengine, the final anomaly detection results will be severely affected.

[0004] In contrast, deep learning-based methods can automatically extract useful features from raw monitoring data, avoiding the complex modeling process and addressing the difficulties of manual feature selection. Therefore, deep learning has shown remarkable potential in the field of aircraft engine anomaly detection. For example, existing technologies use filters for data preprocessing, then combine them with deep convolutional neural networks to diagnose aircraft engine health status. Others employ a set of long-short-term memory networks to learn the complex mapping relationship between performance monitoring parameters and engine health status, thereby enabling aircraft engine anomaly detection under multi-sensor conditions. Others combine deep Bayesian networks with Shapley additive interpretation to construct an interpretable anomaly detection and fault prediction framework, validating its effectiveness on real-world engine datasets. Others use convolutional autoencoders to extract high-level features from raw monitoring data, then combine them with weight-independent networks to diagnose engine anomalies. However, all of these diagnostic methods lack a well-balanced training set.

[0005] An aircraft engine typically undergoes many flight cycles to degrade from a normal state to an abnormal state. Therefore, the number of abnormal samples that can be collected is very limited, far less than the number of normal samples. On the other hand, when an aircraft engine is in an early abnormal state, abnormal samples are very similar to recent normal samples, leading to aliasing between the two. As shown in Figure 1(a), directly using deep learning for aircraft engine anomaly detection faces the dual challenges of data imbalance and sample aliasing. The first challenge: During the training phase, the loss of the deep learning model is dominated by normal samples. That is, as long as normal samples are correctly classified, the training error is very small. In this case, the deep learning model pays more attention to normal samples and tends to diagnose abnormal samples as normal samples. However, this is not allowed in practical engineering applications. The second challenge: abnormal samples are aliased with normal samples. If data augmentation is performed directly, the synthesized abnormal samples may fall into the area where normal samples are concentrated. In this case, some generated samples are closer to normal samples but still carry the "abnormal" label, which will mislead the training of the deep learning model.

[0006] To date, traditional data resampling technology generally uses undersampling technology or oversampling technology to solve the above challenges.

[0007] (1) Many undersampling techniques face the problem of information loss. The random removal strategy adopted by many undersampling techniques is likely to remove important normal samples. For highly aliased data, it may also remove a large number of boundary samples.

[0008] The working principle of undersampling technology is to reduce the number of normal samples N to match the number of abnormal samples A, that is, remove (NA) normal samples, such as Figure 1 As shown in (a)-(b) in the figure. When A is much smaller than N, it will lead to excessive information loss.

[0009] (2) Oversampling technology faces the risk of generating noise samples. For example, the popular SMOTE generates new anomaly samples by randomly sampling on the line between the anomaly sample and its adjacent anomaly samples. The generated anomaly samples are very likely to fall into the dangerous area. In order to alleviate this problem, many variants of SMOTE have emerged. For example, Borderline-SMOTE first defines a set of "dangerous" samples, that is, those anomaly samples that may be misclassified by the classification network, and then oversamples these anomaly samples. KMeans-SMOTE first uses KMeans to divide the input data into different clusters, and then uses SMOTE to generate anomaly samples in each cluster. Although Borderline-SMOTE and KMeans-SMOTE can reduce the risk of generating noise samples to a certain extent, their effect is relatively limited for highly aliased data.

[0010] The working principle of oversampling technology is to increase the number of abnormal samples to match the number of normal samples, that is, to generate (NA) abnormal samples, such as Figure 1 As shown in (a)-(c) above, when abnormal samples are highly aliased with normal samples, the generated abnormal samples are very likely to fall into the area where normal samples are concentrated (called the danger zone) and be considered as noise samples. However, these two technologies process data with high imbalance and aliasing, which affects the effectiveness of diagnosis.

[0011] In summary, there is an urgent need to develop a new aero-engine anomaly detection method based on deep attention data enhancement to solve the above technical problems. Summary of the Invention

[0012] To achieve the above objectives, the technical solution adopted in this application is to provide an aircraft engine anomaly detection method based on deep attention data enhancement, comprising the following steps:

[0013] Step 1: Use a clustering algorithm to cluster normal samples into K clusters to form cluster centers, and then combine the cluster centers and abnormal samples to form the undersampled training set X;

[0014] Step 2: First, use the Encoder network to map the training set X into a feature space where the classes are clustered within the class and separated between classes; then, use the self-attention module to adaptively generate the abnormal sample features Z in the learned feature space. new ; Finally, use the Decoder network to generate the abnormal sample feature Z new Mapping back to the original space to generate abnormal samples Abnormal samples It forms a balanced training set with the under-sampled training set X;

[0015] Step 3: Use the balanced training set to train the attention network, and use the trained attention network as a classification network to diagnose whether the engine has any abnormalities.

[0016] Optionally, a self-attention module is used to adaptively generate abnormal sample features Z new The process is called DA-SMOTE.

[0017] Optionally, in step 2, the trained Encoder network is used to map the undersampled training set X into a feature space where the inter-classes are separated and the intra-classes are clustered, that is, Z=Encoder(X).

[0018] Optionally, DA-SMOTE is used to generate abnormal samples. The process of DA-SMOTE generating abnormal samples is as follows:

[0019] Step S101: compare the abnormal sample feature z with the adjacent abnormal sample features Splicing to form a feature pair Where z∈Z;

[0020] Step S102: feature pair After processing by the attention module, the scalar output θ is obtained, as shown in formula (5):

[0021]

[0022] Where, Represents the ReLU activation function, W1, W2, and W3 represent the parameter matrices of the linear mapping;

[0023] Step S103: transform the scalar output θ to between (0, 1) through the Sigmoid function, as shown in formula (6):

[0024]

[0025] Where, is the learned interpolation factor.

[0026] Step S104, generate abnormal sample feature Z new As shown in formula (7):

[0027]

[0028] Through training, the generated abnormal sample features z can be reduced new Risk of falling into a hazardous area.

[0029] Optionally, use the trained Decoder network to transform the enhanced set [Z, Z new ] is mapped back to the original space, that is,

[0030] Original Space Together with the training set X, it forms a balanced training set

[0031] Optionally, the interpolation factor is learned using two loss functions: reconstruction loss and three-center loss;

[0032] Reconstruction loss: Use class-balanced l2-norn as reconstruction loss As shown in formula (8):

[0033]

[0034] Where x i is the i-th element in X, and is x i The reconstruction representation, |·| represents the number of elements in the set, and ||·||2 represents the l2-norm, β∈[0,1) is a hyperparameter, β is set to 0.999, when x i For normal samples, n i =K; when x i When it is an abnormal sample, n i =A.

[0035] Three-center loss: The three-center loss is used to guide the learning of feature representation and interpolation factors, as shown in formula (9):

[0036]

[0037] Where z i is Z∪Z new The i-th element in , d is a hyperparameter, d is set to 5;

[0038] When z i When it is a normal sample feature, then c i is the characteristic of the normal sample center, and is the characteristic of the center of the abnormal sample; when z i When it is an abnormal sample feature, then c i is the characteristic of the center of the abnormal sample, and is a characteristic of the center of a normal sample;

[0039] Combining the reconstruction loss and the three-center loss, DA-SMOTE, the encoder network, and the decoder network are trained simultaneously. The loss function is shown in formula (10):

[0040]

[0041] Optionally, in step 3, the classification network is trained using binary cross entropy loss, as shown in formula (11):

[0042]

[0043] Where O is the output of X after the last Softmax layer in the attention network, and O new yes The corresponding output; o i is the output probability of the i-th sample, y i is the true label of the i-th sample;

[0044] After calculating the binary cross entropy loss, the backpropagation algorithm is used to optimize the weight parameters of the classification network.

[0045] Optionally, in step 1, the clustering algorithm is a K-Means algorithm, which is:

[0046] Set the normal sample set X n ={x1, x2, ..., x N}, the normal sample set X n Divided into K clusters, the jth cluster C j The samples in the N(μ j , I), normal sample set X n The log-likelihood function ln L(μ|X n ) is shown in formula (1):

[0047]

[0048] Where, X n represents a set of N normal samples, d represents the dimension of the sample; x i represents the set of normal samples of the i-th type; C j represents the jth cluster; μ j Represents the jth cluster C j The normal distribution N(μ j , I) expectation; K represents the normal sample set X n The number of clusters;

[0049] Maximize the log-likelihood function ln L(μ|X n ) is equivalent to minimizing the sum of squared errors of formula (2):

[0050]

[0051] Where, J K The objective function optimized by the K-Means algorithm.

[0052] Optionally, the set of clusters is calculated using the K-Means algorithm as C * , as shown in formula (3):

[0053]

[0054] Where x i represents the set of normal samples of the i-th type; μ j Represents the jth cluster C j The normal distribution N(μ j , I) expectation; K represents the normal sample set X n The number of clusters; C j represents the jth cluster; C K represents the Kth cluster;

[0055] The set of cluster centers CC * As shown in formula (4):

[0056]

[0057] Where, is the Jth cluster The cluster center.

[0058] Optionally, a set consisting of A abnormal samples is preset as X a , X a ={x1, x2, ..., x A}, the set CC composed of cluster centers * and the abnormal sample set X a , forming the under-sampled training set X = CC * ∪X a ;

[0059] The quantitative difference between normal samples and abnormal samples is reduced from (NA) to (KA).

[0060] This application provides an aircraft engine anomaly detection method based on deep attention data enhancement, which has the following beneficial effects:

[0061] (1) Using clustering algorithms to divide normal samples into different clusters, and then using cluster centers and abnormal samples to form an undersampled training set, can simultaneously reduce the imbalance and aliasing of the data.

[0062] (2) DA-SMOTE can adaptively generate valid anomaly samples based on data distribution. DA-SMOTE uses an attention module to learn interpolation factors, which can adaptively generate valid anomaly samples based on data, so that the generated anomaly samples are far away from the dangerous area.

[0063] (3) The encoder and decoder networks based on the self-attention mechanism learn a feature space where inter-class separation and intra-class convergence are achieved through joint training with the three-center loss and reconstruction loss. In this feature space, normal samples are far from the center point of abnormal samples and close to the center point of normal samples; similarly, abnormal samples are far from the center point of normal samples and close to the center point of abnormal samples. At this time, using DA-SMOTE to perform data augmentation in this feature space can further reduce the risk of generating noisy data.

[0064] (4) Using the attention mechanism as the core component of feature extraction can effectively extract complex temporal features from multidimensional time series. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0066] Figure 1 This is a schematic diagram of the data undersampling and oversampling process in the background technology of this application;

[0067] Figure 2 Schematic diagram of the deep attention data enhancement process for this application;

[0068] Figure 3 Schematic diagram of DA-SMOTE;

[0069] Figure 4 Schematic diagram of the sample construction process of DA-SMOTE embedded in the Encoder network and Decoder network;

[0070] Figure 5 Schematic diagram of the engine structure and sample construction process. DETAILED DESCRIPTION

[0071] In order to make the technical problems, technical solutions and beneficial effects to be solved by this application more clearly understood, this application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0072] The present invention provides an aircraft engine anomaly detection method based on deep attention data enhancement. The aircraft engine anomaly detection method based on deep attention data enhancement includes the following steps:

[0073] Step 1: Use clustering algorithm to cluster normal samples (such as Figure 2 (a) is clustered into K clusters to form cluster centers, and then the cluster centers and abnormal samples form the under-sampled training set X (as shown in Figure 2 (as shown in (b)).

[0074] Step 2: First, use the Encoder network to map the training set X into a feature space where the classes are clustered within the class and separated between the classes (e.g. Figure 2 (c) in the figure), that is, samples of different types are far away from each other, while samples of the same type are close to each other. Then, DA-SMOTE is used to adaptively generate abnormal sample features Z in the learned feature space. new (like Figure 2 Finally, the Decoder network is used to transform the generated abnormal sample features Z new Mapping back to the original space to generate abnormal samples Abnormal samples The self-attention module is used to adaptively generate abnormal sample features Z. new The process is called DA-SMOTE.

[0075] Step 3: Use the balanced training set to train the attention network, and use the trained attention network as a classification network to diagnose whether the engine has any abnormalities.

[0076] Furthermore, in step one, the clustering algorithm is the K-Means algorithm. The K-Means algorithm is not only computationally efficient and simple to implement, but can also find the relatively optimal cluster division to form a cluster center. The cluster center is the cluster center, which can better represent the samples in the entire cluster.

[0077] Undersampling, a technique within hybrid sampling, is used to address data imbalance caused by inter-class aliasing. The undersampled training set X is obtained by undersampling normal samples using clustering techniques and merging them with abnormal samples. Because the average distance between cluster centers and normal samples in the cluster is minimal, cluster centers can represent the data in the entire cluster. Therefore, this application uses all cluster centers to replace normal samples, thereby reducing the number of normal samples.

[0078] The K-Means algorithm is:

[0079] Set the normal sample set X n ={x1, x2, ..., x N}(ie normal sample set X n represents a set of N normal samples), the normal sample set X n Divided into K clusters, the jth cluster C jThe samples in the N(μ j , I), normal sample set X n The log-likelihood function ln L(μ|X n ) is shown in formula (1):

[0080]

[0081] Where d represents the dimension of the sample; x i represents the set of normal samples of the i-th type; C j represents the jth cluster; μ j Represents the jth cluster C j The normal distribution N(μ j , I) expectation; K represents the normal sample set X n The number of clusters.

[0082] Maximize the log-likelihood function ln L(μ|X n ) is equivalent to minimizing the sum of squared errors of formula (2):

[0083]

[0084] Where, J K The objective function optimized by the K-Means algorithm.

[0085] The cluster center μ calculated by the K-Means algorithm j is the normal distribution N(μ j ,I)’s expectation, can well represent the samples in the entire cluster.

[0086] The set of clusters calculated using the K-Means algorithm is C * , as shown in formula (3):

[0087]

[0088] Where x i represents the set of normal samples of the i-th type; μ j Represents the jth cluster C j The normal distribution N(μ j , I) expectation; K represents the normal sample set X n The number of clusters; c j represents the jth cluster; C K represents the K-th cluster.

[0089] The set of cluster centers CC * As shown in formula (4):

[0090]

[0091] Where, is the Jth cluster The cluster center.

[0092] The set consisting of A abnormal samples is assumed to be X a , X a ={x1, x2, ..., x A}, the set CC composed of cluster centers * and the abnormal sample set X a , forming the under-sampled training set X = CC * ∪X a The quantitative difference between normal samples and abnormal samples is reduced from (NA) to (KA).

[0093] This application adopts mixed sampling technology to reduce the number of normal samples and increase the number of abnormal samples to ensure that the two have the same number, K (A < K < N), where K represents the normal sample set X n The hybrid sampling technique removes (NK) normal samples and generates (KA) abnormal samples. Clearly, (NK) < (NA) indicates that hybrid sampling can reduce information loss compared to undersampling; and (NK) < (NA) indicates that hybrid sampling can reduce the risk of generating noise samples compared to oversampling. Hybrid sampling mitigates the shortcomings of both undersampling and oversampling techniques, reducing both the loss of normal sample information and the risk of synthesizing noise samples.

[0094] In step 2, first, the trained Encoder network is used to map the undersampled training set X into a feature space where the inter-classes are separated and the intra-classes are clustered, that is, Z = Encoder(X); Figure 4 As shown in Figure 1, the Encoder network consists of multiple Transformer encoding layers, which can effectively extract complex time series features from multidimensional time series.

[0095] Secondly, DA-SMOTE uses linear interpolation to generate abnormal samples x. After learning through the attention module, the interpolation factor is obtained

[0096] In the prior art, SMOTE generates new abnormal samples by randomly sampling on the line connecting the abnormal sample and its adjacent abnormal samples. The process is: randomly select an interpolation factor (obeying the standard uniform distribution U(0,1)) and calculate and generate abnormal samples. Because the interpolation factor in the prior art is randomly selected, when the generated abnormal sample is very close to the normal sample, the abnormal sample is relatively far away from the real abnormal sample, which increases the difficulty of the classification network to distinguish between normal and abnormal samples. Therefore, in the feature space, using the DA-SMOTE of this application, selecting the appropriate interpolation factor can generate high-quality abnormal samples.

[0097] DA-SMOTE is used to generate abnormal samples. The specific learning process of DA-SMOTE is as follows: Figure 3 As shown:

[0098] Step S101: compare the abnormal sample feature z with the adjacent abnormal sample features Splicing to form a feature pair Among them, z∈Z.

[0099] Step S102: feature pair After processing by the attention module, the scalar output θ is obtained, as shown in formula (5):

[0100]

[0101] Where, Represents the ReLU activation function, W1, W2, and W3 represent the parameter matrices of the linear mapping.

[0102] Step S103: transform the scalar output θ to between (0, 1) through the Sigmoid function, as shown in formula (6):

[0103]

[0104] Where, is the learned interpolation factor.

[0105] Step S104, generate abnormal sample feature Z new As shown in formula (7):

[0106]

[0107] Through training, the generated abnormal sample features z can be reduced new Risk of falling into a hazardous area.

[0108] Step S105: Use the trained Decoder network to transform the enhanced set [Z, Z new 】Mapped back to the original space, that is The decoder network has a similar structure to the encoder network, except that a linear mapping is added to keep the output dimension of the decoder network consistent with the input data.

[0109] Original Space Together with the training set X, it forms a balanced training set

[0110] To further reduce the risk of generating noisy data, this application uses two loss functions to learn interpolation factors, so that the generated abnormal sample features are far away from the danger zone. The two loss functions are reconstruction loss and three-center loss. The feature representation learned using these two loss functions can not only reconstruct the input data well, but also reduce the degree of aliasing between class data and the degree of dispersion of intra-class data.

[0111] Reconstruction loss: In order to ensure that the model can better reconstruct the input data (i.e., data after cluster downsampling and before DA-SMOTE), this application uses class-balanced l2-norn as the reconstruction loss As shown in formula (8):

[0112]

[0113] Where x i is the i-th element in X, and is x i The reconstructed representation of . |·| represents the number of elements in the set, and ||·||2 represents the l2-norm. β∈[0,1) is a hyperparameter, β is set to 0.999. When x i For normal samples, n i =K; when x i When it is an abnormal sample, n i =A.

[0114] Triple-center loss: guides the learning of latent representation and interpolation factors, as shown in formula (9):

[0115]

[0116] Where z i is Z∪Z new The i-th element in . d is a hyperparameter and is set to 5. i When it is a normal sample feature, then c i is the characteristic of the normal sample center, and c p i is the characteristic of the center of the abnormal sample; when z i When it is an abnormal sample feature, then c i is the characteristic of the center of the abnormal sample, andc p i is the characteristic of the center of a normal sample.

[0117] This application combines the reconstruction loss and three-center loss loss functions to train the DA-SMOTE and Encoder-Decoder networks simultaneously. The final loss function is shown in formula (10):

[0118]

[0119] Step 3: Use the balanced training set to train the attention network. The trained attention network is used as the classification network to diagnose whether the engine has an abnormality. This application uses the self-attention network (SANet) as the classification network. The attention mechanism can map the input data into different subspaces, then assign different weights to select important temporal features, and finally merge them into richer and more recognizable fusion features.

[0120] This application selects binary cross entropy loss to train the classification network, as shown in formula (11):

[0121]

[0122] Where O is the output of X after the last Softmax layer in the attention network, and O new yes The corresponding output; o i is the output probability of the i-th sample, y i is the true label of the i-th sample;

[0123] After calculating the binary cross entropy loss, the backpropagation algorithm is used to optimize the weight parameters of the classification network.

[0124] This application uses the performance monitoring data of a real aircraft engine of an Asian airline as an example for verification and explanation. The selected research object is a twin-rotor, axial-flow, high-bypass turbofan engine, such as Figure 5 As shown in (a), Figure 5 (a) is a schematic diagram of the engine structure.

[0125] This application uses EGTM, DEGT, DN2 and DFF as the key performance parameters of aircraft engines, such as Figure 5 As shown in (b), EGTM is the exhaust temperature margin, DEGT is the exhaust temperature deviation, DN2 is the high-pressure rotor speed deviation, and DFF is the fuel flow deviation, which serve as the basis for engine monitoring and fault diagnosis.

[0126] First, if Figure 5(b) shows the sample construction process. Using a sliding window method (window size l = 10, step size s = 10), a total of 954 normal samples and 63 abnormal samples were collected. The dataset was then partitioned using a five-fold cross-partitioning scheme, as shown in Table 1. Furthermore, to mitigate the randomness of a single experiment, ten replicates were performed on each partitioned dataset.

[0127] Table 1: Dataset

[0128]

[0129] In this application, the test set is also unbalanced. For example, the first fold consists of 191 normal samples and 13 abnormal samples, with an imbalance ratio of 13 / 191 = 0.068. In this case, the overall accuracy will be biased towards normal samples, making it of no significant reference value. Therefore, this application uses a confusion matrix to evaluate the effectiveness of anomaly detection, as shown in Table 2.

[0130] Table 2 Confusion matrix example

[0131]

[0132] Furthermore, the true positive rate (TPR) is used to evaluate the detection effect of the model on abnormal samples, while the true negative rate (TNR) is used to evaluate its detection effect on normal samples. Balanced Accuracy is used to evaluate the overall detection effect of the model. The above indicators can be expressed as:

[0133] TPR=N 11 / (N 11 +N 12 ) (12)

[0134] TNR=N 22 / (N 21 +N 22 ) (13)

[0135] balanced accuracy=(TPR+TNR) / 2 (14)

[0136] The experimental platform for this application is Python 3.9 and PyTorch 1.11.0. Adam is selected to optimize the parameters of the neural network, with a learning rate of 0.0002 and a minimum batch size of 128.

[0137] When selecting key hyperparameters, first, the key hyperparameter related to clustering is the number of clusters, which is set to 90; second, the key hyperparameters related to the self-attention-based Encoder-Decoder are the number of Transformer encoding layers of 2, the number of attention heads of 4, and the feature dimension of 128; then, the hyperparameter related to DA-SMOTE is the compression ratio of 16; finally, the key hyperparameters related to the classification network are also the number of Transformer encoding layers of 2, the number of attention heads of 4, and the feature dimension of 128.

[0138] In addition, the hyperparameter settings of the classification network in the comparison method are consistent with the classification network mentioned above.

[0139] Through the above experiments, this application first selected a SANet method that does not use any resampling technology and 8 popular resampling techniques for comparison. The comparison methods considered are summarized as follows:

[0140] (1) SANet method: directly use the imbalanced training set to train the attention network, and then use the trained attention network to classify the test set.

[0141] (2) The eight resampling techniques can be further divided into classic resampling techniques and deep learning-based resampling techniques. Classic resampling techniques include four oversampling techniques: ADASYN, SMOTE, Borderline-SMOTE (BSMOTE), and KMeans-SMOTE (KSMOTE), and two hybrid sampling techniques: SMOTE-Tomek and SMOTE-ENN. Deep learning-based resampling techniques include VAE and GAN. The specific approach is to first use resampling technology to balance the training set. Then, use the balanced training set to train the attention network. Finally, use the trained attention network to classify the test set.

[0142] Table 3: TPR of various anomaly detection models

[0143] Method <![CDATA[1 st -Fold]]> <![CDATA[2 nd -Fold]]> <![CDATA[3 rd -Fold]]> <![CDATA[4 th -Fold]]> <![CDATA[5 th -Fold]]> Average SANet 84.62 71.54 76.15 59.17 57.5 69.79 ADASYN 90 76.15 70 60.83 51.67 69.73 SMOTE 91.54 83.08 72.31 55.83 56.67 71.88 BSMOTE 90.77 78.46 77.69 56.67 55 71.72 KSMOTE 86.15 76.92 80.77 60 56.67 72.1 SMOTE-Tomek 92.31 81.54 73.08 60 60 73.38 SMOTE-ENN 92.31 86.15 83.08 54.17 73.33 77.81 VAE 74.62 81.54 58.46 58.33 40 62.59 GAN 81.54 73.85 75.38 62.5 60.83 70.82 Developed 92.31 99.23 93.08 75 88.33 89.59

[0144] Table 3 shows the TPR of various methods. TPR can be used to compare the ability of different methods to diagnose abnormal samples. As can be seen, for abnormal samples, the method proposed in this application (i.e., Developed in Table 3) surpasses the SANet method and eight compared resampling methods in all five fold experiments. Furthermore, in the second fold experiment, its TPR approaches 100%, an improvement of 13.08% over the second-place method.

[0145] Table 4: TNR of various anomaly detection models

[0146] Method <![CDATA[1 st -Fold]]> <![CDATA[2 nd -Fold]]> <![CDATA[3 rd -Fold]]> <![CDATA[4 th -Fold]]> <![CDATA[5 th -Fold]]> Average SANet 98.80 98.80 99.79 97.8 99.74 98.98 ADASYN 99.01 98.64 99.53 97.49 99.42 98.82 SMOTE 99.01 98.64 99.32 97.64 99.63 98.85 BSMOTE 98.69 98.74 99.27 97.85 99.68 98.85 KSMOTE 98.90 98.32 99.58 97.64 99.11 98.71 SMOTE-Tomek 98.80 98.74 99.32 98.01 99.63 98.9 SMOTE-ENN 98.01 98.12 99.32 97.33 99.16 98.39 VAE 98.27 97.91 98.22 98.43 97.74 98.11 GAN 99.21 98.85 99.42 97.75 99.84 99.02 Developed 91.83 94.19 93.93 95.65 94.16 93.95

[0147] Table 4 shows the TNRs of various methods. TNR can be used to compare the ability of different methods to diagnose normal samples. It can be seen that for normal samples, although the method proposed in this application (i.e., Developed in Table 4) is not as good as the comparison method, its TNR also exceeds 91.83%. It is worth noting that the TPR of the comparison method is much lower than the TNR, which shows that the comparison method is more biased towards normal samples and ignores abnormal samples. This is obviously not suitable for aircraft engines with extremely strict safety requirements.

[0148] Furthermore, the average values ​​of TPR, TNR, and Balanced Accuracy are summarized in Table 5.

[0149] In summary:

[0150] (1) When using TPR and Balanced Accuracy as evaluation metrics, seven resampling techniques outperformed the SANet method, and one was very close. Furthermore, when using TNR, eight resampling techniques were very close to the SANet method. This shows that appropriate resampling techniques can improve the performance of classification networks on imbalanced datasets by balancing the training set.

[0151] (2) When using TPR and Balanced Accuracy as evaluation metrics, the hybrid sampling techniques SMOTE-Tomek and SMOTE-ENN surpass the popular oversampling techniques ADASYN, SMOTE, BSMOTE, KSMOTE, VAE, and GAN. Furthermore, when using TNR, they are very close to the aforementioned oversampling techniques. This demonstrates the advantages of hybrid sampling techniques.

[0152] (3) Compared with the comparison methods considered, the proposed method achieved the best average diagnostic results in terms of balanced accuracy. In other words, the proposed method has the best comprehensive diagnostic capability. Moreover, compared with the hybrid sampling techniques SMOTE-Tomek and SMOTE-ENN, the proposed method achieved an improvement of 5.63% and 3.67% in terms of balanced accuracy, respectively. This verifies the effectiveness of the proposed method.

[0153] Table 5: Average TPR, TNR, Balanced Accuracy, and Inference Time of various anomaly detection models

[0154]

[0155] Table 5 also shows the inference time of the anomaly detection methods on the test set. The Baseline method has the fastest diagnosis speed, inferring 204 test samples in just 1.399 milliseconds. This application (Developed in Table 5) takes 1.473 seconds, a difference of only 0.074 milliseconds, indicating that this application also has a very fast diagnosis speed.

[0156] This application proposes a new attention-based data augmentation method, and uses it together with clustering-based undersampling technology to balance the training set. First, a clustering algorithm is used to cluster normal samples into K clusters to form cluster centers. The cluster centers and abnormal samples are then combined to form an undersampled training set X. Second, an encoder network is used to map the training set X into a feature space with intra-class aggregation and inter-class separation. DA-SMOTE is used to adaptively generate abnormal sample features Z in the learned feature space. new , balance the under-sampling training set X; use the Decoder network to generate abnormal sample features Z new Mapping back to the original space to generate abnormal samples Abnormal samples Together with the undersampled training set X, a balanced training set is formed. Finally, the balanced training set is used to train the attention network, and the trained attention network is used as a classification network to diagnose whether the engine has an abnormality. This solves the imbalance problem under the conditions of intra-class dispersion and inter-class aliasing. This application adopts clustered undersampling technology to reduce the imbalance between abnormal samples and normal samples; this application adopts clustered oversampling technology to balance the training set and keep the generated abnormal samples away from the dangerous area. This method has a better data enhancement effect.

[0157] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for aircraft engine anomaly detection based on deep attention data enhancement, characterized by: The following steps are involved: Step 1: Use clustering algorithm to cluster normal samples into clusters to form cluster centers, and then the cluster centers and abnormal samples constitute the under-sampled training set ; Step 2: First, use the Encoder network to convert the training set Mapping to a feature space where the classes are clustered within a class and separated between classes; then, using the self-attention module in the learned feature space to adaptively generate abnormal sample features ;Finally, use the Decoder network to generate abnormal sample features Mapping back to the original space to generate abnormal samples , abnormal samples With the undersampled training set Construct a balanced training set; Step 3: Use the balanced training set to train the attention network, and use the trained attention network as a classification network to diagnose whether the engine has an abnormality; Adopting self-attention module to adaptively generate abnormal sample features The process is called DA-SMOTE; In step 2, the trained Encoder network is used to convert the undersampled training set Mapped into a feature space where the classes are separated and the classes are clustered, that is, ; DA-SMOTE is used to generate abnormal samples. The process of DA-SMOTE generating abnormal samples is as follows: Step S101: abnormal sample features and adjacent abnormal sample characteristics Splicing to form a feature pair ;in, ; Step S102: feature pair After processing by the attention module, a scalar output is obtained , as shown in formula (5): （5） Where, represents the ReLU activation function, 、 、 The parameter matrix representing the linear mapping; Step S103, output the scalar through the Sigmoid function Transform to As shown in formula (6): （6） Where, is the interpolation factor obtained through learning; Step S104: Generate abnormal sample features As shown in formula (7): （7）。 2. The method for aircraft engine anomaly detection based on deep attention data enhancement according to claim 1, characterized in that: Use the trained Decoder network to enhance the set Mapping back to the original space, that is ; Original space With the training set Together they form a balanced training set .

3. The method for aircraft engine anomaly detection based on deep attention data enhancement according to claim 2, characterized in that: The interpolation factor is learned using two loss functions: reconstruction loss and three-center loss. Reconstruction loss: using class-balanced As the reconstruction loss , as shown in formula (8): （8） Where, yes The elements, and yes The reconstruction representation of represents the number of elements in the collection, and represent , is a hyperparameter, Set to 0.999, when For normal samples, ;when For abnormal samples, ; Three-center loss: The three-center loss is used to guide the learning of feature representation and interpolation factors, as shown in formula (9): （9） Where, yes The elements, is a hyperparameter, Set to 5; when When is a normal sample characteristic, is the characteristic of the normal sample center, and is the characteristic of the center of the abnormal sample; when When is an abnormal sample feature, is the characteristic of the center of the abnormal sample, and is a characteristic of the center of a normal sample; Combining the reconstruction loss and the three-center loss, DA-SMOTE, Encoder network, and Decoder network are trained simultaneously. The loss function is shown in formula (10): （10）。 4. The method for aircraft engine anomaly detection based on deep attention data enhancement according to claim 3, characterized in that: In step 3, the classification network is trained using binary cross entropy loss, as shown in formula (11): （11） Where, yes After the output of the last Softmax layer in the attention network, yes The corresponding output; It is The output probability of a sample, It is The true labels of samples; After calculating the binary cross entropy loss, the backpropagation algorithm is used to optimize the weight parameters of the classification network.

5. The method for aircraft engine anomaly detection based on deep attention data enhancement according to claim 1, characterized in that: In step 1, the clustering algorithm is the K-Means algorithm, which is: Set up a normal sample set , the normal sample set Divided into Cluster, Clusters The samples in the , normal sample set The log-likelihood function of As shown in formula (1): （1） Where, represent A set of normal samples, Represents the dimension of the sample; Representative A set of normal samples; Representative clusters; Representative Clusters The samples in the normal distribution obey expectations; Represents a normal sample set The number of clusters; Maximize the log-likelihood function This is equivalent to minimizing the sum of squared errors of formula (2): （2） Where, The objective function optimized by the K-Means algorithm.

6. The method for aircraft engine anomaly detection based on deep attention data enhancement according to claim 5, characterized in that: The set of clusters calculated using the K-Means algorithm is , as shown in formula (3): （3） Where, Indicates the clusters; The set of cluster centers As shown in formula (4): （4） Where, It is Clusters The cluster center.

7. The method for aircraft engine anomaly detection based on deep attention data enhancement according to claim 6, characterized in that: Preset by The set of abnormal samples is , , the set of cluster centers and abnormal sample sets , forming the under-sampled training set ; The quantitative difference between normal samples and abnormal samples is is reduced to .

Images