A method for small-sample fault diagnosis of aero-engines based on deep twin self-attention networks

Abstract

This application provides a method for small-sample fault diagnosis of aero-engines based on a deep Siamese self-attention network. It combines time-series clustering and Siamese neural networks to diagnose small-sample aero-engine faults. This application addresses the multi-label classification problem under small-sample conditions by learning a similarity metric, effectively improving the accuracy of multi-fault classification under such conditions and mitigating the overfitting problem of deep neural networks. SANet is used as the feature extraction module of the Siamese neural network to learn richer temporal features, thereby improving the separability of different types of samples in the mapping space. SANet can simultaneously extract short-term and long-term dependencies of time series data, and outperforms RNN models in extracting long-term dependencies. This effectively alleviates the training problem caused by imbalanced samples and provides better diagnostic results for fault samples.

Description

Technical Field

[0001] This application belongs to the technical field of engine fault diagnosis methods, and more specifically, it relates to a small-sample fault diagnosis method for aero-engines based on deep twin self-attention networks. Background Technology

[0002] As the primary power source for aircraft, the reliability of aero engines directly impacts flight safety. Operating under high temperature, high pressure, and high speed conditions for extended periods, engines inevitably experience performance degradation and even malfunctions in their components, such as exhaust temperature indication (EGT) and total air temperature (TAT) sensor indication failures. If engine malfunctions are not addressed promptly, they can potentially lead to serious flight accidents and substantial repair costs. Therefore, real-time performance monitoring and fault diagnosis of engines are essential measures to ensure safe flight and reduce maintenance costs.

[0003] Engine fault diagnosis methods are mainly divided into two categories: physical model-based methods and data-driven methods. Physical model-based methods diagnose faults by establishing a physical model of the engine, and the results are generally well-interpretable. However, the application limitations of this method are also quite significant. First, the modeling process requires a large amount of prior knowledge, and may even involve some confidential or unmeasurable design parameters. Second, the damage propagation process and dynamic response of complex equipment are very complex, making the entire modeling process extremely challenging. Finally, physical models are usually built under standard operating conditions; in actual operation, complex environmental factors may lead to lower accuracy of the physical model.

[0004] Conversely, data-driven methods learn a classification model from historical monitoring data and then use this model to infer the engine's health status. As an important branch of data-driven methods, deep learning can automatically learn rich and useful features from historical monitoring data, eliminating the need for manual feature selection and making the fault diagnosis process more direct. In recent years, some scholars have conducted various studies on fault diagnosis based on deep learning methods. For example, some scholars have developed a deep residual shrinkage network, which reduces noise or redundant information in features by adding a soft threshold, improving the fault diagnosis effect for rotating machinery. Some scholars use a two-layer hierarchical convolutional neural network to extract features with higher discriminative power, thereby improving the prediction accuracy of fault classification and fault grading. Some scholars use a bidirectional GRU network to automatically learn features from monitoring data and have verified the effectiveness and generalization of the proposed model on three types of mechanical parts: cutting tools, gearboxes, and bearings. Other scholars have constructed a deep belief network using a conditionally constrained Boltzmann mechanism and accurately predicted the health status of hydraulic pumps. Therefore, under the condition of balanced and sufficient sample label categories, the effectiveness of deep learning methods in mechanical fault diagnosis has been fully validated.

[0005] However, for engines, tagged fault data is very limited. On the one hand, engines are relatively mature industrial equipment, and the number of failures they experience is relatively low. On the other hand, engines are complex pieces of equipment with many components, and the types of faults they experience are numerous, making it difficult to collect sufficient samples of the same type of fault. Figure 1 As shown, this application takes two common faults of 63 engines of a certain airline as examples and collected a total of 1,047 samples, including 984 normal samples, 38 exhaust temperature indication fault samples and 25 inlet total temperature indication fault samples.

[0006] Therefore, engine fault diagnosis is a small-sample, multi-label classification problem under imbalanced conditions. Directly using deep learning for fault diagnosis will encounter the following challenges:

[0007] (1) Multi-label classification problem under small sample conditions. Modern deep neural networks contain a large number of parameters that need to be optimized. If only a small number of labeled samples are used to train a deep neural network based on cross-entropy, the deep neural network is prone to overfitting. Because engine monitoring data contains very few fault samples, if only a small number of labeled fault samples are used to train a deep neural network based on cross-entropy, the deep neural network is prone to overfitting, resulting in low diagnostic accuracy of fault samples.

[0008] (2) Multidimensional Time Series Problem. Engine monitoring data is a typical example of multidimensional time series. Current deep neural networks, such as RNNs, still have the drawback of not being able to extract both short-term and long-term dependencies simultaneously when processing multidimensional time series. In other words, it is difficult to extract all the time-series features from the monitoring data.

[0009] (3) Imbalanced Samples. The number of faulty samples in a typical training set is far less than the number of normal samples. For example, a diagnostic training set might include 787 normal samples, 30 exhaust temperature indicator faulty samples, and 20 inlet total temperature indicator faulty samples. This results in 310,753 sample pairs belonging to the same type, while only 39,950 samples belong to different types, with an imbalance ratio of approximately 7.78:1. The imbalance problem remains unresolved. Therefore, during training, the gradient of normal samples will dominate the total gradient of the deep neural network based on cross-entropy, causing the deep neural network model to focus more on training normal samples. In other words, even if the deep neural network diagnoses all training samples as normal, the loss function can be very small. In this case, faulty samples are easily misclassified as normal samples.

[0010] Therefore, this application proposes an aero-engine diagnostic method that can effectively improve the accuracy of multi-fault classification under small sample conditions. It can effectively extract complex temporal features from multi-dimensional time series, thereby making the extracted high-dimensional features more separable; effectively alleviate the training problem caused by sample imbalance, and improve the overall accuracy of engine fault diagnosis. Summary of the Invention

[0011] To achieve the above objectives, the technical solution adopted in this application is: to provide a method for diagnosing small-sample faults in aero-engines based on a deep twin self-attention network, which combines time series clustering and twin neural networks to achieve the diagnosis of small-sample faults in aero-engines.

[0012] Optionally, a training phase may be included, which includes the following steps:

[0013] Step 1: Collect and process aircraft engine condition monitoring data, and divide the collected data into: normal samples, EIF samples, and TSIF samples. Proceed to Step 2.

[0014] Step 2: Use the K-Means method to analyze normal samples. Clustering is performed to obtain the set of cluster centers (CPD) of normal samples. NOR Proceed to step three;

[0015] Step 3: Set the cluster centers (CPD) of normal samples. NORThe set of EIF samples and the set of TSIF samples are paired to construct the set of sample pairs SD for training DSSAN. (TRAIN) Proceed to step four;

[0016] Step 4: Train the DSSAN model for 1 epoch;

[0017] The feature extraction module will extract sample pairs They are mapped to the same feature space, thereby generating their respective embedded representations. The similarity calculation module calculates the distance between embedded representations. Based on the contrastive loss function L sia Optimize the DSSAN model and proceed to step five;

[0018] Step 5: If the training accuracy is 100% or Epoch > 20, proceed to Step 6; otherwise, proceed to Step 4.

[0019] Step Six: Use the trained model to diagnose aircraft engine faults.

[0020] Optionally, a test phase may also be included;

[0021] Segment testing phase: For any test sample With set Pair each element in the SD with another element, calculate the Euclidean distance between the paired samples, and then find the element in the SD that is paired with the element in the SD. The closest element Final judgment With FSS * They belong to the same type.

[0022] Optionally, in step one, the collected data includes exhaust temperature margin (EGTM), exhaust temperature deviation (DEGT), core engine speed deviation (DN2), and fuel flow deviation (DFF).

[0023] Samples are constructed based on the collected exhaust temperature margin (EGTM), exhaust temperature deviation (DEGT), core engine speed deviation (DN2), and fuel flow deviation (DFF). The sample construction process includes the following steps:

[0024] Step 101: Determine the time of engine failure based on the maintenance report, thereby obtaining historical monitoring data of its key state characteristic parameters during its operation on the wing. The key state characteristic parameters of the k-th engine are shown below:

[0025]

[0026] In the formula, m kThis represents the total number of flight cycles before the k-th engine failed.

[0027] Step 102: Construct fault symptom samples from key state feature parameters using a sliding window. The sliding window size is set to 10 x 4, the step size is set to 5, and the fault symptom samples are located at the tail of the key state feature parameters. The sliding window slides from right to left. The p-th fault symptom sample of the k-th engine is shown below:

[0028]

[0029] When p=1, FSS k，p Indicates a fault sample; when At that time, FSS k，p Indicates a normal sample;

[0030] Step 103, denote the set of normal samples as:

[0031] This represents the i-th normal sample;

[0032] The set of exhaust temperature indication fault samples is denoted as:

[0033] This represents the i-th exhaust temperature indication fault sample;

[0034] The set of samples of imported total temperature indicator malfunctions is denoted as:

[0035] This represents the i-th faulty sample of the inlet total temperature indicator;

[0036] N NOR N EIF and N TSIF These represent the total number of samples of each type;

[0037] The dataset is divided using a five-fold crossover method. For each partition, N NOR N EIF and N TSIF Divide it into two parts in an 8:2 ratio.

[0038] Right now

[0039] and

[0040] The training set is then:

[0041]

[0042] The test set is:

[0043]

[0044] Optionally, in step two, the K-Means method is used for clustering normal samples to form the set of cluster centers CPD for normal samples. NOR ;

[0045] For normal samples The method for clustering is as follows:

[0046] Assumption Clustering There are clusters, and the set of clusters is... The goal of clustering is to minimize the sum of squared intra-cluster distances:

[0047]

[0048] In the formula, For the i-th cluster Cluster centers.

[0049] Optionally, in step three, the set of normal sample cluster centers CPD obtained from step two clustering... NOR Together with the EIF sample set and the TSIF sample set, they form the sample pair set SD used to train the DSSAN model. (TRAIN) :

[0050] CPD was calculated NOR Next, a set of sample pairs, denoted as SD, needs to be constructed for training the Siamese neural network. (TRAIN) ,but

[0051]

[0052] And sample pairs The tag y k The rule for setting the label is as follows: when the two samples in a sample pair belong to the same type, the corresponding label is 1; when the two samples in a sample pair belong to different types, the corresponding label is 0.

[0053]

[0054] If CPD NOR The number of samples and and If the number of samples in each sample is approximately equal, then the number of sample pairs with the label 1 is:

[0055]

[0056] The number of sample pairs with a label of 0 is:

[0057]

[0058] The imbalance rate of sample pairs labeled 1 and 0 is approximately:

[0059]

[0060] Optionally, in step four, the DSSAN model includes a feature extraction module and a similarity calculation module, and the feature extraction module includes two SANets;

[0061] The feature extraction module is composed of two SANets with identical structures and shared weights, and is presented in the form of sample pairs. As input, learn a mapping F W (·) maps sample pairs to the same feature space, thereby generating their respective embedded representations.

[0062] Optionally, SANet includes two encoder layers and one fully connected layer;

[0063] Each Encoder includes a multi-head attention mechanism, two residual connections and layer regularization, and a fully connected feedforward layer;

[0064] Multi-head attention mechanisms include multiple self-attention modules;

[0065] The functional expression for the multi-head attention mechanism is:

[0066] MH k =MultiHead(Q k K k V k =Concat(H) k，1 H k，2 H k，h W O

[0067] and

[0068]

[0069] In the formula, Q k K k V k Is it input FSS? k EFSS is obtained after encoding. k Then, after a linear transformation, the Query, Key, and Value, i.e., Q, are obtained. k =EFSS k W Q K k =EFSS k WK V k =EFSS k W V W Q W K W V , W O Represents a linear mapping. It is Q k Dimensions.

[0070] Optionally, layer regularization ensures that all nodes in the same layer share the same mean and variance, while different samples can have different means and variances. The formula for calculating layer regularization is as follows:

[0071]

[0072]

[0073] In the formula, It is MH k The component at the i-th node of this layer, N h This indicates the number of hidden nodes in this layer;

[0074] The fully connected feedforward layer consists of two linear transformations, calculated as follows:

[0075] FFN(MH k )=(MH k W1+b1)σ(1.702(MH k W1+b1))W2+b2

[0076] In the formula, W1 and W2 represent linear transformations, b1 and b2 represent biases, and σ(·) represents the cumulative distribution function of the Logistic Distribution.

[0077] Optionally, the similarity calculation module represents the similarity between sample pairs by calculating a simple distance between embedded representations, using Euclidean distance D. E (·,·);

[0078] The definition of adversarial loss is as follows:

[0079]

[0080] In the formula, d represents a threshold that restricts the Euclidean distance between different types of embedded representations to the range [0, d]. That is, when the distance exceeds d, the loss is set to 0.

[0081] During training, if the sample pairs If they belong to the same type, then the distance It will get smaller and smaller, that is The similarity between samples increases; conversely, the similarity between sample pairs decreases. Not of the same type, distance It will get bigger and bigger, that is The similarity between them decreases.

[0082] This application proposes a method for small-sample fault diagnosis of aero-engines based on a deep Siamese self-attention network. The core of this method is the Siamese neural network, which learns a similarity metric to solve the multi-label classification problem under small-sample conditions. The beneficial effects of this application are:

[0083] (1) This application proposes a new fault diagnosis framework, namely the TSC-DSSAN model, which can effectively improve the accuracy of multi-fault classification under small sample conditions.

[0084] (2) The feature extraction module of the Siamese neural network is composed of SANet, which can extract richer temporal features from multidimensional time series. In engine fault diagnosis, on the one hand, the input of the Siamese neural network changes from samples to sample pairs. By pairing samples, the number of samples in the training set can be increased quadratically. Therefore, the number of samples in the training set is greatly expanded, which can effectively alleviate the overfitting problem of deep neural networks. On the other hand, during the training phase, the Siamese neural network minimizes the adversarial loss so that the embedded representations of sample pairs of the same type are close to each other in the mapping space, while the embedded representations of sample pairs of different types are far apart. That is to say, fault samples and normal samples belong to different types of sample pairs and will be far apart in the mapping space, which can enhance their separability. SANet can extract both short-term and long-term dependencies of the time series at the same time and is superior to RNN models in extracting long-term dependencies. This application uses SANet as the feature extraction module of the Siamese neural network in order to learn richer temporal features and thus improve the separability of different types of samples in the mapping space.

[0085] (3) Using sample pairs consisting of cluster centers of normal samples and faulty samples to train DSSAN can effectively alleviate the training problem caused by sample imbalance. Normal samples are clustered, and then the cluster center set is paired with the faulty sample set to form sample pairs. If the number of cluster centers is approximately equal to the number of faulty samples, then the imbalance rate between sample pairs of the same type and different types is approximately 1:1, which can effectively alleviate the training problem caused by sample imbalance.

[0086] (4) The proposed model was validated using real monitoring data from the engine, showing that the TSC-DSSAN model has a better diagnostic effect on fault samples. Attached Figure Description

[0087] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0088] Figure 1 Visualization of the collected samples based on t-SNE;

[0089] Figure 2 (a) is a schematic diagram of the structure of an aero-engine;

[0090] Figure 2 (b) is a flowchart of the processing of aero-engine performance test data;

[0091] Figure 3 DSSAN model framework;

[0092] Figure 4 This is a basic flowchart of the fault diagnosis method;

[0093] Figure 5 TER for the methods considered on the test set;

[0094] Figure 6 TTR for the methods considered on the test set;

[0095] Figure 7 The confusion matrix diagram for the methods considered on the test set;

[0096] Figure 8 A visualization of the high-dimensional features of the test samples based on t-SNE. Detailed Implementation

[0097] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0098] This application now describes a method for diagnosing small-sample faults in aero-engines based on a deep Siamese self-attention network. This method combines time-series clustering and Siamese neural networks to diagnose small-sample faults in aero-engines.

[0099] It includes a training phase and a testing phase. The training phase includes the following steps, such as... Figure 4 As shown:

[0100] Step 1: Collect and process aircraft engine condition monitoring data, and divide the collected data into: normal samples, EIF samples, and TSIF samples. Proceed to Step 2.

[0101] Step 2: Use the K-Means method to analyze normal samples. Clustering is performed to obtain the set of cluster centers (CPD) of normal samples. NOR Proceed to step three;

[0102] Step 3: Set the cluster centers (CPD) of normal samples. NOR The set of EIF samples and the set of TSIF samples are paired to construct the set of sample pairs SD for training DSSAN. (TRAIN) Proceed to step four;

[0103] Step 4: Train the DSSAN model for 1 epoch;

[0104] The feature extraction module will extract sample pairs They are mapped to the same feature space, thereby generating their respective embedded representations. The similarity calculation module calculates the distance between embedded representations. Based on the contrastive loss function L sia Optimize the DSSAN model and proceed to step five;

[0105] Step 5: If the training accuracy is 100% or Epoch > 20, proceed to Step 6; otherwise, proceed to Step 4.

[0106] Step Six: Use the trained model to diagnose aircraft engine faults.

[0107] Testing phase: For any test sample With set Pair each element in the SD with another element, calculate the Euclidean distance between the paired samples, and then find the element in the SD that is paired with the element in the SD. The closest element Final judgment With FSS * They belong to the same type.

[0108] In step one, the collected data include exhaust temperature margin (EGTM), exhaust temperature deviation (DEGT), core engine speed deviation (DN2), and fuel flow deviation (DFF).

[0109] Figure 2(a) shows the basic configuration of the turbofan engine studied in this application, mainly composed of units such as a fan, low-pressure compressor, high-pressure compressor, combustion chamber, high-pressure turbine, and low-pressure turbine. Related research indicates that the state characteristic parameters of an engine can reflect the performance state of the entire engine and its units, and extracting fault symptoms (fault characteristics) from these parameters is a key step in fault diagnosis. Among the many state characteristic parameters of a turbofan engine, the most important and effective are exhaust gas temperature (EGT), low-pressure rotor speed (N1), high-pressure rotor speed (N2), and fuel flow rate (FF).

[0110] In actual flight, the measured values ​​of the aforementioned state characteristic parameters are also affected by factors such as external environmental conditions, operating status, and engine thrust. Directly using the measured values ​​may lead to misjudgments of the engine's performance status. To more accurately reflect the engine's performance status, original equipment manufacturers (OEMs) combine engine performance baseline models, external environmental conditions, and engine thrust magnitude to convert measured values ​​into baseline deviation values. For example... Figure 2 As shown in (b), GE converts exhaust temperature, core engine speed, and fuel flow rate into exhaust temperature margin (EGTM), exhaust temperature deviation (DEGT), core engine speed deviation (DN2), and fuel flow deviation (DFF) based on parameters such as flight altitude, Mach number, and thrust. Then, it combines these parameters with fingerprint diagrams for engine fault diagnosis. Therefore, this application also selects EGTM, DEGT, DN2, and DFF as key characteristic parameters (KCP) for engine fault diagnosis.

[0111] Samples are constructed based on the collected exhaust temperature margin (EGTM), exhaust temperature deviation (DEGT), core engine speed deviation (DN2), and fuel flow deviation (DFF). The sample construction process includes the following steps:

[0112] Step 101: Determine the time of engine failure based on the maintenance report, thereby obtaining historical monitoring data of its key state characteristic parameters during its operation on the wing. Therefore, the key state characteristic parameters of the k-th engine are as follows:

[0113]

[0114] In the formula, m k This represents the total number of flight cycles before the k-th engine fails.

[0115] Step 102: Construct fault symptom samples from the key state feature parameters using a sliding window. The sliding window size is set to 10 x 4, and the step size is set to 5. The fault symptom samples are located at the end of the key state feature parameters, and the sliding window slides from right to left for easy analysis. Therefore, the p-th fault symptom sample of the k-th engine is shown below:

[0116]

[0117] When p=1, FSS k，p Indicates a fault sample; when At that time, FSS k，p This indicates a normal sample.

[0118] Step 103, denote the set of normal samples as:

[0119] This represents the i-th normal sample;

[0120] The set of exhaust temperature indication fault samples is denoted as:

[0121] This represents the i-th exhaust temperature indication fault sample;

[0122] The set of samples of imported total temperature indicator malfunctions is denoted as:

[0123] This represents the i-th inlet total temperature indicator fault sample.

[0124] In addition, N NOR N EIF and N TSIF These represent the total number of samples of each type.

[0125] Given the small number of faulty samples, this application employs a five-fold cross partitioning method to divide the dataset. For each partition, N... NOR N EIF and N TSIF Divide it into two parts in an 8:2 ratio.

[0126] Right now

[0127] and

[0128] The training set is then:

[0129]

[0130] The test set is:

[0131]

[0132] The five-fold cross method for partitioning the dataset can be implemented using existing technologies and is not an innovation of this application, so it will not be elaborated further.

[0133] In step two, to alleviate the problem of imbalanced samples and considering the local spatial clustering of normal samples, clustering is performed on the normal samples. Normal samples within the same cluster exhibit high similarity, while those in different clusters show significant dissimilarity. Furthermore, the set of cluster centers effectively represents the original normal samples. This application employs the K-Means method for clustering normal samples, with the set of cluster centers (CPD) constituting the normal sample clusters. NOR .

[0134] For normal samples The method for clustering is as follows:

[0135] Assumption Clustering There are clusters, and the set of clusters is... The goal of clustering is to minimize the sum of squared intra-cluster distances:

[0136]

[0137] In the formula, For the i-th cluster Cluster centers.

[0138] In step three, the set of normal sample cluster centers CPD obtained from the clustering in step two is... NOR Together with the EIF sample set and the TSIF sample set, they form the sample pair set SD used to train the DSSAN model. (TRAIN) .

[0139] For Siamese neural networks, the input data during training consists of sample pairs and their corresponding labels. The CPD is then calculated. NOR Next, a set of sample pairs, denoted as SD, needs to be constructed for training the Siamese neural network. (TRAIN) ,but

[0140]

[0141] And sample pairs The tag y k The rule for setting the label is as follows: when the two samples in a sample pair belong to the same type, the corresponding label is 1; when the two samples in a sample pair belong to different types, the corresponding label is 0.

[0142]

[0143] If CPD NOR The number of samples and and If the number of samples in each sample is approximately equal, then the number of sample pairs with label 1 is .

[0144] The number of sample pairs with label 0 is The imbalance rate of sample pairs labeled 1 and 0 is approximately:

[0145]

[0146] Therefore, this can effectively alleviate the training problems caused by imbalanced samples.

[0147] In step four, the DSSAN model includes a feature extraction module and a similarity calculation module. The feature extraction module includes two SANets.

[0148] The feature extraction module is composed of two SANets with identical structures and shared weights, and is presented in the form of sample pairs. As input, learn a mapping F W (·) maps sample pairs to the same feature space, thereby generating their respective embedded representations.

[0149] The SANet used in this application is... Figure 3 As shown, SANet assigns greater weight to important information in the input and less weight to secondary information. During training, the model focuses more on important information and ignores secondary information, thus improving the accuracy of feature extraction. Furthermore, the attention score allows analysis of the relationship between input and output, enhancing the model's interpretability. This application uses SANet to learn complex temporal features from engine monitoring data, and its structure is as follows: Figure 3 As shown, it includes two encoder layers and one fully connected layer.

[0150] Each Encoder layer includes a multi-head attention mechanism, two residual connections and layer regularization, and a fully connected feedforward layer.

[0151] Multi-head attention is a core component of SANet, consisting of multiple self-attention modules. Different self-attention modules focus on the important information represented by the time series in different mapping subspaces, i.e., different temporal patterns, thus learning richer temporal features. The functional expression of the multi-head attention mechanism is:

[0152] MH k =MultiHead(Q k K k V k=Concat(H) k，1 H k，2 H k，h W O

[0153] and

[0154]

[0155] In the formula, Q k K k V k Is it input FSS? k EFSS is obtained after encoding. k Then, after a linear transformation, the Query, Key, and Value, i.e., Q, are obtained. k =EFSS k W Q K k =EFSS k W K V k =EFSS k W V W Q W K W V , W O Represents a linear mapping. It is Q k Dimensions.

[0156] Layer regularization normalizes data so that all nodes in the same layer share the same mean and variance, while different samples can have different means and variances. Layer regularization can significantly reduce training time. The formula for calculating layer regularization is shown below:

[0157]

[0158]

[0159] In the formula, It is MH k The component at the i-th node of this layer, N h This indicates the number of hidden nodes in this layer.

[0160] The fully connected feed-forward layer (FFN) consists of two linear transformations. To enhance the model's non-linear learning capability, a GELU activation function is added between the two linear transformations. The calculation formula for this layer is shown below:

[0161] FFN(MHk )=(MH k W1+b1)σ(1.702(MH k W1+b1))W2+b2

[0162] In the formula, W1 and W2 represent linear transformations, b1 and b2 represent biases, and σ(·) represents the cumulative distribution function of the Logistic distribution.

[0163] The similarity calculation module represents the similarity between sample pairs by calculating the simple distance between embedded representations, using Euclidean distance D. E (·,·).

[0164] Adversarial loss can directly process paired data and is frequently used in the training of Siamese neural networks. The definition of adversarial loss is as follows:

[0165]

[0166] In the formula, d represents a threshold that restricts the Euclidean distance between different types of embedded representations to the range [0, d]. That is, when the distance exceeds d, the loss is set to 0.

[0167] During training, if the sample pairs If they belong to the same type, then the distance It will get smaller and smaller, that is The similarity between samples increases; conversely, the similarity between sample pairs decreases. Not of the same type, distance It will get bigger and bigger, that is The similarity between input samples decreases as they grow. Therefore, the DSSAN model does not directly predict the classification label of the input samples, but instead learns a similarity measure between inputs.

[0168] In this application, to ensure the effectiveness of the proposed fault diagnosis model by collecting engine monitoring data, the data is derived from actual operational data from an Asian airline, rather than simulated data. According to the maintenance report, a total of 1047 samples were collected under three conditions, including 984 normal samples, 38 exhaust temperature indication fault samples, and 25 inlet total temperature indication fault samples. Figure 1As shown in Table 1, the dataset was then partitioned using a five-fold cross-validation method. Unless otherwise specified, the number of samples in the training set is based on the first fold. In the training set, the imbalance rate between normal samples and exhaust temperature indicator fault samples is 787 / 30≈26.2, while the imbalance rate between normal samples and inlet total temperature indicator fault samples is even higher at 787 / 20≈39.3, indicating that the training set is extremely imbalanced. After clustering the normal samples, the cluster center set (30 cluster centers) and fault samples were used to generate sample pairs for training the Siamese neural network, resulting in 3 sets of 2200 sample pairs of the same type and 3 sets of 2100 sample pairs of different types, as shown in Table 2. On the one hand, the reconstructed dataset contains 4300 samples, significantly expanding the sample size. On the other hand, the imbalance rate between sample pairs of the same type and sample pairs of different types is only... Therefore, the reconstructed training set is almost balanced.

[0169] Table 1: Aircraft Engine Dataset

[0170]

[0171] Table 2: Set of Aero-engine Sample Pairs

[0172] Sample I Sample II Sample pairs Label total NOR NOR <NOR,NOR> 1 30＊30＝900 NOR EIF <NOR,EIF> 0 30＊30＝900 NOR TSIF <NOR,TSIF> 0 30＊20＝600 EIF EIF <EIF,EIF> 1 30＊30＝900 EIF TSIF <EIF,TSIF> 0 30＊20＝600 TSIF TSIF <TSIF,TSIF> 1 20＊20＝400

[0173] Experimental environment: Python 3.7 and PyTorch 1.8.0, processor: Intel i7-9750H, memory: 8G DDR4.

[0174] Optimizer: The neural network was optimized using Adam, with a learning rate of 0.0002, momentum β1 = 0.5, and β2 = 0.999. The batch size was set to 128, and preliminary experiments determined the maximum number of epochs for the TSC-DSSAN model to be 20.

[0175] Hyperparameter settings: This application uses a grid search method to determine the hyperparameters of the TSC-DSSAN model. The number of cluster centers in the clustering algorithm is set to 30. The TSC-DSSAN model contains two encoder layers and one fully connected layer with an output dimension of 64. The multi-head attention mechanism in each encoder layer is a 2-head attention mechanism, and the output dimension of the fully connected feedforward layer is 128.

[0176] If the number of fault samples of each type is not significantly different and is close to the number of normal samples, overall accuracy can be used to evaluate the classification performance of the fault diagnosis model. However, in the test set, the number of fault samples in the engine is much smaller than the number of normal samples, as detailed in Table 1. In this case, overall accuracy has no significant reference value. For example, in the test set, the number of normal samples is 197, while the total number of exhaust temperature indication fault samples and inlet total temperature indication fault samples is only 13. Even if all test samples are diagnosed as normal samples, the overall accuracy is still as high as 93.81%. This application selects a confusion matrix to evaluate the fault diagnosis model, as shown in Table 3.

[0177] Table 3: Confusion Matrix

[0178]

[0179] Furthermore, the diagnostic effectiveness of normal samples, exhaust temperature indicator fault samples, and inlet total temperature indicator fault samples can be evaluated using TNR, TER, and TTR, respectively, while the overall diagnostic effectiveness of the model can be evaluated using the average of the three, denoted as Average accuracy. These indicators can be easily calculated based on the confusion matrix, and the specific calculation formulas are shown below:

[0180]

[0181]

[0182]

[0183]

[0184] To verify the effectiveness of the proposed TSC-DSSAN model, this application constructed three sets of comparative experiments. In the first set of comparative experiments, popular neural network classification models were directly used to diagnose the monitoring data. The selected classification models included convolutional neural networks (ConvNet), RNNs, long short-term memory networks (LSTM), GRU, and SANet. In the second set of comparative experiments, oversampling (OS) was combined with SANet, denoted as OS-SANet. The oversampling technique was implemented using SMOTE from the imblearn package, and after oversampling, the number of samples for each of the three types was 787. In the third set of comparative experiments, downsampling (DS) was combined with SANet, denoted as DS-SANet. The downsampling technique was implemented using ClusterCentroids from the imblearn package, and after downsampling, the number of samples for each of the three types was 20.

[0185] Five experiments were conducted using different training samples, and the TER and TTR of each experiment were recorded as follows: Figures 5 to 6 As shown.

[0186] Table 4 shows the average values ​​of evaluation metrics TNR, TER, and TTR in the five-fold crossover experiment, from which the following conclusions can be drawn. First, compared to ConvNet, RNN, LSTM, and GRU, SANet achieves the best results in average accuracy. More specifically, SANet improves average accuracy by 4.17%, 3.68%, 3.36%, and 0.64% respectively on the test set. Clearly, SANet benefits from its multi-head attention mechanism layer. That is, the multi-head attention mechanism layer can capture more potentially important information from multi-dimensional time series, making it very suitable for engine monitoring data. Second, compared to SANet, OS-SANet and DS-SANet both show significant improvements in diagnostic performance, increasing by 4.44% and 6.61% respectively. On imbalanced datasets, both oversampling and undersampling can bring gains to classification models, with undersampling yielding greater gains. On one hand, this illustrates the importance of balanced datasets, i.e., balancing the numerical differences between faulty and normal samples. On the other hand, it is also one of the important reasons why normal samples are clustered rather than faulty samples when constructing sample pairs. Third, the TSC-DSSAN model achieved the best results in all three evaluation metrics: TER, TTR, and Average accuracy. In particular, in terms of Average accuracy, compared to ConvNet, RNN, LSTM, GRU, SANet, OS-SANet, and DS-SANet, the TSC-DSSAN model improved by 13.54%, 13.05%, 12.73%, 10.01%, 9.37%, 4.93%, and 2.76%, respectively. The Siamese neural network's strategy is to learn a similarity metric rather than directly learning the classification labels of the samples. This demonstrates that this strategy of Siamese neural networks is more effective in solving multi-label classification problems with small samples.

[0187] Table 4: Average values ​​of evaluation metrics TNR, TER, and TTR in the five-fold crossover experiment.

[0188]

[0189]

[0190] Given the poor performance of ConvNet, RNN, LSTM, and GRU, these four models will not be included in the following analysis. To more intuitively observe the fault diagnosis effectiveness of each model, Figure 7The confusion matrices of the four methods (SANet, OS-SANet, DS-SANet, and TSC-DSSAN) presented in the paper are shown on the test set. We can clearly see the diagnostic accuracy of each model on different types of samples. For example, with K=5, the TERs of the four models are 57%, 0.71%, 86%, and 100%, respectively.

[0191] The t-SNE method maps high-dimensional features learned by a neural network model to a two-dimensional space, providing a perspective for observing the distribution of test samples in high-dimensional space. This application uses K=5 as an example for analysis. Figure 8 As shown, for SANet, OS-SANet, and DC-SANet, there is overlap between exhaust temperature indicator fault samples and normal samples, making it easy for fault samples to be misclassified as normal samples. However, for the TSC-DSSAN model, there is almost no overlap between exhaust temperature indicator fault samples and normal samples, making them easier to diagnose.

[0192] To address multi-class classification problems under small sample conditions, this application proposes a novel fault diagnosis framework that combines time-series clustering and Siamese neural networks. In this framework, time-series clustering is used to obtain cluster centers for normal samples. These cluster centers are then paired with faulty samples to form a balanced dataset suitable for training the Siamese neural network. Furthermore, the trained Siamese neural network can measure the similarity between the test sample and samples of different types, thereby determining the specific type of the test sample. In addition, to extract richer temporal features from multi-dimensional time series data, this application uses a self-attention network as the feature extraction module of the Siamese neural network.

[0193] The effectiveness of the TSC-DSSAN model was validated using real monitoring data from gas turbines. Compared with ConvNet, RNN, LSTM, GRU, SANet, OS-SANet, and DS-SANet, the TSC-DSSAN model improved average accuracy by 13.54%, 13.05%, 12.73%, 10.01%, 9.37%, 4.93%, and 2.76%, respectively. This is due to two main factors: firstly, the self-attention network in the model is well-suited for processing gas turbine monitoring data; secondly, the Siamese network effectively measures the similarity between sample pairs, thus determining whether a pair belongs to the same type.

[0194] The fault diagnosis framework proposed in this application not only combines time series clustering and DSSAN, but also employs several tricks to improve fault diagnosis performance under small sample conditions. First, only normal samples are clustered, excluding fault samples, aiming to retain as much fault information as possible while ensuring balanced training samples. Second, an early termination strategy is adopted, terminating training once more than 20 epochs have been completed or the training accuracy reaches 100%, which can alleviate overfitting to some extent. Finally, a five-fold cross-split dataset is used to evaluate the model, avoiding the limitations and randomness of a fixed dataset partition.

[0195] This application proposes a method for small-sample fault diagnosis of aero-engines based on a deep Siamese self-attention network. The core of this method is the Siamese neural network, which learns a similarity metric to solve the multi-label classification problem under small-sample conditions. This application can effectively improve the accuracy of multi-fault classification under small-sample conditions. The feature extraction module of the Siamese neural network is composed of SANet, which can extract richer temporal features from multi-dimensional time series. In engine fault diagnosis, on the one hand, the input of the Siamese neural network changes from samples to sample pairs. Through pairwise pairing of samples, the number of samples in the training set increases quadratically. Therefore, the number of samples in the training set is significantly expanded, which can effectively alleviate the overfitting problem of deep neural networks. On the other hand, during the training phase, the Siamese neural network minimizes adversarial loss, making the embedded representations of sample pairs of the same type close to each other in the mapping space, while the embedded representations of sample pairs of different types are far apart. That is, fault samples and normal samples belong to different types of sample pairs and will be far apart in the mapping space, which can enhance their separability. SANet can simultaneously extract short-term and long-term dependencies of the time series and outperforms RNN models in extracting long-term dependencies. This application uses SANet as the feature extraction module of a Siamese neural network to learn richer temporal features, thereby improving the separability of different types of samples in the mapping space. Training DSSAN using sample pairs consisting of cluster centers of normal samples and faulty samples effectively alleviates the training problem caused by sample imbalance. Normal samples are clustered, and then the cluster center set is paired with the faulty sample set to form sample pairs. If the number of cluster centers is approximately equal to the number of faulty samples, the imbalance rate between sample pairs of the same type and different types is approximately 1:1, thus effectively mitigating the training problem caused by sample imbalance. The proposed model is validated using real engine monitoring data, showing that the TSC-DSSAN model has a better diagnostic effect on faulty samples.

[0196] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for small-sample fault diagnosis of aero-engines based on deep twin self-attention networks, characterized in that: A combination of time series clustering and Siamese neural networks was used to diagnose small-sample faults in aero-engines. The training phase includes the following steps: Step 1: Collect and process aircraft engine condition monitoring data, and divide the collected data into: normal samples, EIF samples, and TSIF samples. Proceed to Step 2. Step 2: Use the K-Means method to analyze normal samples. Perform clustering to obtain a set of cluster centers for normal samples. Proceed to step three; Step 3: Set the cluster centers of normal samples The EIF sample set and the TSIF sample set are paired to construct a sample pair set for training DSSAN. Proceed to step four; Step 4: Train the DSSAN model for 1 epoch; The feature extraction module will extract sample pairs They are mapped to the same feature space, thereby generating their respective embedded representations. The similarity calculation module calculates the distance between the embedded representations. According to the contrastive loss function Optimize the DSSAN model and proceed to step five; Step 5: If the training accuracy is 100% or the Epoch is greater than 20, proceed to Step 6; otherwise, proceed to Step 4. Step Six: Use the trained model to diagnose aircraft engine faults; It also includes the testing phase; Testing phase: For any test sample , and set Pair each element one by one, calculate the Euclidean distance between the paired samples, and then find... Zhongyu The closest element The final judgment and They belong to the same type; In step four, the DSSAN model includes a feature extraction module and a similarity calculation module. The feature extraction module includes two SANets. The feature extraction module is composed of two SANets with identical structures and shared weights, and is presented in the form of sample pairs. For input, learn a mapping This maps sample pairs to the same feature space, thereby generating their respective embedded representations. .

2. The method for small-sample fault diagnosis of aero-engines based on deep twin self-attention networks as described in claim 1, characterized in that: In step one, the collected data includes exhaust temperature margin EGTM, exhaust temperature deviation value DEGT, core engine speed deviation value DN2, and fuel flow deviation value DFF. Samples are constructed based on the collected exhaust temperature margin EGTM, exhaust temperature deviation value DEGT, core engine speed deviation value DN2, and fuel flow deviation value DFF. The sample construction process includes the following steps: Step 101: Determine the time of engine failure based on the maintenance report, thereby obtaining historical monitoring data of its key state characteristic parameters during its operation on the wing. The key state characteristic parameters of the engine are shown below: In the formula, Indicates the first The total number of flight cycles prior to the engine failure; Step 102: Construct fault symptom samples from the key state feature parameters using a sliding window. The sliding window size is set to 10 x 4, and the step size is set to 5. The fault symptom samples are located at the tail of the key state feature parameters. The sliding window slides from right to left. The first engine The following are some examples of fault symptoms: when hour, Indicates a fault sample; when hour, Indicates a normal sample; Step 103, denote the set of normal samples as , Indicates the first One normal sample; The set of exhaust temperature indication fault samples is denoted as . , Indicates the first One sample of exhaust temperature indicator malfunction; Let the set of samples of imported total temperature indicator failures be denoted as . , Indicates the first One sample of an imported total temperature indicator malfunction; , and These represent the total number of samples of each type; The dataset is divided using a five-fold crossover method. For each partition, , and Divide it into two parts in an 8:2 ratio. Right now , and , The training set is ; The test set is .

3. The method for small-sample fault diagnosis of aero-engines based on deep twin self-attention networks as described in claim 1, characterized in that: In step two, the K-Means method is used to cluster normal samples, forming a set of cluster centers for normal samples. ; For normal samples The method for clustering is as follows: Assumption Clustering Clusters, The set of clusters is The goal of clustering is to minimize the sum of squared distances within clusters: In the formula, For the first cluster Cluster centers.

4. The method for small-sample fault diagnosis of aero-engines based on deep twin self-attention networks as described in claim 1, characterized in that: In step three, the set of cluster centers of normal samples obtained from the clustering in step two... Together with the EIF sample set and the TSIF sample set, they form the sample pair set used to train the DSSAN model. ; In the calculation Next, a set of sample pairs needs to be constructed for training the Siamese neural network, denoted as . ,but And sample pairs tags The rule for setting the label is as follows: when the two samples in a sample pair belong to the same type, the corresponding label is 1; when the two samples in a sample pair belong to different types, the corresponding label is 0. if The number of samples and and If the number of samples in each sample is approximately equal, then the number of sample pairs with the label 1 is: ， The number of sample pairs with a label of 0 is: ， The imbalance rate of sample pairs labeled 1 and 0 is approximately: 。 5. The method for small-sample fault diagnosis of aero-engines based on deep twin self-attention networks as described in claim 1, characterized in that: SANet consists of two encoder layers and one fully connected layer; Each Encoder includes a multi-head attention mechanism, two residual connections and layer regularization, and a fully connected feedforward layer; Multi-head attention mechanisms include multiple self-attention modules; The functional expression for the multi-head attention mechanism is: In the formula, It is input After encoding Then, after a linear transformation, the Query, Key, and Value are obtained. , Represents a linear mapping. yes Dimensions.

6. The method for small-sample fault diagnosis of aero-engines based on deep twin self-attention networks as described in claim 5, characterized in that: Layer regularization ensures that all nodes in the same layer share the same mean and variance, while different samples can have different means and variances. The formula for calculating layer regularization is shown below: In the formula, yes On this floor Components on each node This indicates the number of hidden nodes in this layer; The fully connected feedforward layer consists of two linear transformations, calculated as follows: In the formula, Represents a linear transformation. Indicates bias. This represents the cumulative distribution function of the Logistic Distribution.

7. A method for small-sample fault diagnosis of aero-engines based on deep twin self-attention networks as described in claim 1, characterized in that: The similarity calculation module represents the similarity between sample pairs by calculating the simple distance between embedded representations, using Euclidean distance. ; The definition of adversarial loss is as follows: In the formula, This represents a threshold that limits the Euclidean distance between embedded representations belonging to different types to a certain value. Between, that is, when the distance exceeds If so, then its loss is set to 0; During training, if the sample pairs If they belong to the same type, then the distance It will get smaller and smaller, that is The similarity between samples increases; conversely, the similarity between sample pairs decreases. Not of the same type, distance It will get bigger and bigger, that is The similarity between them decreases.

Images