Aero-engine partial unsupervised cross-domain fault diagnosis method based on inter-domain attention mechanism

By designing an inter-domain attention-based adversarial network (IDAAN), the problem of decreased diagnostic performance due to differences in data distribution in cross-domain fault diagnosis of aero-engines was solved. By optimizing feature extraction and health status recognition, higher diagnostic accuracy and adaptability were achieved.

CN116310496BActive Publication Date: 2026-01-09NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310044490.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-01-30
Publication Date
2026-01-09
Estimated Expiration
2043-01-30

AI Technical Summary

Technical Problem

Existing aero-engine fault diagnosis methods suffer from decreased diagnostic performance when faced with differences in cross-domain data distribution, especially when the health status label spaces of the source and target domains are inconsistent. Furthermore, traditional methods have failed to effectively mitigate the negative migration effects of outlier source data.

Method used

We designed an adversarial network (IDAAN) based on the inter-domain attention mechanism. By introducing an inter-domain attention encoding module, we assigned higher weights to source samples in the shared health space. Combining the entropy minimization criterion and adversarial learning, we optimized feature extraction and health status identification, mitigated negative transfer, and improved diagnostic accuracy.

Benefits of technology

It effectively reduces negative migration caused by outlier source samples, improves the accuracy and adaptability of aero-engine fault diagnosis, and performs particularly well in some unsupervised cross-domain scenarios, improving the model's diagnostic accuracy in the target domain.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116310496B_ABST
    Figure CN116310496B_ABST
Patent Text Reader

Abstract

The application provides an aero-engine partial unsupervised cross-domain fault diagnosis method based on an inter-domain attention mechanism, designs an inter-domain attention encoding module, and gives different weights to source samples belonging to an outlier health space and source samples belonging to a shared health space through the inter-domain attention mechanism, so as to reduce the negative transfer caused by the outlier source samples in the domain alignment process. And through the adversarial network, the source samples belonging to the shared health space and the target domain samples are aligned to realize positive transfer. Experiments prove that the application is superior to other similar methods in the aero-engine partial unsupervised cross-domain fault diagnosis task. The method is suitable for real industrial application scenarios, and application of the method can assist researchers to fully utilize a large amount of historical accumulated data to establish a high-accuracy fault diagnosis model suitable for each monitored engine, which has important significance for engine state monitoring and maintenance decision.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application belongs to the field of aero-engine fault diagnosis, and solves the problem of part unsupervised cross-domain fault diagnosis of an aero-engine by designing an adversarial network based on an inter-domain attention mechanism. BACKGROUND

[0002] With the development of artificial intelligence, intelligent fault diagnosis (IFD) technology based on machine learning has made great progress. IFD uses machine learning methods such as support vector machines (SVM), extreme learning machines (ELM), and deep neural networks to mine the potential patterns between measurable parameters and the health status of aero-engine gas path components. Machine learning-based methods use historical aero-engine monitoring data as training data to train models to identify unlabeled data in the engine under current operating conditions. Most of these methods assume that training and testing data come from the same probability distribution, and there is enough labeled data to train a reliable high-precision diagnostic model. However, in actual aero-engine gas path fault diagnosis tasks, due to changes in aero-engine operating conditions, performance degradation, differences in whole machine assembly before and after maintenance, etc., the probability distribution of the data originally used to establish the diagnostic model is usually different from the probability distribution of the new data that needs to be identified, which is called domain shift. In this case, directly applying the old model will result in a decline in diagnostic performance.

[0003] In recent years, unsupervised domain adaptation methods have been widely used in aero-engine fault diagnosis, which can effectively solve the problem of training and testing data coming from different distributions, i.e., cross-domain fault diagnosis. Although unsupervised domain adaptation methods have achieved good results in fault diagnosis, most existing methods assume that the health status label space of the source domain and the target domain is the same. In actual engineering applications, the health status in the target domain is often a subset of the source domain, and there is no prior information about the target health status. This scenario is defined as part unsupervised cross-domain fault diagnosis. In this case, if the standard domain adaptation method is directly applied, the out-of-distribution source data will cause negative transfer. Therefore, in this scenario, the more difficult task is how to mitigate or even eliminate the influence of source label data in the abnormal health status label space to alleviate the negative transfer caused by it.

[0004] To solve this problem, Cao et al. proposed partial adversarial domain adaptation (PADA) in the field of image classification, which adds class-level weights obtained from the target domain to the final optimization objective. Further, Cao et al. constructed a selective adversarial network (SAN) to simultaneously learn target application class-level weights and sample-level weights. In addition, Cao et al. proposed an example transfer network (ETN) that jointly learns domain-invariant representations across domains and quantifies the transferability of source samples through a gradual weighting scheme. In the field of fault diagnosis, Deng et al. proposed a novel generative adversarial network based on double-layer attention, which uses the maximum mean difference (MMD) metric between samples to construct two attention matrices for samples and discriminators. It conducted multiple cross-condition and cross-machine fault diagnosis experiments on three bearing datasets, demonstrating the effectiveness of the method in solving partial cross-domain fault diagnosis problems.

[0005] The above methods all weaken the influence of samples belonging to the source outlier health space on domain alignment by assigning different weights to the final optimization objective, and fail to use deep neural networks themselves to explore the relationship between source samples and target samples. Hou et al. proposed BatchFormer to solve the long-tail recognition problem, which modified the Transformer Encoder and applied it to the batch dimension of each mini-batch to explore the relationship between samples during training through a deep network. These methods have made significant improvements in many tasks such as long-tail recognition and domain generalization, proving the advantage of deep network structure itself in exploring the relationship between samples. Therefore, it is reasonable to believe that a network can be designed to directly explore the relationship between source samples and target samples to better solve the problem of partial cross-domain fault diagnosis of aero-engines. SUMMARY

[0006] To solve the problem of current data-driven methods in the field of aero-engine cross-domain fault diagnosis, the present application proposes an inter-domain attention mechanism-based adversarial network (IDAAN) from the perspective of transfer learning, which can effectively solve the problem of partial cross-domain fault diagnosis of aero-engines.

[0007] The dynamic inter-domain attention mechanism-based adversarial network (IDAAN) comprises four modules, namely, a feature extractor, an inter-domain attention mechanism-based encoding module, a domain discriminator, and a health condition identifier. The IDAAN inherits all advantages of a DANN (Domain-adversarial neural network) network, can learn domain-invariant features in end-to-end adversarial training, and realizes network optimization through a general stochastic gradient descent method. Compared with the DANN network, the IDAAN introduces an IDANN module before the domain discriminator, gives higher weights to source samples belonging to a shared health space, enables the network to focus on the source samples belonging to the shared health space when performing domain alignment, effectively slows down negative transfer caused by outlying source samples. In addition, in the final optimization target of the health condition identifier, a semi-supervised optimization term of unlabeled target domain data is introduced according to the entropy minimization criterion, which improves the final diagnostic accuracy of the model on the target and data to a certain extent.

[0008] The application provides an inter-domain attention mechanism-based aero-engine partial unsupervised cross-domain fault diagnosis method, which comprises the following steps:

[0009] Step 1: normalizing collected source domain and target domain data as training set data; the source domain and target domain data are aero-engine state parameters;

[0010] Necessary notes of step 1: the source domain and target domain data have the same feature space, that is, But the target domain health space is a subset of the source domain health space, that is, We denote the shared health space of the source domain and the target domain as , and the outlying health space of the source domain as In addition, the probability distribution p of the source data is different from the probability distribution q of the target data, that is, p≠q. And in the shared health state space , the probability distribution p of the source domain data is different from the probability distribution q of the target domain data, that is,

[0011] Step 2, constructing an inter-domain attention mechanism-based domain adversarial network (IDAAN);

[0012] The method comprises a feature extractor G f , an inter-domain attention encoding module G a , a health state identifier G y , and a domain discriminator G d ; the feature extractor G f extracts source sample data features f s and target data features f t , and f​s with f t respectively into the health status recognizer G y classifies the health status of source domain and target domain samples;

[0013] with f s and f t into the inter-domain attention encoding module G a , and the source sample feature f s is weighted through the inter-domain attention mechanism to obtain the weighted source sample feature f sa .

[0014] with f sa and f t into the domain discriminator G d , which classifies the domain to which the data belongs.

[0015] Step 2: The feature extractor G f is an improved 1-dimensional AlexNet network used to extract features from the input monitoring time series segment data. It includes three convolutional pooling modules and two independent convolutional layers, each followed by a linear rectifier unit (ReLU) as the activation function. In order to avoid overfitting, BatchNorm operation is performed after using the ReLu activation. The health status recognizer is used to identify the health pattern. The domain discriminator discriminates whether the data comes from the source domain or the target domain, a total of two. The last layer of the health status recognizer G y uses a softmax function for multi-classification. The last layer of the domain discriminator G d uses sigmoid as the activation function for domain binary classification. Domain alignment, i.e., knowledge transfer, is achieved through adversarial learning between the feature extractor and the domain discriminator. The feature extractor maximizes the domain classification loss, while the domain discriminator minimizes the domain classification loss.

[0016] In particular, the inter-domain attention encoding module is composed of two encoders based on the inter-domain attention mechanism. The inter-domain attention mechanism assumes that the difference between the source data in the outlier health space and the target domain data is significantly greater than the difference between the source data in the shared health space and the target data. By weighting the source sample through the inter-domain attention mechanism, the proportion of outlier source samples in the domain alignment process can be effectively reduced, effectively slowing down the negative transfer caused by them. Specifically, we first design a scaling dot product inter-domain attention mechanism, whose input is n s ×d m dimensional source sample feature S and n t ×d m dimensional target sample feature T, where d m is the size of the sample feature, and n s is the number of dimensions.where n and n are the number of source and target samples respectively. The calculation expression of inter-domain attention matrix A is:

[0017]

[0018] where the dimension of A is n s ×n t , and the softmax function is used to calculate the correlation (attention score) between each target and source sample. Dot product scaling is performed by to avoid too high feature dimension making dot product result too large to push the softmax function to the area of very small gradient. Here we assume that the correlation of target samples with source samples in the shared health status label space is higher than the correlation of source samples in the outlier health status label space. Further, the weight vector of source samples with dimension n s is obtained by adding each column element of inter-domain attention matrix A. And the final weight vector w of source samples is obtained by normalizing the weight vector, and the calculation expression is as follows:

[0019]

[0020] And the output S a of the final basic scaled dot product inter-domain attention mechanism can be obtained by:

[0021] S a =repeat(w,d m )⊙S

[0022] where repeat(w,d m ) represents expanding the source sample weight vector w into a matrix with dimension n s ×d m , and represents Hadamard product.

[0023] Further, we improve the basic scaled dot product inter-domain attention mechanism to multi-head inter-domain attention mechanism, so that the output of the attention layer contains more subspace encoding representation information, to increase the expression ability of the model. Specifically, the input query T, key S and value S are projected to d q , d k and d m dimensions respectively through different fully connected layers. Then the scaled dot product inter-domain attention mechanism ID_Attention is executed in parallel to produce d v dimension output values. Finally, all of them are spliced and mapped again through a fully connected layer to obtain the final output, which can be calculated as:

[0024] MHID_Attention(S,T)=Concat(head1,…,head h )Wo

[0025]

[0026] where, and W o represent the parameter matrices corresponding to the fully connected layers with dimensions d m ×d q , d m ×d k , d m ×d v and hd v ×d m , respectively.

[0027] Based on the multi-head inter-domain attention matrix, we design a stackable encoding module, i.e., an inter-domain attention-based encoder, to improve the expression ability of the network. The encoder mainly consists of a multi-head inter-domain attention layer and a fully connected feedforward network. And residual connections are used around the two layers, and then the BatchNorm operation is performed on the features obtained by each layer to obtain the output of each layer. The feedforward network consists of two fully connected layers, the first layer uses ReLU as the activation function, and the second layer does not use the activation function, and the input S 1 and the output S 2 are calculated as follows:

[0028] S 1 = BatchNorm(S + MHID_Attention(S, T))

[0029] S 2 = BatchNorm(S 1 + max(0, S 1 W1 + b1)W2 + b2)

[0030] where the dimensions of S 1 and S 2 are n s ×d m .

[0031] Step 3, calculate the classification loss of the health state recognizer and the domain discriminator respectively, and combine to get the final optimization target.

[0032] The necessary notes of step 3: the health state recognizer is used to distinguish the health state label of the input data, and the source domain data is labeled. The cross-entropy loss function is used to calculate the classification loss of the source data, and the calculation expression is as follows:

[0033]

[0034] where, is the dimension one-hot vector. Moreover, due to the unknown nature of target labels, we further refine the labeled classifier and encourage low-density separation between target domain classes by utilizing unlabeled target examples via entropy minimization principle. The semi-supervised classification loss of target data can be expressed as:

[0035]

[0036] where, is the predicted probability vector of target samples.

[0037] Domain discriminator is used to identify the domain that the data belongs to, and the training target adopts binary cross-entropy loss, which can be calculated as:

[0038]

[0039] where, is the weighted source sample feature output by the inter-domain attention encoding module.

[0040] In the process of adversarial learning, the feature extractor needs to maximize the domain discrimination loss L d , while the domain discriminator needs to minimize L d , so the final optimization target is:

[0041] L(θ f ,θ y ,θ a ,θ d )=L y (θ f ,θ y )+βL h (θ f ,θ y )-λL d (θ f ,θ a ,θ d )

[0042] where, θ f is the feature extractor parameter, θ y is the health state recognizer parameter, θ a is the inter-domain attention encoding module parameter, θ d is the domain discriminator parameter, β and λ are penalty coefficients.

[0043] Step 4, input the labeled source domain data and unlabeled target domain data into the constructed network, and perform back propagation through the stochastic gradient descent method for network training until convergence, and obtain the final fault diagnosis model.

[0044] Necessary notes for step 4: the final training target is to find the optimal parameters and can be expressed as:

[0045]

[0046] We denote the parameters to be learned as Θ = {θ f , θ y , θ a , θ d}, and the update process of Θ can be expressed as:

[0047]

[0048] where α is the learning rate. In the training phase, in order to realize end-to-end training, we introduce a gradient reversal layer (GRL) in front of the domain discriminator. Specifically, the gradient reversal layer is defined as a pseudo function R λ , and assuming that its input is X, its forward calculation and backward gradient calculation can be expressed as:

[0049] R λ (X) = X

[0050]

[0051] where I is the identity matrix. In the testing process, the test model only contains the learned feature extractor and the health state identifier.

[0052] Beneficial effects:

[0053] IDAAN effectively reduces the negative transfer phenomenon caused by out-of-distribution source samples in some unsupervised cross-domain fault diagnosis problems by introducing an inter-domain attention encoding module in the domain adaptation process to assign higher weights to shared source samples and lower weights to out-of-distribution source samples. At the same time, the strategy of adversarial learning is used to align the shared health space of the source domain and the target domain, promoting the positive transfer of knowledge. In addition, according to the minimum entropy criterion, the unlabeled target domain data is fully utilized, and a semi-supervised classification loss is introduced into the final optimization target of the health state identifier, effectively improving the recognition ability of the model in the shared health space. When applied to the field of aero-engine fault diagnosis, it can improve the application ability of historical data and make the data-driven fault diagnosis method better adapt to real working scenarios. A large number of partial unsupervised cross-domain fault diagnosis experiments on simulated fault data of a certain type of turbofan engine show that, compared with traditional data-driven methods and many classic domain adaptation methods, IDAAN can often achieve higher diagnostic accuracy on target domain data. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 is the overall framework diagram of the IDAAN model;

[0055] Figure 2 is a structural diagram of a feature extractor;

[0056] Figure 3 is a flow chart of scaled dot product inter-domain attention mechanism computation;

[0057] Figure 4 is a flow chart of multi-head inter-domain attention mechanism computation;

[0058] Figure 5 is a structural diagram of an inter-domain attention mechanism-based encoder;

[0059] Figure 6 is a schematic diagram of a turbofan engine structure;

[0060] Figure 7 is a schematic diagram of MMD distance between three datasets;

[0061] Figure 8 is a schematic diagram of test accuracy variation of different methods with variation of the number of healthy patterns contained in the target domain;

[0062] Figure 9 is a schematic diagram of diagnostic accuracy of IDAAN under different training batch sizes;

[0063] Figure 10 is a schematic diagram of diagnostic accuracy of IDAAN under different numbers of inter-domain attention mechanisms. DETAILED DESCRIPTION

[0064] IMPLEMENTATION CASE

[0065] This case takes a certain type of turbofan engine component-level simulation model as the research object, and the structure is as shown in Figure 6 In the experiment, we select: high-pressure shaft speed N h , low-pressure shaft speed N l , compressor inlet temperature T 22 , compressor inlet pressure P 22 , compressor outlet temperature T3, compressor outlet pressure P3, high-pressure turbine outlet temperature T 42 , high-pressure turbine outlet pressure P 42 , low-pressure turbine outlet temperature T 46 , and low-pressure turbine outlet pressure P 46, 10 state parameters as monitoring parameters. The engine gas path component faults mainly include fan (Fan) fault, high pressure compressor (HPC) fault, high pressure turbine (HPT) fault, low pressure turbine (LPT) fault, i.e. including normal state, finally need to identify 5 kinds of health status. Without loss of generality, we usually simulate component faults by adjusting the efficiency coefficient SE and flow coefficient SW of each component of the simulation model. In this case, we use the health parameter vector HP = [SE1, SW1, SE2, SW2, SE3, SW3, SE4, SW4] to represent the performance degradation of the engine, where "1" corresponds to the fan, "2" corresponds to the compressor, "3" corresponds to the high pressure turbine, and "4" corresponds to the low pressure turbine.

[0066] In this case, we obtain 3 groups of data with different distributions by changing the initial degradation degree (health parameter) of the engine and the working condition (flight height, flight Mach number and fuel supply law). Specifically, the working points, initial degradation degrees and sample numbers of the three groups of data are shown in Table 1. Among them, we collected 60 samples under each health condition at each working point, i.e. each data set includes 360x5 samples, and a single sample is a time series with a length of 60 and a dimension of 10. We assume that the sensor data noise conforms to the normal distribution, and in all data sets, we add Gaussian noise with a mean of 0 and a variance of 1% to make the obtained data set more consistent with the real situation.

[0067] Table 1 Working point table corresponding to each data set

[0068]

[0069] In order to illustrate the difference in data distribution caused by the difference in initial degradation degree of the engine and the change in working condition, we divide the data set without adding Gaussian noise into training set and test set in the ratio of 7:3. First, we select the maximum mean difference (MMD) measure to calculate the MMD distance between the three data sets, and the final result is as follows Figure 7As shown in Table 2, it can be seen that the MMD distance between different data sets is much larger than the difference between the training set and the test set of the same data set. Moreover, we use the CNN method without domain adaptation to perform cross-domain fault diagnosis between different data sets, that is, the training set of one data set is used for training, and the test set of another data set is used for testing. It should be noted that the feature extractor of this method is the same as IDAAN. The final diagnosis results are shown in Table 2. It can be seen that the diagnosis accuracy of the model on other data sets has decreased significantly compared with the test set of its own. Therefore, the difference in historical data and current monitoring data caused by different working conditions and initial degradation performance will greatly reduce the diagnosis accuracy of the model. It is necessary to effectively use the historical data of the aircraft engine to train a fault diagnosis model with sufficient accuracy.

[0070] Table 2 Test accuracy of CNN method on different data sets

[0071]

[0072] In order to evaluate the effectiveness of the IDAAN of the present application, we selected six methods, CNN, Residual transfer network (RTN), Partial adversarial domain adaptation (PADA), Example transfer network (ETN), DA-GAN, and BatchFormer, as comparative algorithms. Among them, the CNN method is not a domain adaptation method; RTN and DANN are standard domain adaptation methods, that is, these two methods are designed for the case where the health space of the source domain and the target domain is completely the same; PADA, ETN and DA-GAN are partial domain adaptation methods, which all alleviate the negative transfer caused by outlier source samples through weighting the final optimization target; BatchFormer is designed to solve the long-tail distribution problem, and in this case, it refers to replacing the inter-domain attention encoding module with a Transformer Encoder to verify the effectiveness of the inter-domain attention mechanism in the present application in the problem of partial unsupervised cross-domain fault diagnosis of aircraft engines.

[0073] The network structure of the feature extractor used in all the above methods is as shown in Figure 2 The network is trained by the Adam method. During the training process, the batch size of all methods is set to 512, the learning rate is set to 0.001, and the weight decay coefficient is set to 0.0001. For the IDAAN of the present application, the overall framework is as shown in Figure 1The number of encoders based on the inter-domain attention mechanism in the inter-domain attention encoding module is fixed as 2, and the number of heads of the inter-domain attention mechanism is fixed as 8 in all experiments. In addition, during the training process, β is fixed as 0.5, and λ is adjusted according to the following strategy:

[0074]

[0075] wherein q = p / 2000 and p is the corresponding training step number, the value range is 0 to 2000, that is, the epoch of training is set to 1500. In order to avoid the randomness of experimental results, each group of experiments is repeated 10 times to take the average value.

[0076] Firstly, by using the collected three groups of data, 12 groups of cross-domain fault diagnosis experiments are designed, and each group of specific conditions is shown in Table 3. In Table 3, '1' corresponds to the normal state, '2' corresponds to the fan fault, '3' corresponds to the compressor fault, '4' corresponds to the low-pressure turbine fault, and '5' corresponds to the high-pressure turbine fault. Among them, in addition to task T f1 , in all other tasks, the target domain health space is a subset of the source domain health space. The fault diagnosis results of different tasks are shown in Table 4. In general, the performance of the proposed method in most tasks is obviously better than that of other comparison methods, which shows the effectiveness of the IDAAN of the present application. In addition, by comparing the diagnosis results of the CNN in Table 2 with Table 4, it can be found that after adding noise to the data set, the diagnosis effect of the model only decreases slightly, which shows that the deep network has good robustness to noise.

[0077] Table 3 Partial Cross-Domain Fault Diagnosis Task Setting Information Table

[0078]

[0079] Table 4 Partial Cross-Domain Fault Diagnosis Experiment Diagnosis Average Precision and Standard Deviation Table

[0080]

[0081] It can be found from Table 4 that task T f1 is a standard unsupervised cross-domain fault diagnosis task. In this task, the diagnosis accuracy of the proposed method is higher than that of DANN and RTN designed for this scene. In the real-world fault diagnosis scene, the more common situation is that the target domain only contains normal state data. In this scene, T f9 , T f10 , T f11 and T f12There are four groups of experiments. The results show that the IDAAN is superior to other comparison methods in the four tasks. And in this case, DANN and RTN both show significant negative transfer compared to the baseline model CNN without domain adaptation. This is because they try to align all health states contained in the source domain with the target domain, but there are health states in the source domain that do not exist in the target domain, i.e. out-of-distribution health states. Matching source out-of-distribution data and target domain data will make the health state identifier more likely to classify the target data as out-of-distribution health states, and these out-of-distribution data will cause negative transfer.

[0082] In order to illustrate the effectiveness of the inter-domain attention mechanism in the present application, which is a design specifically used to explore the relationship between source samples and target domains, the diagnostic results of BatchFormer are compared with the diagnostic results of IDAAN. It can be seen that IDAAN is superior to BatchFormer in all extreme local domain adaptation scenarios. The difference between the two is only in the selection of attention layers. BatchFormer explores the relationship between all samples through cross-attention, so the experimental results can illustrate the effectiveness of the inter-domain attention mechanism design in the present application.

[0083] In addition, in the present case, we can find that in tasks T f1 , T f3 , T f6 , T f7 , T f9 , the health patterns contained in the target domain are decreasing in turn. Drawing the experimental results into a line graph gives Figure 8 . It can be seen from Figure 8 that the decrease in the number of health states contained in the target domain usually leads to a decrease in the test accuracy of the model. And this trend is more obvious in DANN and RTN. This is consistent with the current understanding that when there are fewer shared classes, the partial unsupervised cross-domain fault diagnosis problem becomes more difficult. However, in the extreme partial unsupervised cross-domain fault diagnosis task, the accuracy of the proposed method is greater than 90%, which still maintains high performance. Therefore, the present application has unique advantages in the partial unsupervised cross-domain fault diagnosis problem.

[0084] Further, in the present case, we explore the influence of the setting of the batch size and the number of inter-domain attention heads during training on the performance of the present application IDANN. The batch size determines the size of the inter-domain attention matrix, and using multi-head inter-domain attention can improve the representation ability of the model. We set the batch size to 32 and the number of inter-domain attention heads to 8 in tasks T f3 , T f6 , T f7 , T f9 , T f10 , T f11 and T f12The size of the two parameters is adjusted respectively, and the experimental results are shown in Figure 9 and Figure 10 . Among the selected seven tasks, the number of health states contained in the target domain decreases in turn. Figure 9 and Figure 10 indicate that when the target domain contains fewer health state labels, the model is more sensitive to changes in batch size and the number of attention heads. Specifically, by observing Figure 9 , it can be found that, except for tasks T f3 and T f12 , the model with a batch size of 512 has a higher diagnostic accuracy than other batch sizes. At the same time, by observing Figure 10 , it can be found that in all seven tasks, the diagnostic accuracy of the model with four inter-domain attention heads is better than that of other models. This indicates that multi-head inter-domain attention improves the learning ability of the model without increasing the number of model parameters. However, it is worth noting that the more heads of inter-domain attention is not necessarily better. In this case, the model with 8 heads has even worse diagnostic accuracy than the model with 2 heads on multiple tasks (T f3 , T f6 , T f7 , T f9 , T f10 , T f12 ). Therefore, the number of attention heads and the size of the batch size should be adjusted according to the specific task to obtain a better diagnostic model.

Claims

1. An aero-engine partial unsupervised cross-domain fault diagnosis method based on an inter-domain attention mechanism, characterized in that, The method comprises the following steps: Step 1: normalizing the collected source domain and target domain data as training set data; the source domain and target domain data are monitoring sample segments of processed aero-engine state parameters; Step 2: constructing a domain adversarial network containing an inter-domain attention mechanism; The domain adversarial network comprises a feature extractor G f , an inter-domain attention encoding module G a , a health state identifier G y , and a domain discriminator G d ; the feature extractor G f extracts source sample data features f s and target sample data features f t ; f s and f t are input into the health state identifier G y to classify the health states of the source domain and the target domain samples; meanwhile, f s and f t are input into the inter-domain attention encoding module G a , f s is weighted through an inter-domain attention mechanism, and weighted source sample features f sa are obtained; f sa and f t are input into the domain discriminator G d to identify the domain to which the data belongs. Step 3: respectively calculating the classification loss of the health state identifier and the domain discriminator, and combining to obtain a final optimization target; Step 4: inputting the training set data into the constructed domain adversarial network, and performing backward propagation through a stochastic gradient descent method to perform network training until convergence.

2. The method of claim 1, wherein, In step 1, the source domain and the target domain data have the same feature space, i.e. But the target domain health space is a subset of the source domain health space, i.e. Let denote the shared health space of the source domain and the target domain, and let denote the outlier health space of the source domain; and the source data probability distribution p is different from the target data probability distribution q, i.e. p≠q; Also, in sharing the health state space the probability distribution of source domain data is different from the probability distribution q of target domain data, i.e.

3. The method of claim 1, wherein, In step 2, the feature extractor G f For the improved 1-dimensional AlexNet network, used to extract features from the input monitoring time series segment data.

4. The method of claim 1, wherein, In step 2, the health condition recognizer G y The last layer adopts a softmax function as an activation function for multi-classification; the domain discriminator G d The last layer adopts a sigmoid as an activation function for domain binary classification.

5. The method of claim 2, wherein, The inter-domain attention encoding module in step 2 is composed of two encoders based on an inter-domain attention mechanism; the inter-domain attention mechanism weights the source samples, and assumes that the difference between the source data in the outlier health space and the target domain data is greater than the difference between the source data in the shared health space and the target data.

6. The method of claim 1, wherein, In step 2, domain alignment is achieved through adversarial learning between the feature extractor and the domain discriminator, promoting forward transfer, that is, the feature extractor maximizes the domain classification loss, and the domain discriminator minimizes the domain classification loss.

7. The method of claim 1, wherein, Health condition identifier G in step 3 y With cross-entropy as its classification loss function, it learns under supervision for source domain data, and the calculation expression is: where n s denotes the number of source domain samples, denotes the true label of the source domain sample, denotes the input network's source domain sample data; for target domain data, it is unlabeled; the entropy minimization principle is used to further refine the labeled classifier and encourage low-density separation between target domain classes using unlabeled target examples; the semi-supervised classification loss of the target data is represented as: where n t denotes the number of target domain samples, is the prediction probability vector of the target sample, denotes the target domain sample data input into the network; the domain discriminator G d Adopt binary cross-entropy as its classification loss function, 0 represents the target domain, 1 represents the source domain, and its calculation expression is: wherein, is a feature of the source sample output by the inter-domain attention-based encoder module.

8. The method of claim 7, wherein, In step 3, the final optimization loss function is: L(θ f ,θ y ,θ a ,θ d ) = L y (θ f ,θ y ) + βL h (θ f ,θ y ) - λL d (θ f ,θ a ,θ d ) where θ f is the feature extractor parameter, θ y is the health state identifier parameter, θ a is the inter-domain attention encoding module parameter, θ d is the domain discriminator parameter, β and λ are the trade-off hyperparameters; the final training objective is to find the optimal hyperparameters and is represented as:

9. The method of claim 1, wherein, During the training process of step 4, the features input into the domain discriminator G d first pass through the gradient reversal layer GRL to ensure the implementation of adversarial training in end-to-end learning; specifically, the gradient reversal layer is defined as a pseudo function R λ , and assuming that its input is X, its forward calculation and back propagation gradient calculation are respectively represented as: R λ (X) = X Where I is an identity matrix.

Citation Information

Patent Citations

  • Method for quickly mixing high-order attention domain adversarial network based on transfer learning

    CN112446423A

  • Domain adaptive method based on attention mechanism

    CN113112005A