Fault diagnosis method for UAV sensors based on weighted transfer domain adaptation learning

Abstract

The present invention relates to the technical field of unmanned aerial vehicle fault diagnosis, and in particular to a method for unmanned aerial vehicle sensor fault diagnosis based on weighted transfer domain adaptation learning. It includes: constructing a multi-scale neural network model, including three multi-scale neural network modules of a feature extractor, a domain discriminator and a classifier; the feature extractor extracts high-dimensional feature information of the data; the domain discriminator performs adversarial training with the feature extractor during the model training process, so that the domain discriminator cannot distinguish whether the data comes from the source domain or the target domain; the classifier identifies and classifies the sample features output by the feature extractor; obtains a source domain sample data set and a target domain sample data set; trains the constructed multi-scale neural network model; and performs fault identification and diagnosis on the target domain data through the trained multi-scale neural network model. The advantage is that in the absence of target domain data set labels, a better learning model is established for unmanned aerial vehicle fault diagnosis, and fault diagnosis of the target domain data set is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle fault diagnosis, and in particular to a method for unmanned aerial vehicle sensor fault diagnosis based on weighted transfer domain adaptation learning. Background Art

[0002] With the rapid development of semiconductor electronics, computers, the Internet, communications and other technologies, the level of intelligence of modern new equipment is getting higher and higher, and the density of electronic unit modules of equipment electronic and electrical subsystems is growing exponentially. As one of the representative products of the development of modern new equipment, drones have been widely used in various industries due to their good mobility, high execution efficiency, low cost and convenience. In recent years, the intelligence level of drones has been continuously improved, and there are many types of electronic devices on airborne platforms, showing the characteristics of high transmission power, high density and high sensitivity. However, due to the constraints of its own volume and weight, the equipment is densely installed and the space isolation is limited. The equipment is greatly affected by internal and external electromagnetic signals, which is prone to electromagnetic compatibility problems. When encountering extreme weather and advanced high-power electromagnetic interference weapons, strong transient electromagnetic pulses can interfere with the receiving system of airborne equipment at a long distance through coupling and destroy it at a close distance. This causes various faults in the communication, command and control and other electronic equipment data transmission processes of drones, seriously affecting the technical and tactical indicators and work efficiency of combat equipment.

[0003] In the process of performing tasks, drones are not operated by pilots on site, and emergency measures cannot be taken in time when failures occur. As a result, there is a large gap between drones and manned aircraft in terms of reliability and safety, and related anomaly detection and fault analysis are very difficult. According to the statistics of catastrophic accident rates of drones in the "2013-2038 Drone Roadmap", the catastrophic accident rate of drone flights is 1 to 2 orders of magnitude higher than that of manned aircraft. According to the current reports of accidents involving related drone products at home and abroad, the failures of drones are usually complex and have no obvious rules, the fault characteristics are multi-layered and interrelated, and the time of failure is uncertain. It is difficult to make correct analysis and judgment on abnormalities or faults in drones based on traditional empirical methods, which makes the safety and reliability of drone equipment face increasingly complex challenges. The entire life cycle of drones needs to rely on advanced predictive maintenance and support technologies to improve the dynamic safety of the system. "Remotely Piloted Aircraft Guidance: Vision and Enabling Concepts 2013-2038" makes a detailed plan for the research on in-flight diagnosis and prediction technologies to accelerate maintenance turnover and improve the comprehensive support capabilities of drones. The "Integrated Drone Roadmap for Fiscal Years 2017-2042" emphasizes that in order to achieve flight safety and reliability, unmanned systems require not only offline testing, but also real-time monitoring, prediction, and timely and feasible fault recovery.

[0004] UAV flight data can reflect the key status information of specific equipment and is the main means and important basis for evaluating whether the UAV is in an abnormal state. Before a UAV fails or is in the early stage of a failure, the flight data is abnormal. Timely detection of abnormalities and accurate fault diagnosis can avoid catastrophic losses of UAVs. The flight data of UAV systems are usually obtained by installing various types of sensors on them. During the operation of the UAV, these sensors are responsible for monitoring the data of the UAV itself and the surrounding environment to ensure the safety and stability of the UAV when performing tasks. Sensors usually have the characteristics of high sensitivity and wide spectrum, making them sensitive sources and highly sensitive to electromagnetic interference. In some harsh electromagnetic environments, the data collected by sensors are prone to large deviations. Sending wrong instructions to the UAV causes the UAV to make wrong decisions, which in turn leads to hidden dangers in flight safety and even causes failures and crashes. Therefore, by analyzing the UAV sensor data, the abnormal working state of the UAV can be discovered early, and timely and appropriate processing can be carried out to avoid or reduce the occurrence of component failures or loss of system components, extend the service life of the UAV, reduce maintenance costs, improve the reliability and safety of the UAV system, and provide predictive maintenance guidance for the health management of the UAV throughout its life cycle.

[0005] UAV flight data comes from various subsystems inside the aircraft, and has the characteristics of high dimension, few abnormal label data, many data parameters, dynamic changes, and coexistence of continuous and discrete data. There are many fault diagnosis methods based on UAV flight data, mainly including analytical model-based, knowledge-based, and data-driven methods. Among them, the analytical model-based method describes the characteristics of the aircraft by constructing an accurate physical model, and detects faults by comparing the difference between the actual system output and the output of the constructed model. However, the UAV system is complex and the system modeling is difficult. Fault diagnosis is very sensitive to modeling errors, parameter noise and external interference factors, which makes the model not effective in actual application of the system. The knowledge-based method uses qualitative description to obtain the monitoring model through causal models and expert prior knowledge bases. Common types include expert systems, fault tree reasoning, fuzzy reasoning, etc. The establishment of a knowledge model requires a comprehensive understanding and accurate expression of the system, but the faults of most UAV systems cannot be fully obtained, and it is difficult to establish a large and comprehensive expert knowledge base model, and the ability to distinguish UAV fault diagnosis is limited.

[0006] With the rapid development and application of cutting-edge technologies such as computers and artificial intelligence, data-driven methods represented by deep learning have become a hot topic for experts at home and abroad. Deep learning methods can train fault learning models from a large amount of UAV flight data information, accurately identify fault anomalies, and effectively improve the efficiency of UAV fault diagnosis and the level of predictive maintenance. Data-driven deep learning methods have high fault identification capabilities and generally need to meet two conditions: the same data sample distribution and complete data annotation. However, due to the complex composition of UAV systems and special service missions, the distribution of sample data obtained varies greatly, and the type of fault data and corresponding labels under different working conditions are unknown; moreover, the equipment is usually in a healthy state during actual operation, and the probability of failure is low, resulting in scarce fault sample data. As a result, common deep learning models have insufficient sample data in UAV fault diagnosis and identification, limited fault feature extraction capabilities, and poor model generalization performance, resulting in high false positive and missed detection rates of the model, making it difficult to train an intelligent diagnosis model with high accuracy in identifying the health status of UAV equipment. This restricts the application of deep learning methods in UAV fault diagnosis.

[0007] At present, the common data-driven deep learning method for detecting UAV fault diagnosis generally needs to meet two conditions: the same data sample distribution and complete data annotation. However, due to the complex composition of UAV systems and special service missions, the distribution of sample data obtained varies greatly, and the type of fault data and corresponding labels under different working conditions are unknown; moreover, the equipment is usually in a healthy state during actual operation, and the probability of failure is low, resulting in scarce fault sample data. As a result, common deep learning models have insufficient sample data in UAV fault diagnosis and identification, limited fault feature extraction capabilities, and poor model generalization performance, resulting in high false detection and missed detection rates of the model, making it difficult to train an intelligent diagnosis model with high accuracy in identifying the health status of UAV equipment. In addition, most of the existing fault diagnosis methods are designed for the fault types of specific components of UAVs, and their model robustness is difficult to meet the expected requirements of the flight control system.

[0008] Transfer learning is a branch of machine learning. This method can transfer existing knowledge to solve problems in different but similar or related fields. When the UAV target mission lacks high-quality training data, transfer learning can reuse existing knowledge and experience, save the cost of re-collecting a large number of labeled samples, improve the efficiency of model training, and identify anomalies in the target data set. The establishment of a transfer learning model requires that the distribution difference between the source domain and the target domain is similar. However, the distribution of the source domain and the target domain is usually different. If the distribution difference is large, it will affect the effect of transfer learning. Domain adaptation methods can reduce the difference in the distribution of the two domains. The generative adversarial network method can confuse the source domain data set and the target domain data set through the adversarial game between the generative network and the discriminant network, reduce the distribution difference between the two domains, and train cross-domain shared features, so that the model can finally have the ability to diagnose faults in the target domain data set. Summary of the invention

[0009] In order to solve the above problems, the present invention provides a UAV sensor fault diagnosis method based on weighted transfer domain adaptation learning.

[0010] The present invention aims to provide a method for diagnosing unmanned aerial vehicle sensor faults based on weighted transfer domain adaptation learning, which specifically includes the following steps:

[0011] S1. Construct a multi-scale neural network model, including three multi-scale neural network modules: feature extractor, domain discriminator and classifier;

[0012] Among them, the feature extractor receives input data as an entry, maps the input data to a specific space, and extracts high-dimensional feature information of the data; the domain discriminator is used to confuse the source domain sample data set and the target domain sample data set, and conducts adversarial training with the feature extractor during the model training process, so that the domain discriminator cannot distinguish whether the data comes from the source domain or the target domain; the classifier identifies and classifies the sample features output by the feature extractor, and finally determines the category corresponding to the data;

[0013] S2. Obtain source domain sample datasets and target domain sample datasets: The source domain dataset is selected from the public drone failure dataset, the air laboratory failure and anomaly dataset; sensor data from unlabeled drone actual flight records are collected as the target domain sample dataset;

[0014] S3. Training the multi-scale neural network model constructed in step S1;

[0015] S4. Perform fault identification and diagnosis on the target domain data through the trained multi-scale neural network model.

[0016] Preferably, the source domain sample data set and the target domain sample data set both include samples of four fault types, and the four fault types include constant deviation, drift, mutation and noise deviation.

[0017] Preferably, the order of data samples under each fault type of the source domain sample data set is shuffled, and randomly divided into a source domain training set and a source domain verification set in a ratio of 8:2;

[0018] Several samples are selected from the target domain sample data set and divided into a target domain training set and a target domain validation set in a ratio of 8:2; then the same number of samples are extracted from the target domain sample data set as the test set.

[0019] Preferably, the feature extractor includes 4 convolution blocks, a compression layer and a fully connected layer; each convolution block consists of a convolution layer, a pooling layer and an activation layer; the function of the compression layer is to convert multi-dimensional data into one-dimensional data output; the domain discriminator includes three fully connected layers and a Softmax function; the classifier includes two fully connected layers and a Softmax function.

[0020] Preferably, step S3 specifically includes the following sub-steps:

[0021] S301. Initialize network parameters: Before training begins, perform parameter initialization settings, and use source domain samples to pre-train the multi-scale neural network model for several iterations;

[0022] S302. Calculate the loss function of the domain discriminator, the loss function of the classifier, and the loss function of the multi-core maximum mean difference distance between the source domain and the target domain data of the feature extractor during the training process respectively; and finally calculate the total target loss function;

[0023] S303. Bring the training set into the input layer of the model and start training the model. When the number of network iterations reaches the set number of iterations, bring the validation set into the model to validate the model and fine-tune the parameters.

[0024] S304. Bring in the test set to test the recognition effect of the model; when the model parameters are optimized to the target value, the model training is completed.

[0025] Preferably, the loss function of the domain discriminator is a binary cross entropy loss function, as shown in the following formula:

[0026] ;

[0027] in, represents the i-th sample in the source domain, is the total number of source domain samples; represents the jth sample in the target domain, is the total number of samples in the target domain; represents the network parameters of the feature extractor; represents the network parameters of the domain discriminator; F represents the feature extractor, and D represents the domain discriminator;

[0028] The loss function of the classifier is as follows:

[0029] ;

[0030] in, represents the network parameters of the feature extractor, represents the network parameters of the classifier, is the total number of source domain samples, is the number of category labels; express The probability of belonging to category k; F represents the feature extractor and C represents the classifier;

[0031] The loss function of the multi-core maximum mean difference distance between the source domain and the target domain data during the feature extractor training process is expressed as follows:

[0032] ;

[0033] in, represents the i-th sample in the source domain, is the total number of source domain samples; represents the jth sample in the target domain, is the total number of samples in the target domain; represents the network parameters of the feature extractor, represents a Hilbert space.

[0034] Preferably, the calculation process of the total objective loss function is as follows:

[0035] The weight value of the source domain sample is calculated by the discrimination difficulty of the domain discriminator; let Represents the weight of the i-th sample in the source domain, then there is the following formula:

[0036] ;

[0037] The weight after scaling the weight using the minimum-maximum normalization method for:

[0038] ;

[0039] In the formula, ;

[0040] The source domain sample allocation weights are introduced into the loss function of the domain discriminator. The loss function As shown below:

[0041] ;

[0042] The total target loss function is obtained as follows:

[0043] ;

[0044] In the formula, is the balance parameter between classification error and domain error, and α is a hyperparameter.

[0045] Preferably, the learning rates of the loss functions of the feature extractor, the domain discriminator and the classifier are set to 0.001, 0.0001 and 0.0001 respectively.

[0046] Preferably, the number of pre-training iterations in step S301 is 50 times; and the number of network iterations in step S303 is set to 200 times.

[0047] Preferably, the model parameter optimization target in step S304 is as follows:

[0048]

[0049] ;

[0050] In the formula, Respectively represent the optimized network parameters corresponding to the feature extractor, classifier and domain discriminator;

[0051] The parameter optimization uses the stochastic gradient descent method to update the parameters in the opposite direction of the gradient vector. , , , will be updated as follows:

[0052]

[0053]

[0054] ;

[0055] in, is the learning rate, is the balance parameter between classification error and domain error, and α is a hyperparameter; μ , α are set to 1.2λ respectively p , 0.75λ p ;

[0056] The momentum of the stochastic gradient descent method is set to 0.85; the gradient reversal layer method is used to automatically reverse the gradient during the back propagation of the domain discriminator and remain unchanged during the forward propagation.

[0057] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0058] 1. The method of the present invention is based on an unsupervised transfer learning method. The training process does not rely on the label data of the target domain dataset. The model can use the knowledge and experience learned in the source domain to perform cross-domain online diagnosis of fault data in the target domain dataset. In this way, the previously trained model can be used as a basis, avoiding global training of new tasks and greatly improving the training efficiency of the model.

[0059] 2. The feature extractor can not only extract high-dimensional features of the input data, but also generate high-quality pseudo target domain data through adversarial training with the domain discriminator to improve the imbalance between the two domain samples. The domain discriminator can accumulate cross-domain common features during the training process of distinguishing which domain different samples belong to, and at the same time apply multi-level domain adaptation methods to reduce the distribution difference between the source domain and the target domain data.

[0060] 3. The model uses the loss function to quantitatively measure the difference between the predicted performance and the actual performance, and improves the performance of the model by optimizing the parameters of the loss function. In order to avoid the negative transfer of the model due to the large difference between the source domain samples and the target domain samples, the source domain samples are given weights by calculating the contribution of different samples to the model, thereby enhancing the adaptability of model training.

[0061] In summary, the present invention realizes online anomaly recognition and fault diagnosis of unlabeled sensor data in the target domain by constructing a transfer domain adaptation learning method based on unsupervised learning and learning the knowledge of labeled sensor data in the source domain. Combining the advantages of transfer learning and domain adaptation methods, the limitations of the above-mentioned traditional methods in UAV fault diagnosis are overcome. Taking the typical faults of UAV sensor data as the research object, a weighted transfer domain adaptation network model is invented. In the absence of target domain data set labels, a better learning model is established for UAV fault diagnosis, and fault diagnosis of the target domain data set is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a structural framework diagram of a multi-scale neural network model provided according to an embodiment of the present invention.

[0063] Figure 2 This is a migration strategy training process provided according to an embodiment of the present invention; in the figure, (a) represents the original input data, (b) represents the migration training, and (c) represents the domain adaptation result.

[0064] Figure 3It is a schematic diagram of the multi-scale neural network model training process provided according to an embodiment of the present invention.

[0065] Figure 4 is an accuracy curve of the multi-scale neural network model provided according to an embodiment of the present invention.

[0066] Figure 5 is a loss function curve provided according to an embodiment of the present invention.

[0067] Figure 6 It is a confusion matrix of the test results under different methods provided by the embodiments of the present invention; in the figure, (a) represents a CNN neural network, (b) represents a transfer component analysis (TCA), (c) represents a deep domain adaptation method (DAN), (d) represents a dynamic adversarial adaptive network (DAAN), and (e) represents the method of the present invention.

[0068] Figure 7 It is a t-SNE graph of the test results under different methods provided according to the embodiments of the present invention; in the figure, (a) represents a CNN neural network, (b) represents a transfer component analysis (TCA), (c) represents a deep domain adaptation method (DAN), (d) represents a dynamic adversarial adaptive network (DAAN), and (e) represents the method of the present invention. DETAILED DESCRIPTION

[0069] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same modules are represented by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, the detailed description thereof will not be repeated.

[0070] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0071] The present invention provides a method for diagnosing faults of unmanned aerial vehicle sensors based on weighted transfer domain adaptation learning, which specifically comprises the following steps:

[0072] S1. Construct a multi-scale neural network model: including three multi-scale neural network modules, namely feature extractor F, domain discriminator D and classifier C;

[0073] The feature extractor F receives input data as an entry. Part of its function is to map the input data to a specific space and extract the high-dimensional feature information of the data. The function of the domain discriminator D is to confuse the source domain sample data set and the target domain sample data set. During the model training process, it conducts adversarial training with the feature extractor F, and ultimately makes the domain discriminator D unable to distinguish whether the data comes from the source domain or the target domain. The classifier C identifies and classifies the sample features output by the feature extractor F, and ultimately determines the category corresponding to the data.

[0074] The feature extractor F consists of 4 convolution blocks, a compression layer and a fully connected layer; each convolution block consists of a convolution layer, a pooling layer and an activation layer; the compression layer is used to convert multi-dimensional data into one-dimensional data output;

[0075] The domain discriminator D consists of three fully connected layers and a Softmax function;

[0076] Classifier C includes two fully connected layers and a Softmax function; the network structure parameter settings are shown in Table 1.

[0077] Table 1 Network structure parameter table

[0078]

[0079] Through the interaction of three modules, the network needs to achieve two tasks: one part realizes the adversarial domain adaptation training and learning function, and maximizes the distribution difference between domains through adversarial training between feature extractors and domain discriminators. The other part is transfer learning, which uses the knowledge information obtained from the source domain to accurately identify and classify the unlabeled fault data in the target domain and minimize the classification error of the classifier. The core of the transfer domain adaptation learning method is the interactive adversarial training between the feature extractor and the discriminator, so that the feature extractor can extract cross-domain common features that are difficult for the discriminator to distinguish, that is, domain invariant features. Then, the trained classifier is directly applied to the target domain dataset to identify and classify the data. The network framework structure is as follows: Figure 1 shown.

[0080] Brief description of the principle: When drones are in different types of service missions, environmental factors are complex and changeable, and the flight status also needs to be constantly adjusted. The data distribution of the sample data sets obtained by the onboard sensors is very different, and the types of fault data and corresponding labels under different working conditions are unknown; the probability of equipment failure is small, resulting in scarce fault sample data. The labeled supervised learning algorithm is used to train these unlabeled target domain data. The model generalization error is large and cannot effectively identify abnormal sample data in the target domain. The present invention constructs a weighted transfer domain adaptation network model, which mainly includes three parts.

[0081] S2. Obtain source domain sample datasets and target domain sample datasets: Obtain the AirLab Failure and Anomaly (ALFA) dataset of Carnegie Mellon University as the source domain sample dataset; collect actual flight data of the small fixed-wing UAV to be tested as the target domain sample dataset;

[0082] Among them, the source domain sample data set records the actual flight data of a small fixed-wing UAV, the model of which is CarbonZ T-28; the source domain sample data set contains fault diagnosis knowledge related to UAV sensors, and selects 4 sensor sensitive parameters from the flight state parameters of the UAV IMU module, including the total pressure and static pressure of airspeed, X-axis linear acceleration, Y-axis magnetic field, and Z-axis angular velocity. Each sensor type data includes four fault type labels: constant deviation (C), drift (D), mutation (I), and noise deviation (B); each sensor type signal is cut into data samples with a length of 1024 without overlap, and 100 samples are established for each of the four fault type samples; the data set is established as shown in Table 2 below;

[0083] Table 2 Sample dataset

[0084]

[0085] The target domain sample data set uses real UAV flight data. The UAV is a fixed-wing UAV model CG-200. The flight data records the real UAV flight data at different altitudes and wind speeds on different dates in a certain place in China. Four sensor sensitive parameters in the flight state parameters of the UAV IMU module are also selected, including the total pressure and static pressure of airspeed, X-axis linear acceleration, Y-axis magnetic field, and Z-axis angular velocity. Each sensor type data includes four fault type labels: constant deviation (C), drift (D), mutation (I), and noise deviation (B); each sensor type signal is cut into data samples with a length of 1024 without overlap, and 100 samples are established for each type of sensor data;

[0086] The order of data samples under each fault type in the source domain sample data set is shuffled and 80% of the samples are randomly selected as the source domain training set, and the remaining 20% ​​of the samples are used as the source domain verification set; 100 samples are selected from the target domain sample data set, of which 80% are used as the training set and 20% are used as the verification set; and 100 samples are randomly selected from the target domain sample data set as the test set (see Table 2).

[0087] S3. Train the multi-scale neural network constructed in step S1; in the process of training the source domain and target domain data sets by the feature extractor, reduce the distribution gap between the source domain and target domain samples by the domain adaptation method; propose to assign a weighting factor to the source domain samples by the discrimination difficulty of the domain discriminator, measure the contribution of the source domain samples to the training model by the weighting factor, and improve the training effect of the model; specifically include the following steps:

[0088] S301. Initialize network parameters: Before training, perform parameter initialization and set the model. Use source domain samples to pre-train the model for 50 iterations. Use stochastic gradient descent to update the parameters of each sample. The learning rates of the feature extractor, domain discriminator, and classifier loss functions are set to 0.001, 0.0001, and 0.0001, respectively. The momentum of the stochastic gradient descent method is set to 0.85. The gradient reversal layer method is used to automatically reverse the gradient during the back propagation of the domain discriminator, and remains unchanged during the forward propagation. Set to 1.2 respectively , 0.75 .

[0089] In a specific embodiment, the experimental running software and hardware environment is Pycharm, the processor is Intel (R) Core (TM) i7-10870CPU@2.20GHz, the memory is 64G, and the Windows 10 operating system is run. The algorithm proposed in this article is implemented through the Pytorch framework, where Pytorch=1.11.0 and python=3.9.12.

[0090] S302. In order to quantitatively measure the prediction performance of the model, a loss function is used to measure the gap between the model prediction result and the true category of the sample; the loss function of the domain discriminator, the loss function of the classifier, and the loss function of the multi-core maximum mean difference distance between the source domain and the target domain data of the feature extractor during the training process are calculated respectively; and finally the total target loss function is calculated; the details are as follows:

[0091] The loss function of the multi-core maximum mean difference distance between the source domain and the target domain data during the feature extractor training process is expressed as follows:

[0092] ;

[0093] in, represents the i-th sample in the source domain, is the total number of source domain samples; represents the jth sample in the target domain, is the total number of samples in the target domain; represents the network parameters of the feature extractor, represents a Hilbert space;

[0094] When the feature extractor extracts the input source domain and target domain data, the multi-kernel maximum mean discrepancy MK-MMD (Multi-kernel Maximum Mean Discrepancy, MK-MMD) method is used to minimize the distribution distance between the source domain and the target domain:

[0095] .

[0096] Brief description of the principle: The distribution of source domain and target domain data is different, and the size of the distribution difference between the two determines the recognition effect of the model. In order to measure the distance between the feature distribution of the source domain and different target domains, and then narrow the gap in feature distribution between the source domain and the target domain, and improve the recognition effect of the model, a domain adaptation method is added between the layers of the network model. This study specifically adopts the multi-kernel maximum mean difference MK-MMD (Multi-kernel Maximum Mean Discrepancy, MK-MMD) method. Among them, the maximum mean difference (Maximum Mean Discrepancy, MMD) method is the basis, which reduces the difference in feature distribution between different domains through distance measurement. Given the source domain S and the target domain T, Xs and Xt represent the sample sets of the source domain and the target domain respectively, is the number of samples in the source domain, is the number of samples in the target domain. The samples of the source domain and the target domain are mapped to the Hilbert space (Reproducing Kemel Hilbert Space, RKHS) through feature mapping, and the difference in the mean in RKHS is used as the distance between the data distributions of the two domains. The MMD distance measurement formula is as follows:

[0097] ;

[0098] in, represents a Hilbert space; Represents the mapping from the original feature space of the data to the nonlinear space. By minimizing the above formula, the desired nonlinear mapping can be found However, direct optimization of the above formula often falls into local extreme points, and transforming the problem into a kernel learning problem can avoid falling into local extreme points. With kernel mapping The related feature map, the mean distance between the two domains in the above formula is converted into an unbiased quadratic estimate of MMD based on empirical multi-kernel mean embedding, and its formula is:

[0099] ;

[0100] Compared with the single kernel transformation MMD, MK-MMD is the optimal kernel obtained by multi-kernel linear combination. Multi-kernel k can use different kernels to enhance the effect of MK-MMD, and finally achieve an optimal and reasonable kernel selection. represents a specific k Hilbert space, and It is defined as a convex combination of m PSD kernels, so the problem of traditional kernel parameter selection is converted into a convex optimization problem of multi-core kernel function, where the set of multi-core Gaussian kernel function is:

[0101] ;

[0102] The function set is a convex combination of m Gaussian kernel functions with different kernel widths, is the convex optimization parameter set. To solve the optimal value , construct the following function:

[0103] ;

[0104] In the formula, is the weight, is the characteristic core;

[0105] To make the function set To obtain the maximum value, there is the following convex optimization problem:

[0106] ;

[0107] ;

[0108] By the maximum minimization formula:

[0109] , which can adapt to the probability distribution of cross-domain transfer learning, that is, there is a solution to the following problem: .

[0110] Domain Discriminator The binary cross entropy loss function is used, and the loss function is shown as follows:

[0111] ;

[0112] in, represents the i-th sample in the source domain, is the total number of source domain samples; represents the jth sample in the target domain, is the total number of samples in the target domain; represents the network parameters of the feature extractor; represents the network parameters of the domain discriminator.

[0113] The purpose of the classifier is to distinguish the category labels of the samples, and the loss function is as follows:

[0114] ;

[0115] in, represents the network parameters of the feature extractor, represents the network parameters of the classifier, is the total number of source domain samples, is the number of category labels; express The probability of belonging to class k.

[0116] In order to quantitatively measure the prediction performance of the model, a loss function is usually used to measure the gap between the model prediction results and the true category of the sample. represents the discriminator loss function, represents the loss function of the classifier, the MMD distance between the source domain and the target domain data in the feature extractor, The loss function representing the MK-MMD distance between source and target domain data in the feature extractor.

[0117] Calculate the total target loss function; the specific process is as follows:

[0118] The weight value of the source domain sample is calculated by the discrimination difficulty of the domain discriminator; let Represents the weight of the i-th sample in the source domain, then the formula is as follows:

[0119] ;

[0120] The minimum-maximum normalization method is used to scale the weight, and the normalized weight is for:

[0121] ;

[0122] In the formula, .

[0123] The source domain sample allocation weights are introduced into the domain discriminator loss function. The loss function As shown below:

[0124] ;

[0125] The total target loss function is obtained as follows:

[0126] ;

[0127] In the formula, is the balance parameter between classification error and domain error, and α is a hyperparameter.

[0128] Principle: Traditional unsupervised domain adversarial adaptive methods usually adapt the weight distribution ratio of source domain samples to the target domain data. When the difference between source domain samples and target domain samples is too large, negative transfer of the diagnostic model will occur. In order to better calculate the contribution of different samples to the model, this study proposes to calculate the weight value of the source domain sample by the discrimination difficulty of the domain discriminator.

[0129] S303. Bring the training set into the input layer of the multi-scale neural network model, start training the multi-scale neural network model, and when the number of network iterations reaches the set number of iterations, bring in the validation set to validate the model and fine-tune the parameters;

[0130] Specifically, the number of training iterations of the model is set to 200.

[0131] S304. Bring in the test set to test the recognition effect of the model; when the model parameters are optimized to the target value, the model training is completed.

[0132] The model parameter optimization has two goals: the first is to accurately identify the fault data in the target domain, that is, to minimize the classification error of the classifier; the second is to confuse the source domain data with the target domain data, that is, to maximize the inter-domain classification error. The model parameter optimization goal is as follows:

[0133]

[0134] ;

[0135] In the formula, Respectively represent the optimized network parameters corresponding to the feature extractor, classifier and domain discriminator;

[0136] The parameter optimization uses the Stochastic Gradient Descent (SGD) method to update the parameters in the opposite direction of the gradient vector. , , , will be updated as follows:

[0137]

[0138]

[0139] ;

[0140] in, is the learning rate, is a balance parameter between classification error and domain error, and α is a hyperparameter. Since the gradient of domain discrimination loss and the gradient of target classification loss have opposite training directions, a gradient reversal layer (GRL) is designed between the feature extractor and the domain discriminator so that the gradient direction of the domain discrimination error is automatically reversed during the back propagation process and the identity transformation is achieved during the forward propagation process. The relevant mathematical expression is shown in the following formula:

[0141] ;

[0142] In the formula, R λ (x) is the function that defines the gradient reversal layer, x is the domain discrimination error, λ is the back propagation coefficient, and I is the domain discrimination unit matrix; the gradient reversal layer is set between the feature extractor and the domain discriminator, and the parameter λ is not a fixed value, but changes dynamically, as shown in the following formula:

[0143] ;

[0144] In the formula, p represents the relative value of the iteration process, that is, the ratio of the current number of iterations to the total number of iterations. The growth trajectory follows dynamic changes.

[0145] The model training process is as follows Figure 2 The first step is to build a source domain dataset The nonlinear mapping from the source domain label space to the trained model acquires source domain knowledge, such as Figure 2 As shown in (b); the second step is to establish a cross-domain model to reduce the distribution difference between the source domain and the target domain and extract domain-invariant features, such as Figure 2 As shown in (c), the model finally obtains the classification and recognition of the target domain dataset without labels.

[0146] The specific training and testing process is as follows Figure 3 shown.

[0147] Specifically, each type of migration experiment was repeated 10 times during the training process and the average value was taken.

[0148] In order to show the model transfer learning effect in a generalized manner, 6 groups of transfer experiments were conducted, namely A→B, B→C, C→D, D→A, A→C and B→D. The accuracy was used as the model measurement annotation. Each group of experiments was performed 10 times and the average value was taken.

[0149] The evaluation indicators of the present invention are precision, recall, accuracy (ACC), and F1 score, which are used to evaluate the performance of the model; precision refers to the ratio of the actual positive samples to the number of positive samples predicted by the model; recall refers to the ratio of the actual positive samples predicted by the model to the positive samples; F1 score is the harmonic average of precision and recall. The calculation formula is as follows:

[0150] ;

[0151] Among them, TN, TP, FN, and FP represent the number of true negative, true positive, false negative, and false positive samples, respectively.

[0152] Table 3 Model test results

[0153]

[0154] As shown in Table 3, the network's cross-domain fault recognition accuracy can be maintained above 85% between different data sets, and the data set pairs with strong cross-domain recognition capabilities can exceed 90%. The network has a strong recognition ability for the four types of fault labels, and when the number of samples in the source domain sample data set and the target domain sample data set are unbalanced, the model can maintain a high fault label recognition rate.

[0155] Taking experiment A→B as an example, the model accuracy and loss value change as follows Figure 4 , Figure 5 As shown in the figure, acc represents accuracy, loss represents loss function value, and epoch represents the number of iterations. Figure 4 It can be seen that in the model training stage, with the increase of training batches, the training accuracy of the label classifier continues to rise, approaching 90% and tending to be stable after about 120 iterations, and the accuracy fluctuates greatly during the iteration process; in the test stage, the test accuracy of the label classifier is close to 90% and tends to be stable after about 70 iterations, and the accuracy fluctuates less during the iteration process. This shows that after gradually learning the characteristics of the four types of label signals, this method can basically correctly judge the fault label based on the characteristics. From the loss Figure 5 It can be seen that the total loss of the model gradually decreases with the increase of training batches, and the parameters are continuously updated in the direction of reducing the difference between the source domain and the target domain, and finally tends to 0, indicating that the domain discriminator can basically no longer distinguish whether the data comes from the source domain or the target domain, and has mapped the common features of the source domain and the target domain to the same feature space.

[0156] S4. Perform fault identification and diagnosis on the target domain data through the trained multi-scale neural network model.

[0157] In order to evaluate the effectiveness of the UAV sensor fault diagnosis method based on weighted transfer domain adaptation learning provided by the present invention, four diagnostic methods were used to compare the transfer learning model effect with the method proposed by the present invention. The compared methods include convolutional neural network (CNN), transfer component analysis (TCA), deep domain adaptation method (DAN), and dynamic adversarial adaptive network (DAAN) method. The experimental parameter settings are the same as above (training batch size, number of iterations, SGD parameter settings), and during the network training process, 400 sample source domain data for each state are selected as the training set, and 100 data are randomly selected from the target domain sample data set as the test set for training. The comparison results are shown in Table 4.

[0158] Table 4 Test results of comparison methods

[0159]

[0160] As can be seen from Table 4, the accuracy of the CNN method in cross-domain fault identification is less than 50%, and it cannot identify the target domain faults well. This shows that the knowledge obtained from the source domain by the traditional deep learning method cannot extract enough feature information to identify the target domain data. The TCA method has improved in fault identification, up to more than 70%. The accuracy of the DAN model is about 80%, and the accuracy of the DAAN method is further improved to 87%. The accuracy of the method provided by the present invention (abbreviated as WTDAN) reaches about 90%, which is higher than other compared methods and can effectively identify cross-domain fault diagnosis of common drone sensors.

[0161] In order to further demonstrate the diagnostic effects of different methods on faults, the confusion matrix of the above experiment is visualized, as shown in Figure 6 shown in Figure 6 In (a)-(e), there are 100 test data for each state, and 0-4 correspond to normal and 4 state faults respectively. The source domain normal, constant deviation, drift, mutation, and noise deviation fault labels are set to S_0, S_1, S_2, S_3, S_4 respectively, and the target domain is T_0, T_1, T_2, T_3, T_4 respectively. The horizontal axis is the predicted label and the vertical axis is the real label. According to the confusion matrix, various methods have different abilities to identify sensor faults of different state categories. Overall, the method proposed in the present invention has the best effect on fault identification.

[0162] t-SNE (t-distributed stochastic neighbor embedding) is a nonlinear dimensionality reduction technique used to explore the structure of high-dimensional data. It can preserve the local structure of the original high-dimensional data in a low-dimensional space, so it is particularly suitable for the visualization of high-dimensional data. In order to clearly demonstrate the classification effect of the model, the classification results of the test data obtained from the above 7 groups of experiments were processed by t-SNE dimensionality reduction. t-SNE visualizes high-dimensional data by mapping samples in the original feature space to a two-dimensional space, such as Figure 7 As shown in (a)-(e).

[0163] From the t-SNE graphs of the test results, it can be seen that the cluster distribution of each fault feature obtained by the CNN method is not clear in the two-dimensional space, the clusters of different categories of features are overlapped together, the fault recognition rate is low, and the separability of the differences between different features is poor. This shows that under the condition of small samples in the target domain, the model trained by the CNN method cannot identify fault features across domains. The clustering regions of different features extracted by the TCA method are identifiable, but the distribution of feature points of the same sample is relatively scattered, and the cluster distribution of different samples is overlapped at the regional boundaries, and the regional boundaries are not clear. This shows that the TCA network is shallow and the ability to extract high-dimensional feature information is limited. Therefore, the clustering effect of similar features in the t-SNE graph is not concentrated. The clustering effect of the DAN method on feature points is more concentrated than that of the TCA method, but the cluster distribution of the same feature points in some source domains and target domains is not aligned. This shows that when extracting domain-invariant features of different categories of source domains and target domains, the model is not robust enough and the recognition accuracy of some fault categories is not enough. The clustering performance of the DAAN method is further improved, the clustering distribution of feature points of different categories is concentrated, the boundaries are clear, and the overlap of common fault features in the target domain and the source domain is roughly consistent. However, there are still a few cases of boundary aliasing and misalignment of source and target domain clusters. This shows that although the model has good recognition performance for most categories, the conditional probability distribution on some small sample data sets still needs to be optimized. The features extracted by the proposed method in the present invention are more compactly clustered between samples of the same type in two-dimensional space, and the feature regions of different categories are significantly divided, with better separability. The distribution of common fault features in the source domain and the target domain can be aligned, which has better beneficial effects.

[0164] The invention relates to a transfer learning method, which is a machine learning method that can transfer existing knowledge to solve problems in different but similar or related fields. When the target task of the unmanned aerial vehicle lacks high-quality training data, the knowledge and experience of the existing data can be reused, saving the cost of recollecting a large number of labeled samples, and improving the efficiency of model training. The establishment of the transfer learning model needs to meet the distribution difference between the source domain and the target domain. If the distribution difference is large, negative transfer problems will occur. Therefore, by introducing a domain adaptation method, the distribution difference between the source domain and the target domain is reduced. The generative adversarial network can confuse the source domain data set and the target domain data set through a mutual game between a generator and a discriminator, minimize the distribution difference between the two domains, train cross-domain shared features, and finally enable the model to have the ability to diagnose faults in the target domain data set. Compared with other deep learning methods, the present invention has the label information that does not depend on the target domain data set, can reuse the knowledge and experience of the existing data set, and better generalize to the problem solving of new tasks. At the same time, the global repeated training of new tasks is reduced, and the training efficiency of the model is improved. The trained model has the advantages of good robustness, high fault recognition rate, wide application range, etc.

[0165] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0166] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for fault diagnosis of unmanned aerial vehicle sensors based on weighted transfer domain adaptation learning, characterized in that: The specific steps include: S1. Construct a multi-scale neural network model, including three multi-scale neural network modules: feature extractor, domain discriminator and classifier; Among them, the feature extractor receives input data as an entry, maps the input data to a specific space, and extracts high-dimensional feature information of the data; the domain discriminator is used to confuse the source domain sample data set and the target domain sample data set, and conducts adversarial training with the feature extractor during the model training process, so that the domain discriminator cannot distinguish whether the data comes from the source domain or the target domain; the classifier identifies and classifies the sample features output by the feature extractor, and finally determines the category corresponding to the data; A gradient reversal layer is designed between the feature extractor and the domain discriminator so that the gradient direction of the domain discrimination error is automatically reversed during the back propagation process and the identity transformation is achieved during the forward propagation process; S2. Obtain source domain sample dataset and target domain sample dataset: the source domain dataset is selected from the public dataset of drone faults, the aerial laboratory fault and anomaly dataset; sensor data from the actual flight records of unlabeled drones are collected as the target domain sample dataset; the source domain sample dataset and the target domain sample dataset both include four types of fault samples, including constant deviation, drift, mutation and noise deviation; S3. Training the multi-scale neural network model constructed in step S1; specifically comprising the following sub-steps: S301. Initialize network parameters: Before training begins, perform parameter initialization settings, and use source domain samples to pre-train the multi-scale neural network model for several iterations; S302. Calculate the loss function of the domain discriminator, the loss function of the classifier, and the loss function of the multi-core maximum mean difference distance between the source domain and the target domain data of the feature extractor during the training process respectively; and finally calculate the total target loss function; The loss function of the multi-core maximum mean difference distance between the source domain and the target domain data during the feature extractor training process is expressed as follows: ； in, represents the i-th sample in the source domain, is the total number of source domain samples; represents the jth sample in the target domain, is the total number of samples in the target domain; represents the network parameters of the feature extractor, represents a Hilbert space; S303. Bring the training set into the input layer of the model and start training the model. When the number of network iterations reaches the set number of iterations, bring the validation set into the model to validate the model and fine-tune the parameters. S304. Bring in the test set to test the recognition effect of the model; when the model parameters are optimized to the target value, the model training ends; S4. Perform fault identification and diagnosis on the target domain data through the trained multi-scale neural network model.

2. The method for fault diagnosis of unmanned aerial vehicle sensors based on weighted transfer domain adaptation learning according to claim 1 is characterized in that: The data samples of each fault type in the source domain sample data set are shuffled and randomly divided into a source domain training set and a source domain verification set in a ratio of 8:2; Several samples are selected from the target domain sample data set and divided into a target domain training set and a target domain validation set in a ratio of 8:2; then the same number of samples are extracted from the target domain sample data set as the test set.

3. The method for fault diagnosis of unmanned aerial vehicle sensors based on weighted transfer domain adaptation learning according to claim 1 is characterized in that: The feature extractor includes 4 convolution blocks, a compression layer and a fully connected layer; each convolution block consists of a convolution layer, a pooling layer and an activation layer; the compression layer is used to convert multi-dimensional data into one-dimensional data output; the domain discriminator includes three fully connected layers and a Softmax function; the classifier includes two fully connected layers and a Softmax function.

4. The method for fault diagnosis of unmanned aerial vehicle sensors based on weighted transfer domain adaptation learning according to claim 3 is characterized in that: The loss function of the domain discriminator is the binary cross entropy loss function, as shown below: ； in, represents the i-th sample in the source domain, is the total number of source domain samples; represents the jth sample in the target domain, is the total number of samples in the target domain; represents the network parameters of the feature extractor; represents the network parameters of the domain discriminator; F represents the feature extractor, and D represents the domain discriminator; The loss function of the classifier is as follows: ； in, represents the network parameters of the feature extractor, represents the network parameters of the classifier, is the total number of source domain samples, is the number of category labels; express The probability of belonging to category k; F represents the feature extractor and C represents the classifier.

5. The method for fault diagnosis of unmanned aerial vehicle sensors based on weighted transfer domain adaptation learning according to claim 4 is characterized in that: The calculation process of the total objective loss function is as follows: The weight value of the source domain sample is calculated by the discrimination difficulty of the domain discriminator; let Represents the weight of the i-th sample in the source domain, then there is the following formula: ； The weight after scaling the weight using the minimum-maximum normalization method for: ； In the formula, ; The source domain sample allocation weights are introduced into the loss function of the domain discriminator. The loss function As shown below: ； The total target loss function is obtained as follows: ； In the formula, is the balance parameter between classification error and domain error, and α is a hyperparameter.

6. The method for fault diagnosis of unmanned aerial vehicle sensors based on weighted transfer domain adaptation learning according to claim 5 is characterized in that: The learning rates of the loss functions of the feature extractor, the domain discriminator, and the classifier are set to 0.001, 0.0001, and 0.0001, respectively.

7. A method for UAV sensor fault diagnosis based on weighted transfer domain adaptation learning according to any one of claims 1 to 6, characterized in that: The number of pre-training iterations in step S301 is 50 times; the number of network iterations in step S303 is set to 200 times.

8. The method for fault diagnosis of unmanned aerial vehicle sensors based on weighted transfer domain adaptation learning according to claim 1, characterized in that: The model parameter optimization target in step S304 is as follows: ； In the formula, Respectively represent the optimized network parameters corresponding to the feature extractor, classifier and domain discriminator; The parameter optimization uses the stochastic gradient descent method to update the parameters in the opposite direction of the gradient vector. , , , will be updated as follows: ； in, is the learning rate, is the balance parameter between classification error and domain error, and α is a hyperparameter; μ , α Set to 1.2λ respectively p , 0.75λ p ; The momentum of the stochastic gradient descent method is set to 0.85; the gradient reversal layer method is used to automatically reverse the gradient during the back propagation of the domain discriminator and remain unchanged during the forward propagation.

Images