Unmanned aerial vehicle real-time fault detection method based on deep domain adaptive BiLSTM

Abstract

The application relates to the technical field of unmanned aerial vehicle fault detection, and discloses an unmanned aerial vehicle real-time fault detection method based on a deep domain adaptive BiLSTM, which comprises the following steps: a feature extractor and a fault detector are pre-trained in a source domain, then past and future features are extracted from the flight data of the unmanned aerial vehicle time sequence under one working condition by using the feature extractor, and a feature set of the source domain and the target domain is obtained; the feature set is sent to a domain adapter for processing, the source domain and the target domain are classified by using the domain adapter, and the maximum difference between the source domain and the target domain is measured; the fault detector is optimized and trained in a minimum health state classification error mode, then the flight data of the unmanned aerial vehicle under any working condition is acquired, the fault detector is used for real-time fault detection of the unmanned aerial vehicle, and a health state report of the unmanned aerial vehicle is obtained. The application has the beneficial effects of reducing the difference between different working condition data and improving the accuracy of the unmanned aerial vehicle fault detection result.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) fault detection technology, and specifically to a real-time UAV fault detection method based on depth domain adaptive BiLSTM. Background Technology

[0002] In recent years, with the advancement and development of science and technology, the technology related to drones has also made significant progress, leading to the increasingly widespread application of drones in daily life. Drones can be remotely controlled or fly automatically according to a predetermined flight path. They have advantages such as light weight, small size, high maneuverability, and less susceptibility to flight environment limitations. Therefore, their application areas are gradually expanding, including firefighting drones, photography drones, reconnaissance drones, and so on.

[0003] During flight, drones are susceptible to complex and ever-changing environmental factors, leading to frequent malfunctions that can cause them to fail or even crash. Therefore, drone fault detection technology is crucial and a fundamental prerequisite for ensuring normal and safe flight. Drones involve many uncontrollable factors, and malfunctions during flight can cause significant losses. Therefore, flight control systems need to be more intelligent to meet safety and reliability requirements.

[0004] Currently, the main methods for UAV fault detection include model-based, expert knowledge-based, and data-driven methods. Model-based methods require an accurate model of the UAV for successful detection. In small UAV control systems, they are susceptible to various uncertainties and interference factors, and accurate models are often lacking. Using model-based methods may fail, thus failing to accurately detect UAV faults. Knowledge-based methods rely heavily on prior expert knowledge and have poor overall compatibility. Data-driven methods have great development potential and are a worthwhile new approach to explore. Therefore, this paper proposes a depth-domain-based UAV fault detection method to improve the accuracy of UAV fault detection results. Summary of the Invention

[0005] The present invention aims to provide a real-time fault detection method for UAVs based on deep domain adaptive BiLSTM, so as to improve the accuracy of UAV fault detection results.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a real-time fault detection method for unmanned aerial vehicles based on depth domain adaptive BiLSTM, comprising the following steps:

[0007] Step S1: Pre-train the feature extractor and fault detector in the source domain, and then use the feature extractor to extract past and future features from the time-series flight data of the UAV under a certain working condition to obtain the feature sets of the source domain and the target domain.

[0008] Step S2: The feature set is sent to the domain adaptor for processing. The domain adaptor is used to classify the source domain and the target domain, and to measure the maximum difference between the source domain and the target domain.

[0009] Step S3: Optimize the fault detector to minimize the health status classification error, then acquire the UAV's flight data under any operating condition, and use the fault detector to perform real-time fault detection on the UAV to obtain the UAV's health status report.

[0010] The principle and advantages of this scheme are as follows: In practical applications, a set of flight condition data of UAVs is randomly collected, and features are extracted from the flight data to train the model. Then, the distribution difference between the source and target domains in the flight data is found, making it impossible for the domain classifier to accurately identify the source and target domains, thus maximizing the domain classification error. At the same time, the distribution difference distance between the source and target domain features is minimized, and the health status classification error of the fault detector is minimized. By performing domain adaptation processing on the data, the difference between two different flight condition data is reduced. After training the domain adaptor and fault detector using a set of flight condition data, it can be applied to fault detection under all flight conditions, making the diagnostic accuracy of UAV faults more accurate. Compared with existing technologies, the advantage of this scheme is that by performing domain adaptation processing on the collected real-time flight data, the source and target domains can achieve a domain adaptation effect, reducing their difference, which is conducive to transfer learning. This minimizes the fault classification error of the fault detector trained on flight data under one flight condition, and can be extended to UAV fault detection under all flight conditions, greatly improving the accuracy of UAV fault detection in different new environments. On the other hand, this solution can also further improve the accuracy of fault detection results for UAVs in unknown flight missions, thereby ensuring the flight safety of UAVs.

[0011] Preferably, as an improvement, the extraction of past and future features includes the following:

[0012]

[0013]

[0014]

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

[0022] Preferably, as an improvement, the domain adaptor includes a domain classifier module and an MMD module. The domain classifier module is used to classify the source domain and the target domain and minimize the domain classification error to accurately identify the source domain and the target domain. The MMD module is used to measure the maximum mean difference between the features of the source domain and the features of the target domain according to a preset calculation formula.

[0023] Preferably, as an improvement, the input to the domain classifier module includes source domain features and target domain features extracted by the feature extractor, and the label of the source domain is defined as 0 and the label of the target domain is 1.

[0024] Preferably, as an improvement, the preset calculation formula is:

[0025]

[0026] Preferably, as an improvement, the fault detector uses the cross-entropy loss function as the objective function to minimize the health status classification error of the source domain during optimization training.

[0027] Preferably, as an improvement, the cross-entropy loss function is:

[0028] Where n is the value of bathsise, k is the number of species, and y j For the actual value, p(y) j ) represents the predicted value.

[0029] Preferably, as an improvement, the domain adaptor also includes a preset loss function to maximize the optimization of the domain classification error. The loss function is:

[0030] Where n is the value of bithsise, y i For the label of the field, 0 or 1, p(y) i ) for y i The predicted value.

[0031] Preferably, as an improvement, when performing combined optimization of the domain adaptor and the fault detector, the optimization of multiple objectives is accomplished using a total optimization formula; the total optimization formula is: L total =L f -λ1L d +λ2L m .

[0032] Preferably, as an improvement, the feature extractor includes two BiLSTM networks with shared weights, and the BiLSTM networks have 3 layers, each containing 64 hidden units. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating an embodiment of the real-time fault detection method for unmanned aerial vehicles based on depth domain adaptive BiLSTM according to the present invention.

[0034] Figure 2 This is a schematic diagram of the transfer learning framework of Embodiment 1 of the real-time fault detection method for UAVs based on deep domain adaptive BiLSTM of the present invention.

[0035] Figure 3 This is a schematic diagram of feature extraction in Embodiment 1 of the real-time fault detection method for UAVs based on depth domain adaptive BiLSTM of the present invention. Detailed Implementation

[0036] The following detailed description illustrates the specific implementation method:

[0037] Example 1:

[0038] This embodiment is basically as shown in the appendix. Figure 1 As shown: A real-time fault detection method for UAVs based on deep domain adaptive BiLSTM includes the following steps:

[0039] Step S1: Pre-train the feature extractor and fault detector in the source domain, and then use the feature extractor to extract past and future features from the time-series flight data of the UAV under a certain working condition to obtain the feature sets of the source domain and the target domain.

[0040] Step S2: The feature set is sent to the domain adaptor for processing. The domain adaptor is used to classify the source domain and the target domain, and to measure the maximum difference between the source domain and the target domain.

[0041] Step S3: Optimize the fault detector to minimize the health status classification error, then acquire the UAV's flight data under any operating condition, and use the fault detector to perform real-time fault detection on the UAV to obtain the UAV's health status report.

[0042] Because flight data records detailed information about the drone's flight process, containing a wealth of crucial information, feature extractors are used to extract knowledge about the drone's health status from the flight data for subsequent fault detection. (See attached image) Figure 2As shown, in the transfer learning framework, the feature extractor consists of two weight-shared 3-layer BiLSTM networks (F1, F2), each with 64 hidden units. The reason for choosing the BiLSTM network in this scheme is that UAV flight data has strong temporal sequence, and the states before and after are related. BiLSTM can handle such temporal data well, and can process both forward and backward data. It can extract past and future features from temporal flight data, and these features contain the key information needed for UAV fault diagnosis.

[0043] The specific process of feature extraction is as follows:

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] Fault detectors are used to accurately detect faults in drones, thereby identifying the drone's health status. Their structure is shown in the attached diagram. Figure 2 As shown, its input consists of features extracted from the source domain data by BiLSTM, and its output is the identification result of the UAV's health status. When optimizing the fault detector, the main goal is to minimize the health status classification error in the source domain; therefore, the cross-entropy loss function is chosen as the objective function for optimization, as shown in the following equation:

[0055] Where n is the value of bathsise, k is the number of species, and y j For the actual value, p(y) j ) represents the predicted value.

[0056] Because drones are affected by multiple factors during flight, such as weather and control errors, flight data for different missions can vary greatly. In order to successfully complete fault detection, a domain adaptor is set up to reduce the difference between the source and target domains and improve the accuracy of fault detection.

[0057] As attached Figure 2 As shown, the domain adaptor includes a domain classifier module and an MMD module. The input to the domain adaptor is the source and target domain features extracted by the feature extractor, with the source domain labeled as 0 and the target domain labeled as 1. The domain classifier's main task is to classify the source and target domains. For a single domain classifier, its optimal state is to accurately identify the source and target domains while minimizing the domain classification error. The entire optimization training process aims to prevent the domain classifier from accurately classifying the source and target domains and to maximize the domain classification error, thereby achieving the domain adaptation effect. The entire optimization process is accomplished using a defined loss function.

[0058] Where n is the value of bithsise, y i For the label of the field, 0 or 1, p(y) i ) for y i The predicted value.

[0059] On the other hand, the MMD module of the domain adaptor is used to measure the maximum difference between the source domain and the target domain. It measures the maximum mean difference between the source domain features and the target domain features according to a preset calculation formula. The purpose of optimization is to minimize the distribution difference distance between the learned features of different domains.

[0060] The preset calculation formula is:

[0061] As attached Figure 3 As shown, the purpose of domain adaptation is to train a feature extractor that extracts similar features from source and target domain data. If similar features can be extracted, it means that the source and target domains have achieved the effect of domain adaptation, and the difference between the two becomes smaller, which is beneficial to transfer learning.

[0062] Therefore, the entire transfer learning framework has the following four optimization objectives:

[0063] 1) Extract as much knowledge about the health status of the drone as possible from the flight data and use it for subsequent fault detection;

[0064] 2) To enable the domain classifier to accurately identify the source domain and the target domain, while the goal of the entire framework is to maximize the domain classification error. If the domain classifier cannot accurately classify the features between the source domain and the target domain, it means that the features are domain invariant.

[0065] 3) Minimize the distribution difference distance between features in the source and target domains;

[0066] 4) Minimize the health status classification error in the fault detector.

[0067] Therefore, by combining the above multiple optimization objectives, we obtain the overall optimization formula:

[0068] L total =L f -λ l L d +λ2L m

[0069] The specific implementation process of this embodiment is as follows:

[0070] 1) Fault detection model training: First, the feature extractor and fault detector are trained 100 times in the source domain, during which the domain adaptor is not involved in the training. Then, flight data of the UAV under a random working condition is collected as the target domain data and simultaneously input into the feature extractor and domain classifier along with the source domain data to train the domain classifier. The source domain data is divided into a training set and a validation set. The validation set supervises the training process and saves the model. All target domain data is used as the test set. The training set and the test set are fed into F1 and F2 respectively, and the deep features fs and ft are output. Then, D performs domain classification on fs and ft, and L... d Backpropagation and updating the parameters of D using the Adam optimizer train its ability to identify the source and target domains; further, fs is input into a fault detector for fault detection, and L is calculated. f Simultaneously, the distribution difference distance L between fs and ft is calculated by the MMD module. m Finally, F and C passed through L. total Backpropagation is performed and the parameters are updated using the Adam optimizer.

[0071] 2) Actual fault detection: acquire flight data of the UAV under any other working conditions, input it into F1 and C, and the fault detector completes specific fault diagnosis and analysis to obtain the health status report of the UAV. Finally, the fault category of the UAV is output from C.

[0072] Currently, drones have become widely used in daily life. However, with the development of drone technology, the requirements for drone use have gradually shifted from simple control to safe and stable control. This is partly due to cost considerations and partly to ensure flight safety and prevent accidental crashes that could injure people. Many uncontrollable factors exist throughout a drone's flight, the most critical being the complex and ever-changing weather and control errors. Since the weather conditions and flight paths encountered by a drone are different each time it performs a mission, the flight data is also different each time. Therefore, analyzing and diagnosing drone malfunctions based on data from a single or limited number of flights is inaccurate.

[0073] This solution specifically addresses the aforementioned issues by proposing a novel UAV fault detection method. It primarily utilizes deep transfer learning combined with domain adaptation processing of collected flight data to reduce the variability of flight data under different flight conditions, thereby improving the accuracy of fault diagnosis. Specifically, this solution first randomly acquires flight data under one flight condition for model training. It then extracts past and future features from the flight data, identifying the distribution differences between the source and target domains. This prevents the domain classifier from accurately identifying the source and target domains, maximizing the domain classification error. Simultaneously, it optimizes the minimization of the distribution difference distance between source and target domain features and the minimization of the health state classification error by the fault detector. This allows for multi-condition flight data fault detection, diagnosis, and analysis after model training based on single flight data, accurately identifying UAV fault types and significantly improving the accuracy of fault detection results. This solution not only improves the diagnostic accuracy of UAV faults but also enables the application of this fault detection method to UAV test flight missions, enhancing the accuracy of fault detection results in unknown flight missions, preventing accidental UAV crashes, and thus improving UAV flight safety.

[0074] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A real-time fault detection method for unmanned aerial vehicles based on deep domain adaptive BiLSTM, characterized by: Includes the following steps: Step S1: Pre-train the feature extractor and fault detector in the source domain, and then use the feature extractor to extract past and future features from the time-series flight data of the UAV under a certain working condition to obtain the feature sets of the source domain and the target domain. The feature extractor comprises two weight-shared BiLSTM networks with 3 layers, each containing 64 hidden units, processing forward and backward data; the feature extractor is used to extract knowledge about the health status of the drone from flight data. During optimization training, the fault detector uses the cross-entropy loss function as the objective function to minimize the health status classification error of the source domain. The cross-entropy loss function is: in, For the value of bathsise, For the number of types, This is the actual value. This is a predicted value; The extraction of past and future features includes the following: Step S2: The feature set is sent to the domain adaptor for processing. The domain adaptor is used to classify the source domain and the target domain, and to measure the maximum difference between the source domain and the target domain. By setting a domain adaptor, the difference between the source domain and the target domain can be reduced; by domain adaptation processing, the differences between UAV flight data under different flight conditions can be reduced. The domain adaptor includes a domain classifier module and an MMD module. The domain classifier module is used to classify the source domain and the target domain, minimize the domain classification error, and accurately identify the source domain and the target domain. The MMD module is used to measure the maximum mean difference between the features of the source domain and the features of the target domain according to a preset calculation formula. The preset calculation formula is: The domain adaptor also includes a preset loss function to maximize and optimize the domain classification error. The loss function is as follows: in, For the value of bithsise, The label for the field is 0 or 1. To The predicted value; Step S3: Optimize the fault detector to minimize the health status classification error, then acquire the flight data of the UAV under any operating condition, and use the fault detector to perform real-time fault detection on the UAV to obtain the health status report of the UAV. When performing combined optimization of the domain adaptor and the fault detector, the overall optimization formula is used to optimize multiple objectives; the overall optimization formula is: .

2. The real-time fault detection method for UAVs based on depth domain adaptive BiLSTM according to claim 1, characterized in that: The input to the domain classifier module includes source domain features and target domain features extracted by the feature extractor, and the label of the source domain is defined as 0 and the label of the target domain is 1.

Images