Fault diagnosis method, device and equipment for flight control system of unmanned aerial vehicle and storage medium

Abstract

The invention discloses an unmanned aerial vehicle flight control system fault diagnosis method, apparatus and device, and a storage medium, the diagnosis method comprises the steps of constructing a training data set, each training sample in the training data set comprising an input vector and a fault label generated according to flight data; constructing a deep learning model fusing the LSTM and the attention mechanism; training the deep learning model by using the training data set to obtain a fault diagnosis model; dynamically adjusting the structure and parameters of the fault diagnosis model according to real-time flight environment information and / or task requirements; carrying out retraining / fine tuning on the dynamically adjusted fault diagnosis model; and performing fault diagnosis on the real-time flight data by using the fault diagnosis model after retraining / fine tuning. According to the method, the fault diagnosis accuracy is improved, the diagnosis time is greatly shortened, and the system can quickly respond to sudden faults.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of unmanned aerial vehicle fault diagnosis, and particularly relates to an unmanned aerial vehicle flight control system fault diagnosis method, device, equipment and storage medium. BACKGROUND

[0002] The unmanned aerial vehicle flight control system collects and analyzes data such as attitude, position, and speed from various sensors in real time, combines preset instructions and algorithms, and accurately controls the flight trajectory, power output, and action execution of the unmanned aerial vehicle. Whether it is autonomous obstacle avoidance in complex terrain, attitude adjustment during high-speed flight, or precise hovering and positioning during tasks such as aerial photography and material delivery, it all relies on the precise control of the flight control system. Once the flight control system fails, the unmanned aerial vehicle will lose stable control, which may lead to flight out of control, task failure, and even cause a crash accident. Therefore, the reliability and stability of the flight control system directly determine the flight safety and task success of the unmanned aerial vehicle, and are the fundamental guarantee for the unmanned aerial vehicle to play its important social value. The unmanned aerial vehicle flight control system generally covers sensors, actuators, and on-board computers, and is the most prone to failure part of the entire unmanned aerial vehicle system. The health status of the flight control system is the key to ensuring the reliable and safe operation of the unmanned aerial vehicle.

[0003] Currently, common unmanned aerial vehicle flight control system fault diagnosis methods mainly include two types: sensor data-based and model-based. The sensor data-based fault diagnosis method relies on statistical methods or traditional machine learning algorithms to build a data model. When the flight environment changes complexly, such as increased electromagnetic interference and severe weather fluctuations, it is difficult to adaptively adjust the diagnosis strategy, and it is difficult to accurately capture changes in data characteristics, resulting in a high error rate in hidden fault diagnosis and failing to meet the safety flight requirements of unmanned aerial vehicles. The model-based fault diagnosis method attempts to diagnose faults through accurate mathematical models, but the complexity of the unmanned aerial vehicle system makes the model construction costly and the parameters difficult to accurately determine. In practical applications, not only a large amount of resources are consumed, but also the deviation between the model and the actual system leads to a decrease in the accuracy of fault diagnosis.

[0004] Most existing diagnosis methods based on deep learning use fixed model structures and parameters, and cannot respond and adjust in time when facing different flight task requirements, such as high-altitude high-speed flight and low-altitude complex terrain reconnaissance, resulting in slow fault diagnosis response speed and difficulty in meeting the real-time requirements of unmanned aerial vehicles. At the same time, these methods generally lack the ability to predict faults and cannot provide early warnings before faults occur, which poses a potential safety hazard to unmanned aerial vehicle flight. SUMMARY

[0005] The unmanned aerial vehicle flight control system fault diagnosis method, device, equipment and storage medium provided by the present application can solve at least one of the problems of low diagnosis accuracy of traditional methods for hidden faults and complex faults, and slow fault diagnosis reflection caused by fixed model structure and parameters, and cannot meet real-time requirements.

[0006] The present application solves the above technical problems by the following technical solutions: an unmanned aerial vehicle flight control system fault diagnosis method, comprising:

[0007] A training data set is constructed; each training sample in the training data set includes an input vector generated according to flight data and a fault label;

[0008] A deep learning model combining LSTM and attention mechanism is constructed;

[0009] The training data set is used to train the deep learning model to obtain a fault diagnosis model;

[0010] According to real-time flight environment information and / or task requirements, it is determined whether the fault diagnosis model needs to be adjusted, if so, the structure and parameters of the fault diagnosis model are dynamically adjusted, the dynamically adjusted fault diagnosis model is retrained / fine-tuned, and the retrained / fine-tuned fault diagnosis model is used to diagnose faults of real-time flight data;

[0011] If not, the unadjusted fault diagnosis model is used to diagnose faults of real-time flight data.

[0012] Further, the training data set is constructed, comprising:

[0013] Flight data of the flight control system is obtained, and each flight data is labeled to obtain corresponding fault labels;

[0014] The flight data is denoised by using a Kalman filtering algorithm;

[0015] The denoised flight data is normalized to generate an input vector corresponding to each flight data.

[0016] Further, the deep learning model comprises at least one LSTM network layer, an attention mechanism layer and a fully connected layer;

[0017] The LSTM network layer is used to extract the time sequence dependence of the flight data;

[0018] The attention mechanism layer is used to calculate attention output based on the time sequence dependence extracted by the LSTM network layer to focus on key fault features;

[0019] The full connection layer is used to obtain a probability distribution of a fault type based on the attention output.

[0020] Further, the attention output is calculated based on a time sequence dependency extracted by an LSTM network layer, and specifically includes:

[0021] The hidden state output by the LSTM network layer is converted into a query vector, a key vector and a value vector through linear transformation;

[0022] The attention score is calculated according to the query vector, the key vector and the value vector;

[0023] The attention output is calculated according to the attention score and the value vector.

[0024] Further, the structure and parameters of the fault diagnosis model are dynamically adjusted, including:

[0025] When it is determined that the real-time flight environment information is a complex environment, the number of layers of the LSTM network layer in the fault diagnosis model is increased;

[0026] When it is determined that the task demand is a rapid response task, the number of layers of the LSTM network layer in the fault diagnosis model is reduced and the number of neurons of each LSTM network layer is reduced;

[0027] When it is determined that the real-time flight environment information is a complex environment and the task demand is a rapid response task, the number of layers of the LSTM network layer in the fault diagnosis model is reduced and the number of neurons of each LSTM network layer is reduced.

[0028] Further, the diagnosis method further includes fault warning, and specifically includes:

[0029] According to the fault diagnosis result, it is determined whether to generate a fault warning instruction, if yes, the fault warning instruction is generated, and the fault warning instruction is sent to the flight control system, so that the flight control system performs fault processing according to the fault warning instruction.

[0030] Based on the same concept, the present application also provides a fault diagnosis device for an unmanned aerial vehicle flight control system, including:

[0031] The first construction module is used to construct a training data set; wherein each training sample in the training data set includes an input vector generated according to flight data and a fault label;

[0032] The second construction module is used to construct a deep learning model integrating LSTM and attention mechanism;

[0033] The first training module is used to train the deep learning model by using the training data set, so as to obtain a fault diagnosis model;

[0034] a judgment module configured to determine whether the fault diagnosis model needs to be adjusted according to real-time flight environment information and / or task requirements;

[0035] an adjustment module configured to dynamically adjust the structure and parameters of the fault diagnosis model when it is determined that the fault diagnosis model needs to be adjusted according to real-time flight environment information and / or task requirements;

[0036] a second training module configured to retrain / retune the fault diagnosis model after dynamic adjustment;

[0037] a diagnosis module configured to perform fault diagnosis on real-time flight data by using the fault diagnosis model before adjustment or the fault diagnosis model after retraining / retuning.

[0038] Based on the same concept, the present application also provides an electronic device comprising a memory, a processor and a computer program / instruction stored in the memory, wherein the processor executes the computer program / instruction to implement the UAV flight control system fault diagnosis method as described above.

[0039] Based on the same concept, the present application also provides a computer readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction is executed by a processor to implement the UAV flight control system fault diagnosis method as described above.

[0040] Compared with the prior art, the present application has the following advantages:

[0041] In the UAV flight control system fault diagnosis method provided by the present application, the fault diagnosis model is a deep learning model based on LSTM architecture and fusion attention mechanism, LSTM can effectively process the time sequence characteristics of flight data, and solve the gradient vanishing or explosion problem of traditional recurrent neural network; the attention mechanism assigns weights by calculating the similarity of data features, so that the model focuses on key information for fault diagnosis; compared with traditional diagnosis models based on sensor data, the diagnosis accuracy for hidden faults and complex faults is significantly improved.

[0042] The present application designs a mechanism for real-time monitoring of flight environment information and flight task requirements, and dynamically adjusts the structure and parameters of the deep learning model accordingly: in complex environment, the number of LSTM network layers is increased and the parameters of the model are adjusted to strengthen feature learning, so that the model adapts to different flight conditions, breaks through the limitations of traditional fixed model structure and parameters, and improves the diagnosis accuracy; in fast response task, the model structure is simplified, and the real-time diagnosis is improved; and compared with complex environment, fast response task has higher priority, which ensures flight safety.

[0043] The application also triggers a fault warning and matches a fault handling strategy library according to the fault diagnosis result, early warns potential faults, and guides the unmanned aerial vehicle to take emergency measures; compared with the prior art which can only diagnose faults that have occurred, the application fills the blank of fault prediction, and enhances the flight safety and reliability of the unmanned aerial vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the following embodiment description will be briefly introduced. Obviously, the drawings in the following description are only one embodiment of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0045] Figure 1 is a flow chart of the fault diagnosis method of the unmanned aerial vehicle flight control system in the embodiment of the present application;

[0046] Figure 2 is a structural block diagram of the fault diagnosis device of the unmanned aerial vehicle flight control system in the embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the present application will be described clearly and completely in combination with the drawings in the embodiment of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the present application.

[0048] The technical solutions of the present application will be described in detail in combination with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in some embodiments.

[0049] Embodiment one

[0050] The traditional unmanned aerial vehicle flight control system fault diagnosis method based on sensor data relies on statistical methods or traditional machine learning algorithms to construct data models. When the flight environment changes complexly, such as electromagnetic interference enhancement and severe weather fluctuation, it is difficult to adaptively adjust the diagnosis strategy, and it is difficult to accurately capture the data feature change, resulting in high error rate of hidden fault diagnosis, and unable to meet the safety flight requirements of the unmanned aerial vehicle. Although the model-based fault diagnosis method attempts to diagnose faults through accurate mathematical models, the complexity of the unmanned aerial vehicle system makes the model construction cost high, and the parameters are difficult to accurately determine. In practical application, not only a large amount of resources are consumed, but also due to the deviation between the model and the actual system, the accuracy of fault diagnosis is reduced.

[0051] Most of the existing diagnosis methods based on deep learning adopt fixed model structure and parameters, which cannot respond and adjust in time when facing different flight task requirements, such as high-altitude high-speed flight and low-altitude complex terrain reconnaissance, resulting in slow fault diagnosis reaction speed and difficulty in meeting the real-time requirements of unmanned aerial vehicles. At the same time, these methods generally lack the ability to predict faults, and cannot provide early warning before the fault occurs, resulting in hidden dangers in the flight safety of unmanned aerial vehicles.

[0052] To solve the above technical problems, the present application provides a fault diagnosis method for unmanned aerial vehicle flight control system based on fusion of LSTM and attention mechanism, which fundamentally breaks through the limitations of traditional methods. The present application adopts a data-driven approach without relying on accurate mathematical models. In the actual flight process of the unmanned aerial vehicle, the flight control system can continuously collect a large amount of flight data, which contains rich information under normal operation and various fault states. The deep learning model takes these raw data as the learning object, automatically mines the dynamic characteristics and long-term dependencies in the data that change over time, such as capturing the evolution law of signals such as unmanned aerial vehicle attitude change and motor speed fluctuation in time series, through the powerful time series modeling capability of LSTM. Combined with the attention mechanism, the deep learning model can adaptively focus on the key features related to faults in the data, and can accurately identify the subtle changes of fault signals even in complex interference environments. This automatic feature extraction and pattern recognition capability avoids the tedious process of manually constructing mathematical models and the difficulty of parameter determination in traditional methods, and reduces the diagnostic errors caused by model simplification or inaccurate parameters.

[0053] At the same time, the deep learning model learns through a large number of diverse flight data samples during the training process, which can effectively adapt to the data feature distribution under different flight scenarios, environmental conditions and equipment states, and has stronger generalization ability. Whether in high-altitude low-temperature environment or in complex areas with strong electromagnetic interference, the model can accurately determine whether the unmanned aerial vehicle flight control system has a fault and the fault type based on the learned feature patterns, significantly improving the applicability of the fault diagnosis method in actual complex systems and providing reliable protection for the safe and stable operation of unmanned aerial vehicles.

[0054] The present application innovatively designs an adaptive adjustment mechanism to monitor the flight environment and task requirement information in real time during the flight of the unmanned aerial vehicle, and dynamically adjusts the structure and parameters of the deep learning model according to these changes, so that the model can always maintain good adaptability to different flight conditions, significantly improving the real-time performance and accuracy of fault diagnosis.

[0055] Figure 1 The present application provides a fault diagnosis method for unmanned aerial vehicle flight control system. As shown in Figure 1 The fault diagnosis method for unmanned aerial vehicle flight control system includes the following steps:

[0056] Step 1: Constructing a training dataset.

[0057] Each training sample in the training dataset includes an input vector generated according to flight data and a fault label. In the specific embodiments of the present application, the training dataset is constructed, including:

[0058] Step 1.1: Obtain flight data of the flight control system, and label each piece of flight data to obtain the corresponding fault label.

[0059] There are various ways to obtain flight data, including semi-physical simulation, physical fault injection experiment, and actual flight record. The present application can select one or more of them according to the actual needs of the training dataset to obtain the required flight data.

[0060] The process of obtaining flight data through semi-physical simulation is as follows: simulate various flight environments and working conditions of the unmanned aerial vehicle through a flight control system semi-physical simulation platform (such as Simulink), and define a fault injection module to inject faults from multiple levels such as sensors, actuators, systems / structures. Specifically, it includes:

[0061] Sensor fault: achieved by tampering with CAN bus message data sent to the flight control system, such as artificially increasing the angular velocity / acceleration noise or drift of the inertial measurement unit (IMU), interfering with the barometric altitude data, etc.

[0062] Actuator fault: achieved by tampering with the dynamics model input, such as simulating the efficiency of a certain motor decreasing to a certain value, rudder sticking or reverse effect, etc.

[0063] System / structure fault: achieved by modifying the unmanned aerial vehicle dynamics model parameters, such as simulating the dramatic change of the center of gravity after dropping the cargo, injecting extreme wind shear or sudden wind model, simulating the aerodynamic damage such as broken arm, etc. This method can safely and unlimitedly generate data of extreme and dangerous fault scenarios, the fault label is accurate and the scenario is reproducible, suitable for large-scale initial dataset construction, but cannot completely restore the physical details in the real environment (such as complex electromagnetic interference, non-ideal aerodynamic effects, etc.).

[0064] The process of obtaining flight data through physical fault injection experiment is as follows: by building a real flight control system fault simulation experiment platform, in a controlled real physical environment, artificially introducing faults (such as hardware interference, component damage or environmental manipulation) to the actual unmanned aerial vehicle, and collecting all the sensor data (including IMU three-axis angular velocity / acceleration, barometric altitude, device current / voltage, etc.) throughout the process. This method captures all the physical phenomena and noise in the real scene, and has high practical value, but the cost is high and the data size is limited.

[0065] The specific process of acquiring flight data by the actual flight record is as follows: the data acquisition module is connected with the flight control system of the unmanned aerial vehicle performing the flight task through the CAN bus, and the original data of various sensors in the flight control system are directly acquired by using the data acquisition module, for example, three-axis angular velocity and three-axis acceleration output by an inertial measurement unit (IMU), height data detected by an air pressure sensor, current intensity of a battery, a motor and other devices detected by a current sensor, and voltage of the battery, the motor and other devices detected by a voltage sensor. The original flight data sequence is formed from the original data of various sensors, and each flight data corresponds to an original flight data sequence.

[0066] According to the fault state corresponding to each flight data, the corresponding original flight data sequence is labeled, and the labeling content includes a fault type and a fault probability. The fault type includes a normal state and different fault states, and the fault probability has a value of 0 or 1. Therefore, the fault label includes the fault probability of each fault type, and the fault label of the flight data in the normal state can be represented as [1, 0, 0, 0, …, 0], wherein 1 represents that the probability of the flight data being in the normal state is 1, and 0 represents that the probability of the flight data being in different fault states is 0; the fault label of the flight data in the fault state A can be represented as [0, 1, 0, 0, …, 0], wherein the first 0 represents that the probability of the flight data being in the normal state is 0 (i.e., being in the fault state), the first 1 represents that the probability of the flight data being in the fault state A is 1, and the 0 after the first 1 represents that the probability of the flight data being in other fault states (fault states other than the fault state A) is 0.

[0067] Step 1.2: The flight data is denoised by using the Kalman filtering algorithm.

[0068] In order to suppress sensor noise and interference, smooth data and enhance signal consistency, the flight data (original flight data sequence) is denoised by using the Kalman filtering algorithm in the present application, so as to provide high-quality and low-noise time sequence input for the subsequent deep learning model, thereby improving the accuracy and robustness of fault diagnosis.

[0069] Step 1.3: The flight data after denoising is normalized to generate an input vector corresponding to each flight data.

[0070] In the specific embodiments of the present application, the minimum-maximum normalization method is used to normalize the flight data after denoising, so that each element in the normalized sequence is mapped to the interval [0, 1] to generate an input vector.

[0071] In the data acquisition and preprocessing stage (denoising and normalization), industrial-grade data acquisition cards and high-speed sampling technology are used, and the Kalman filtering and the minimum-maximum normalization method are combined to ensure data accuracy and model training efficiency.

[0072] Step 2: Construct a deep learning model combining LSTM and attention mechanism.

[0073] LSTM has strong time series modeling capability, which can automatically mine dynamic features and long-term dependencies in data over time, such as capturing the evolution of signals such as UAV attitude changes, motor speed fluctuations, etc. in time series; attention mechanism can adaptively focus on key features related to faults in data, even in complex interference environment, it can accurately identify subtle changes in fault signals. Therefore, the present application constructs a deep learning model combining LSTM and attention mechanism. Specifically, the deep learning model includes LSTM network layer, attention mechanism layer and full connection layer, the number of LSTM network layer is at least 1.

[0074] Step 3: Train the deep learning model using the training data set to obtain the fault diagnosis model.

[0075] The training of the deep learning model using the training data set specifically includes:

[0076] Step 3.1: input the input vector in the training data set into the deep learning model.

[0077] Step 3.2: use the LSTM network layer to extract the time series dependency of the input vector.

[0078] Step 3.3: based on the time series dependency extracted by the LSTM network layer, use the attention mechanism layer to calculate the attention output to focus on the key fault features.

[0079] Step 3.4: based on the attention output output, use the full connection layer to obtain the probability distribution of the fault type, the specific formula is:

[0080] P=softmax(W fc ·output+b fc )(1)

[0081] P=[p1,p2,...,p m ](2)

[0082] Where, P represents the probability distribution of the fault type; W fc represents the weight parameter of the full connection layer; b fc represents the bias of the full connection layer; m represents the number of fault types, p m represents the probability of the mth fault type.

[0083] Step 3.5: calculate the cross-entropy loss according to the probability distribution of the fault type and the corresponding fault label.

[0084] Step 3.6: Adjust the parameters of the LSTM network layer and the parameters of the attention mechanism layer in the deep learning model according to the cross-entropy loss to realize model training.

[0085] By continuously adjusting the model parameters, the loss function value is reduced, and the accuracy of the model fault diagnosis is improved. In the model training process, the Adam optimization algorithm is used to calculate the gradient of the loss function to the parameters:

[0086]

[0087] W' = W + AW (4)

[0088] Wherein, W' represents the adjusted weight parameter; W represents the weight parameter before adjustment; AW represents the adjustment amount of the weight parameter; L represents the loss function; η represents the learning rate.

[0089] In this embodiment, the acquisition mode of flight data includes semi-physical simulation, physical fault injection experiment and actual flight record. The training data set includes a first data set, a second data set, a third data set and a fourth data set, the first data set is constructed by using semi-physical simulation, the second data set is constructed by using physical fault injection experiment, and the third data set is constructed by using actual flight record.

[0090] The training process includes: training the deep learning model using the first data set to obtain an initial fault diagnosis model; verifying and fine-tuning the initial fault diagnosis model using the second data set to obtain an intermediate fault diagnosis model, and improving the generalization ability of the model in the actual scene; by analyzing the performance difference of the simulation model in the semi-physical simulation mode on the second data set, adjusting the simulation model (such as adjusting the noise model, the aerodynamic model, etc.), so that the simulation model is closer to the real one; using the adjusted simulation model to obtain the fourth data set, using the fourth data set to train the intermediate fault diagnosis model, and using the third data set to verify the trained intermediate fault diagnosis model, that is, obtaining the final fault diagnosis model. The process of training the model using the first data set or the fourth data set is as steps 3.1 to 3.6.

[0091] In the specific embodiments of the present application, in step 3.3, based on the time sequence dependence relationship extracted by the LSTM network layer, the attention output is calculated by using the attention mechanism layer, including:

[0092] Step 3.31: convert the hidden state h t output by the LSTM network layer into query vector Q, key vector K and value vector V by linear transformation:

[0093] Q = W Q h t , K = W K h t, V = W V h t (5)

[0094] wherein, W Q , W K , W V respectively represent the corresponding weight parameters.

[0095] Step 3.32: Calculate the attention score score according to the query vector, key vector and value vector:

[0096]

[0097] wherein, d k represents the dimension of the key vector K, and the attention score reflects the similarity between the query vector and each key vector.

[0098] Step 3.33: Calculate the attention output output according to the attention score and the value vector:

[0099] α = softmax(score), output = α · V (7)

[0100] wherein, α represents the attention weight of the value vector, indicating the attention degree of the model to different information; multiplying the attention weight with the value vector, the attention output output after weighted summation is obtained, focusing on the key information of fault diagnosis.

[0101] Through the memory cell and gating structure unique to LSTM, time series data in the flight process of the unmanned aerial vehicle can be effectively processed, the evolution law of fault characteristics at different times can be accurately captured, and diagnostic errors caused by long data sequences and easy loss of features can be avoided; after introducing the attention mechanism, the system can adaptively focus on key fault features; compared with traditional methods, the weights of different data features can be dynamically allocated, and the response ability of the model to complex fault scenarios can be enhanced; the combination of LSTM and attention mechanism not only improves the accuracy of fault diagnosis, but also greatly shortens the diagnosis time, so that the system can quickly respond when facing sudden faults, and can maintain stable and efficient diagnostic performance under different flight conditions, providing more reliable protection for the safe operation of the unmanned aerial vehicle flight control system.

[0102] During model training, a large-scale data set is constructed and scientifically divided into training, validation and test sets, Adam algorithm is used to optimize the cross-entropy loss function, the generalization ability of the model is improved, a complete and efficient data-driven fault diagnosis process is formed, which is different from traditional single diagnostic methods.

[0103] Step 4: determining whether the fault diagnosis model needs to be adjusted according to the real-time flight environment information and / or task requirements, if yes, then dynamically adjusting the structure and parameters of the fault diagnosis model, and then turning to step 5; if no, then turning to step 6 based on the unadjusted fault diagnosis model.

[0104] In the specific embodiment of the present application, dynamically adjusting the structure and parameters of the fault diagnosis model comprises:

[0105] when it is determined to be a complex environment according to the real-time flight environment information, increasing the number of LSTM network layers in the fault diagnosis model;

[0106] when it is determined to be a fast response task according to the task requirements, reducing the number of LSTM network layers in the fault diagnosis model and reducing the number of neurons in each LSTM network layer;

[0107] when it is determined to be a complex environment according to the real-time flight environment information and it is determined to be a fast response task according to the task requirements, reducing the number of LSTM network layers in the fault diagnosis model and reducing the number of neurons in each LSTM network layer.

[0108] In the present embodiment, the flight environment information is at least one of meteorological data (such as wind speed) and geographic information (such as terrain height). When the flight environment information exceeds the corresponding preset threshold, it is determined to be a complex environment, and the number of LSTM network layers is increased. The task requirements are at least one of emergency tasks such as emergency landing instructions. When the task requirements are emergency tasks, the number of LSTM network layers is reduced and the number of neurons in each LSTM network layer is reduced.

[0109] When the real-time flight environment information is a non-complex environment and the task requirements are non-emergency tasks, turn to step 6, i.e. use the fault diagnosis model output in step 3 to perform fault diagnosis on the real-time flight data.

[0110] Step 5: retraining / fine-tuning the dynamically adjusted fault diagnosis model.

[0111] When the number of LSTM network layers is increased, the dynamically adjusted fault diagnosis model is retrained using a training data set; when the number of LSTM network layers is reduced and the number of neurons in each LSTM network layer is reduced, the dynamically adjusted fault diagnosis model is fine-tuned using flight data at n time points before the current time point and corresponding fault labels.

[0112] Step 6: using the unadjusted fault diagnosis model or the retrained / fine-tuned fault diagnosis model to perform fault diagnosis on the real-time flight data.

[0113] The unadjusted fault diagnosis model or the retrained / fine-tuned fault diagnosis model is used to perform fault diagnosis on real-time flight data to obtain fault diagnosis results. The fault diagnosis results are probabilities of the real-time flight data being different fault types. When the probability of a certain fault type is greater than a set fault probability threshold, it is determined that the real-time flight data has occurred the fault of the type. In the embodiment, the fault probability threshold is 95%.

[0114] Step 7: According to the fault diagnosis result, it is judged whether to generate a fault warning instruction. If yes, a fault warning instruction is generated, and the fault warning instruction is sent to the flight control system, so that the flight control system performs fault processing according to the fault warning instruction.

[0115] When at least one probability in the fault diagnosis result is greater than a set warning probability threshold, a fault warning instruction is generated, including the fault type, the severity and the occurrence time. When all probabilities in the fault diagnosis result are less than or equal to the set warning probability threshold, no fault warning instruction is generated, and the monitoring state is continued. The warning probability threshold is less than the fault probability threshold. In the embodiment, the warning probability threshold is set to 80%.

[0116] The fault warning instruction is sent to the flight control system through the CAN bus. The flight control system determines the type of the warning fault according to the fault warning instruction, and then matches the fault processing strategy from the fault processing strategy library according to the fault type, and executes the fault processing strategy to realize fault processing. The flight control system feeds back the fault processing result to the flight control system fault diagnosis device through the CAN bus, and the flight control system fault diagnosis device sends the fault processing result to the ground station through the 4G / 5G communication module or the data radio station, so as to facilitate the ground operator to master the status of the unmanned aerial vehicle and make remote decisions.

[0117] The present application outputs fault prediction probability while outputting fault diagnosis result of the deep learning model, and triggers warning and matches fault processing strategy library through the set probability threshold, so as to early warn potential faults and guide the unmanned aerial vehicle to take emergency measures. Compared with the existing technology which can only diagnose the occurred faults, the present application fills the blank of fault prediction and enhances the flight safety and reliability of the unmanned aerial vehicle.

[0118] Embodiment two

[0119] As shown in Figure 2 The present application also provides an unmanned aerial vehicle flight control system fault diagnosis device, which comprises a core processing module. The core processing module comprises a first construction module, a second construction module, a first training module, a judgment module, an adjustment module, a second training module and a diagnosis module.

[0120] The first construction module is configured to construct a training data set, wherein each training sample in the training data set comprises an input vector generated according to flight data and a fault label; the second construction module is configured to construct a deep learning model integrating LSTM and an attention mechanism; the first training module is configured to train the deep learning model by using the training data set to obtain a fault diagnosis model; the judgment module is configured to judge whether the fault diagnosis model needs to be adjusted according to real-time flight environment information and / or task requirements; the adjustment module is configured to dynamically adjust the structure and parameters of the fault diagnosis model when it is judged that the fault diagnosis model needs to be adjusted according to the real-time flight environment information and / or the task requirements; the second training module is configured to retrain / retune the fault diagnosis model after dynamic adjustment; and the diagnosis module is configured to perform fault diagnosis on real-time flight data by using the fault diagnosis model before adjustment or the fault diagnosis model after retraining / retuning.

[0121] The fault diagnosis device further comprises a data acquisition module connected with the flight control system through a CAN bus, the data acquisition module being configured to construct a training data set according to original data of each sensor sent by the flight control system and to acquire real-time flight data of the flight control system. In addition, the data acquisition module is internally provided with a high-speed data cache chip CY7C1049CV33, which caches the original data of each sensor and the real-time flight data acquired from the flight control system, ensures the continuity and stability of data acquisition, avoids data loss or errors caused by data transmission delay or fluctuation, and provides stable data input for the core processing module.

[0122] The fault diagnosis device further comprises a data storage module connected with the core processing module. The data storage module is configured with a 512GB M.2 interface SSD for storing flight data, a trained deep learning model and a fault diagnosis log. The flight data can be used to construct a training data set, and the fault diagnosis log records detailed information of each diagnosis, facilitating fault review and system maintenance. A 256KB NVRAM chip of Microchip Company is adopted to automatically save key intermediate data and configuration information at the moment of system power failure, such as the parameters of the currently running fault diagnosis model, the status of the unfinished fault diagnosis task, etc., so as to ensure that the system can quickly recover to the working state after restarting and does not affect the continuity of fault diagnosis.

[0123] The fault diagnosis device further comprises a communication module connected with the core processing module. The communication module adopts CAN bus communication and uses a double-channel CAN bus interface. The main channel is used to acquire sensor data and real-time flight data from the flight control system, and the second channel is used to send fault warning instructions to the flight control system and to receive fault processing feedback information. The communication protocol uses a self-defined efficient protocol to compress and check the data, thereby improving the data transmission efficiency and accuracy.

[0124] The fault diagnosis device further comprises a power management module for providing power supply for each module.

[0125] The core processing module selects a high-performance NVIDIA Jetson AGX Xavier development board, which is based on an ARM architecture, is equipped with a 6-core NVIDIA Carmel ARM 64-bit CPU and a 384-CUDA-core Volta architecture GPU, has strong parallel computing and deep learning inference capabilities, and can meet the complex data processing and model operation requirements in the fault diagnosis algorithm.

[0126] In some specific embodiments of the present application, the UAV flight control system fault diagnosis device can combine the features of the UAV flight control system fault diagnosis method in Embodiment One of the present application, and vice versa, which will not be described here.

[0127] The UAV flight control system fault diagnosis device is used to run the UAV flight control system fault diagnosis method integrating LSTM and attention mechanism, realize the collection of various types of data of the UAV flight control system, complete fault diagnosis, output the fault diagnosis result to the flight control system for fault processing, and simultaneously issue the fault processing information to the UAV ground station to remind the ground personnel to flexibly handle different fault conditions.

[0128] Embodiment Three

[0129] The electronic device comprises a memory, a processor and a computer program / instruction stored in the memory, and the processor executes the computer program / instruction to realize the UAV flight control system fault diagnosis method in the embodiments of the present application.

[0130] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to programs and / or data stored in a read-only memory (ROM) or programs and / or data loaded from a storage section into a random access memory (RAM). The processor can be one multi-core processor or can include a plurality of processors. In some embodiments, the processor can include a general-purpose main processor and one or more special-purpose co-processors, such as a central processing unit, a graphics processing unit (GPU), a neural processing unit (NPU), a digital signal processor (DSP), and the like. In the RAM, various programs and data required for device operations are also stored. The processor, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0131] The above processor and memory are used together to execute programs / instructions stored in the memory, which, when executed by a computer, can implement the methods, steps, or functions described in the above embodiments.

[0132] Although not shown, the embodiments of the present application also provide a computer-readable storage medium having stored thereon computer programs / instructions, which, when executed by a processor, implement the unmanned aerial vehicle flight control system fault diagnosis method in the embodiments of the present application.

[0133] The readable storage medium includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0134] The above only discloses specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or modifications within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for diagnosing faults in a UAV flight control system, characterized in that, The diagnostic method includes: Construct a training dataset; wherein each training sample in the training dataset includes an input vector and a fault label generated from flight data; Construct a deep learning model that integrates LSTM and attention mechanisms; The deep learning model is trained using the training dataset to obtain a fault diagnosis model; Determine whether the fault diagnosis model needs adjustment based on real-time flight environment information and / or mission requirements. If so, dynamically adjust the structure and parameters of the fault diagnosis model; retrain / fine-tune the dynamically adjusted fault diagnosis model; and use the retrained / fine-tune fault diagnosis model to diagnose faults in real-time flight data. If not, then the unadjusted fault diagnosis model is used to diagnose faults in real-time flight data.

2. The method for fault diagnosis of UAV flight control system according to claim 1, characterized in that, The construction of the training dataset includes: Acquire flight data from the flight control system and label each data point to obtain the corresponding fault label; The flight data is denoised using the Kalman filter algorithm. The denoised flight data is normalized to generate an input vector corresponding to each flight data point.

3. The method for fault diagnosis of UAV flight control system according to claim 1, characterized in that, The deep learning model includes at least one LSTM network layer, an attention mechanism layer, and a fully connected layer; The LSTM network layer is used to extract the temporal dependencies of flight data; The attention mechanism layer is used to calculate the attention output based on the temporal dependencies extracted from the LSTM network layer, so as to focus on key fault features; The fully connected layer is used to obtain the probability distribution of fault types based on the attention output.

4. The method for fault diagnosis of UAV flight control system according to claim 3, characterized in that, The attention output is calculated based on the temporal dependencies extracted from the LSTM network layers, specifically including: The hidden state output by the LSTM network layer is converted into a query vector, a key vector, and a value vector through a linear transformation. Calculate the attention score based on the query vector, key vector, and value vector; The attention output is calculated based on the attention score and value vector.

5. The method for fault diagnosis of UAV flight control system according to claim 1, characterized in that, The training dataset includes a first dataset, a second dataset, a third dataset, and a fourth dataset; the first dataset is constructed based on a hardware-in-the-loop simulation method, the second dataset is constructed based on a physical fault injection experiment, and the third dataset is constructed based on actual flight record data; Training the deep learning model using the training dataset includes: The deep learning model is trained using the first dataset to obtain an initial fault diagnosis model; The initial fault diagnosis model was validated and fine-tuned using the second dataset to obtain an intermediate fault diagnosis model; Based on the performance evaluation of the intermediate fault diagnosis model on the second dataset, adjust the simulation model in the hardware-in-the-loop simulation method. A fourth dataset was constructed based on the adjusted simulation model; The intermediate fault diagnosis model is trained using the fourth dataset, and the trained intermediate fault diagnosis model is validated using the third dataset to obtain the final fault diagnosis model.

6. The method for fault diagnosis of UAV flight control system according to claim 1, characterized in that, Dynamically adjusting the structure and parameters of the fault diagnosis model includes: When the real-time flight environment information indicates a complex environment, the number of LSTM network layers in the fault diagnosis model is increased. When the task requirement is determined to be a rapid response task, the number of LSTM network layers in the fault diagnosis model is reduced and the number of neurons in each LSTM network layer is reduced. When the real-time flight environment information indicates a complex environment and the mission requirements indicate a rapid response mission, the number of LSTM network layers in the fault diagnosis model is reduced, and the number of neurons in each LSTM network layer is also reduced.

7. The method for diagnosing faults in a UAV flight control system according to any one of claims 1 to 6, characterized in that, The diagnostic method also includes fault early warning, specifically including: Based on the fault diagnosis results, determine whether to generate a fault warning command. If so, generate a fault warning command and send it to the flight control system so that the flight control system can handle the fault according to the fault warning command.

8. A fault diagnosis device for a UAV flight control system, characterized in that, The diagnostic device includes: The first building module is used to build a training dataset; wherein each training sample in the training dataset includes an input vector and a fault label generated from flight data; The second building block is used to build a deep learning model that integrates LSTM and attention mechanisms; The first training module is used to train the deep learning model using the training dataset to obtain a fault diagnosis model. The judgment module is used to determine whether the fault diagnosis model needs to be adjusted based on real-time flight environment information and / or mission requirements. The adjustment module is used to dynamically adjust the structure and parameters of the fault diagnosis model when it is determined that the fault diagnosis model needs to be adjusted based on real-time flight environment information and / or mission requirements. The second training module is used to retrain / fine-tune the dynamically adjusted fault diagnosis model. The diagnostic module is used to diagnose faults in real-time flight data using an unadjusted fault diagnosis model or a retrained / fine-tuned fault diagnosis model.

9. An electronic device comprising a memory, a processor, and a computer program / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the UAV flight control system fault diagnosis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the fault diagnosis method for the UAV flight control system as described in any one of claims 1 to 7.

Images