Unmanned aerial vehicle airborne real-time fault online detection method and system based on deep learning

Abstract

This invention provides a deep learning-based method for real-time online fault detection on UAVs. The method first constructs a high-fidelity simulation environment, systematically injects sensor and actuator faults, and generates a large-scale multi-condition labeled dataset. Based on this, a domain adaptation technique is used to train an initial model, and a lightweight diagnostic network based on temporal-graph fusion is designed and deployed on the airborne end. Transformers and graph convolutional networks are used to extract temporal features and sensor topological features, respectively, and probabilistic online diagnosis is achieved through flow normalization. Furthermore, an edge-cloud collaborative optimization mechanism is established: low-confidence samples are selected and uploaded at the edge, while the cloud evaluates the sample value based on information entropy and diversity, uses knowledge distillation for incremental learning, and achieves hot model updates via OTA. This invention achieves high accuracy, low latency, and continuous adaptive optimization in fault detection.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) power line inspection technology, specifically to a UAV-based real-time online fault detection method based on deep learning. Background Technology

[0002] As the power system transforms and upgrades towards a smart grid, the scale of unmanned substations continues to expand. Traditional manual inspection methods can no longer meet the demands for high-frequency, all-weather, and high-precision equipment status monitoring. Unmanned aerial vehicle (UAV) autonomous inspection technology, with its flexibility and efficiency, has become an important means of intelligent operation and maintenance of substations, and is widely used in scenarios such as high-voltage live equipment inspection and hazardous area surveying.

[0003] However, the substation environment poses a serious challenge to the safe and reliable flight of drones: Environmental complexity: Strong electromagnetic interference can easily cause abnormal data from sensors (such as GPS and magnetic compasses); dense equipment layout creates narrow flight corridors, requiring centimeter-level positioning accuracy; sudden gusts and other aerodynamic disturbances increase the difficulty of flight control and stability.

[0004] Mission complexity: The refined inspection mission involves frequent switching between various flight conditions such as hovering, constant speed cruise, and close maneuvering, which places extremely high demands on the dynamic performance and robustness of the flight control system.

[0005] Multiple risks of failure: Under the complex environment and mission background mentioned above, UAVs are highly susceptible to multiple risks such as sensor failure, actuator performance degradation, and environmental disturbances. If the failure is not detected and dealt with in time, it may lead to mission failure, equipment damage, or even serious safety accidents such as collision with high-voltage equipment.

[0006] Currently, the technologies applied to UAV fault detection have the following limitations: Physical model-based methods rely on accurate UAV dynamics and fault propagation models. However, in the complex electromagnetic and aerodynamic coupling environment of substations, it is difficult to establish accurate analytical models. This results in the residual generator being sensitive to unmodeled dynamics, leading to a high false alarm rate. Furthermore, the model parameters need to be calibrated individually for each UAV, making engineering implementation complex.

[0007] Traditional statistical analysis methods (such as PCA and KPCA) do not require precise modeling and have low computational complexity. However, they are difficult to fully extract deep temporal features and coupling patterns from multi-source heterogeneous sensor data, and fixed fault judgment thresholds cannot adapt to changes in the normal state boundary under different flight conditions, which can easily lead to false alarms or missed alarms when switching between flight conditions.

[0008] Offline training methods based on deep learning, while capable of automatically learning complex features, heavily rely on large amounts of labeled data. In real-world substation scenarios, fault data is extremely scarce, and for safety reasons, faults cannot be actively injected for collection. This leads to problems such as insufficient training samples and imbalanced samples, limiting the model's generalization ability. Furthermore, complex models have a large number of parameters, making it difficult to achieve fast real-time inference on airborne embedded platforms.

[0009] Ground station-based remote diagnostics methods transmit data to the cloud for processing. While this method can utilize powerful computing capabilities, it is limited by the latency, unstable bandwidth, and risk of link interruption in wireless communication. This makes it difficult to meet the real-time requirements of safety-critical functions, and the continuous data transmission consumes a lot of energy, affecting the drone's endurance.

[0010] In summary, existing technologies struggle to achieve highly reliable, adaptive, real-time fault detection by UAVs used for substation inspections under constraints such as scarce fault data, discrepancies between simulated and real-world environments, limited onboard computing power, and the need for high real-time performance. Therefore, there is an urgent need to develop a novel intelligent fault detection method that can comprehensively utilize simulation data, possess lightweight onboard inference capabilities, and continuously evolve online. Summary of the Invention

[0011] The purpose of this invention is to provide a real-time online fault detection method for UAVs based on deep learning, which solves the problems of scarce real fault data, differences in distribution between simulation and real environment, limited onboard computing resources, and inability of models to adaptively optimize online in substation inspection scenarios.

[0012] To achieve the above objectives, the present invention adopts the following technical solution: A deep learning-based method for real-time online fault detection on UAVs includes the following steps: S1. Establish a simulation environment for substation inspection scenarios, and define and execute various flight conditions covering hovering, constant speed cruise, and close-range maneuvering in the simulation environment; S2. Inject sensor failure modes and actuator failure modes into the simulation environment to generate a large-scale labeled multi-condition fault dataset. S3. Based on the multi-condition fault dataset, an initial fault diagnosis model is trained using domain adaptation technology, and the fault diagnosis model is deployed on the UAV's onboard computing unit. S4. During the real-time flight of the UAV, collect sensor time-series data and input it into the fault diagnosis model. Extract and fuse time-series features and topological features through the fault diagnosis model, perform probability modeling through flow normalization, and output the fault diagnosis result and confidence level of the current state. S5. Based on the confidence level or fault detection results output by the model, the edge device of the drone filters out sample data of suspected faults and uploads them to the cloud. S6. The cloud performs value assessment on the sample data based on information entropy and sample diversity measurement, selects samples with values ​​higher than a preset value threshold for labeling, and uses knowledge distillation technology to perform incremental learning of the model and update the model parameters. S7. The updated model in the cloud is wirelessly distributed to the edge of the drone to replace the original model, realizing hot updates and continuous optimization of the model.

[0013] Furthermore, the simulation environment for the substation inspection scenario includes a three-dimensional model of the substation equipment layout, a six-degree-of-freedom dynamic model of the UAV, and an environmental model characterizing electromagnetic interference and aerodynamic disturbances.

[0014] Furthermore, the sensor failure modes include deviation failure, drift failure, and lock-up failure; the actuator failure modes simulate motor performance degradation through an efficiency matrix.

[0015] Furthermore, the fault diagnosis model adopts a dual-branch fusion architecture of temporal feature extraction branch and topology modeling branch: the temporal feature extraction branch is used to capture the long-range dependencies of multi-source sensor temporal data using the Transformer model; the topology modeling branch is used to mine the coupling correlation between sensors based on a predefined sensor causal topology matrix through a graph convolutional neural network.

[0016] Furthermore, the domain adaptation technique uses maximum mean difference (MMD) as a measure of distributional dissimilarity, and performs joint training by minimizing the weighted sum of task loss and domain adversarial loss. The overall loss function is: ; in: Classify task losses based on fault classification; For domain adversarial loss based on MMD, These are the weighting coefficients; This is the overall loss function.

[0017] Furthermore, the sensor causal topology matrix is ​​obtained through Bayesian network learning, which characterizes the conditional dependencies between different sensor measurements.

[0018] Furthermore, the stream normalization employs real-number invariant stream or masked autoregressive stream to map the input data to the latent space to calculate the log-likelihood, which serves as the probabilistic basis for fault detection.

[0019] Furthermore, the comprehensive score calculation formula for the sample value assessment is as follows: ; in: Information entropy is calculated based on the probability distribution predicted by the model; This is the minimum Euclidean distance between the sample and the existing training set in the feature space; and These are preset weighting coefficients; This is the overall score for assessing the value of the sample.

[0020] Furthermore, the loss function for incremental learning is: ; in: For newly labeled samples Cross-entropy loss; The knowledge distillation loss is used to constrain the new model on old data. The output remains consistent with the original model; Weighting for distillation loss; This is the loss function for incremental learning.

[0021] A deep learning-based real-time online fault detection system for unmanned aerial vehicles (UAVs) is provided to implement the deep learning-based real-time online fault detection method for UAVs. The system includes: The simulation and data generation module is used to build a high-fidelity simulation environment and generate multi-condition fault datasets. The model training and transfer module is used to train and optimize the initial fault diagnosis model using domain adaptation techniques. An airborne real-time diagnostic module, deployed on an embedded platform of an unmanned aerial vehicle, includes the aforementioned time-series-graph fusion fault diagnosis network for online fault detection; The edge-cloud collaborative optimization module includes an edge data acquisition and screening unit located locally on the drone, a cloud-based sample evaluation and model training unit located remotely, and a communication unit for model distribution, which enables continuous incremental learning and hot updates of the model.

[0022] As can be seen from the above technical solutions, the present invention has the following technical advantages compared with the prior art: 1. This invention systematically generates large-scale, multi-condition labeled fault data by constructing a high-fidelity simulation environment, fundamentally overcoming the difficulty of obtaining fault data in real-world scenarios; 2. This invention introduces Domain Adaptation (MMD) technology to explicitly align the feature distribution of simulation data with that of real data, effectively mitigating the problem of model performance degradation caused by distribution differences; 3. This invention adopts a time-series-graph fusion dual-branch network architecture, simultaneously modeling the dynamic evolution of fault signals and the topological relationships between sensors, significantly improving diagnostic accuracy; the lightweight design of the model ensures fast real-time inference capabilities on embedded platforms; 4. This invention establishes an intelligent sample selection mechanism based on information entropy and diversity, and combines knowledge distillation for incremental learning, thereby achieving closed-loop continuous optimization and hot updating of the model in the real environment, enabling it to adapt to unknown faults and environmental changes. 5. The edge of the system of this invention is responsible for real-time diagnosis to ensure flight safety, while the cloud is responsible for complex calculations and model evolution. The two work together to ensure real-time performance while giving the system long-term learning and adaptability. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating the steps of the real-time online fault detection method of the present invention; Figure 2 This is a flowchart illustrating the time-series-graph fusion fault diagnosis network. Figure 3 A flowchart illustrating the continuous optimization method for edge-cloud collaboration models. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0025] This invention provides a deep learning-based online real-time fault detection system for unmanned aerial vehicles (UAVs), comprising a simulation and data generation module, a model training and transfer module, an airborne real-time diagnostic module, and an edge-cloud collaborative optimization module. The simulation and data generation module constructs a high-fidelity simulation environment and generates multi-condition fault datasets. The model training and transfer module trains and optimizes an initial fault diagnosis model using domain adaptation techniques. The airborne real-time diagnostic module is deployed on an embedded UAV platform and includes the time-series-graph fusion fault diagnosis network for online fault detection. The edge-cloud collaborative optimization module includes an edge data acquisition and filtering unit located locally on the UAV, a remote cloud-based sample evaluation and model training unit, and a communication unit for model distribution, enabling continuous incremental learning and hot updates of the model.

[0026] like Figure 1 The method for real-time online fault detection on UAVs based on deep learning is implemented based on the above system. The method is described below through specific embodiments.

[0027] Example 1: Simulation Environment Construction and Multi-condition Fault Data Generation In this preferred embodiment, the deep learning-based UAV onboard real-time fault online detection system is deployed in a UAV autonomous inspection project at a 500kV substation to ensure the flight safety and mission reliability of the UAV in environments with strong electromagnetic interference and complex airflow.

[0028] To address the core bottleneck of scarce real-world fault data, this embodiment first constructs a high-fidelity simulation platform. Utilizing laser point cloud and real-scene modeling technologies, a 1:1 substation 3D simulation environment is built, encompassing the main transformer, circuit breakers, disconnectors, busbars, and structural framework. No-fly zones and standard inspection routes are defined, while also considering the unique strong electromagnetic field distribution characteristics and complex aerodynamic environment of substations. Within this substation 3D simulation environment: The UAV is described using a six-degree-of-freedom rigid body dynamics model based on the Newton-Euler equations: ; in: This represents the dynamic behavior of the system; The state vector consists of 12 state variables, including position, attitude angle, velocity, and angular velocity. This represents the control input vector, which is the control command of the system actuator. In a quadcopter UAV, it corresponds to the rotational speed command of the four rotors. This represents the environmental disturbance vector, including gust disturbances, electromagnetic interference (EMI), and other unmodeled environmental disturbances; This represents a function describing the nonlinear dynamics of the system. This formula fully describes the dynamic evolution of the UAV under control and external disturbances, and serves as the fundamental mathematical model for constructing high-fidelity simulation environments and designing fault detection algorithms.

[0029] To address the impact of the strong electromagnetic environment of substations on sensors, this preferred embodiment also establishes an electromagnetic interference model, dynamically adjusting the interference intensity based on the distance between the UAV and the high-voltage equipment. ; in: This represents the sensor measurement value in the simulation environment, that is, the sensor output value after the electromagnetic interference model has been added, which is used to simulate the reading of the sensor after being subjected to electromagnetic interference in the real substation environment. It represents the true physical quantity value of the sensor, that is, the theoretical or standard value that the sensor should measure under ideal interference-free conditions, reflecting the actual attitude, position, speed and other states of the UAV; Indicates the intensity of electromagnetic interference; This represents the spatial distance between the drone and high-voltage equipment in a substation (such as transformers and circuit breakers). Through fitting measured data, it was found that electromagnetic noise intensity is inversely proportional to the distance to the equipment. When the drone approaches strong electromagnetic equipment such as transformers, sensors such as GPS and magnetic compasses exhibit significant measurement deviations. By generating sensor data that more closely resembles the real environment, training data containing electromagnetic interference characteristics is provided for subsequent fault diagnosis models, enhancing the model's robustness in real-world conditions.

[0030] Meanwhile, since substation inspection tasks involve various typical flight conditions, the dynamic characteristics and sensor data distribution of UAVs differ significantly under different conditions; this preferred embodiment systematically designs a set of working condition scenarios covering the entire actual inspection process in the three-dimensional simulation environment, mainly including: In the hovering condition (position standard deviation <0.1m), the UAV remains relatively stationary in front of the target equipment for high-resolution image acquisition and detailed equipment inspection. Under this condition, the position and attitude fluctuations are small, but the ability to resist wind disturbances is required to be high.

[0031] In the constant speed cruise mode (speed 1.5m / s ± 0.2m / s), the UAV flies along a predetermined route at a constant speed, which is used for large-scale inspection and equipment survey. Under this mode, the speed remains stable and the attitude angle changes gradually.

[0032] The close-range maneuvering condition (including uniform acceleration, deceleration, and curvilinear motion) involves the UAV approaching a specific device from a distance for inspection, which involves significant changes in speed and attitude. This condition places high demands on the dynamic response performance of the control system.

[0033] Each operating condition defines a standard reference state and an allowable deviation range, laying the foundation for establishing multi-condition control limits. By repeatedly executing these operating condition tasks in the simulation environment, flight data covering the entire operating condition space can be systematically collected.

[0034] Because the normal state boundaries of UAVs differ significantly under different flight conditions, fixed-threshold fault detection methods are prone to false alarms during condition switching. This invention proposes a multi-condition adaptive control limit establishment method based on statistical learning: For each operating condition, a large number of normal flight missions were performed in the simulation environment to collect sensor data and extract statistical features within the sliding time window, including mean, standard deviation, maximum, and minimum values. These features can effectively characterize the normal fluctuation range of sensor data under different operating conditions. ; in: This represents the extracted multidimensional statistical feature vector, used to characterize the state of the current data window; This indicates that the feature extraction function is applied at time points. A matrix of raw time-series data from multiple sensors acquired within a sliding time window; Represents the mean vector; Represents the standard deviation vector; Represents the vector of maximum values; This represents the vector of minimum values.

[0035] Furthermore, using the kernel density estimation method, a probability distribution model of the normal state is established for each operating condition: ; in: Indicates the condition Multidimensional statistical eigenvectors The probability of occurrence; This represents the confidence level threshold. When the statistical characteristics of the actual flight data fall within the high probability region, it is judged as a normal state; when the probability is lower than the set threshold, a fault alarm is triggered. This probability-based judgment method is more robust than a hard threshold and can adapt to normal fluctuations within the operating conditions.

[0036] Furthermore, in the aforementioned simulation platform, various fault modes can be flexibly injected into sensors and actuators through software configuration without worrying about damage to real equipment or safety risks.

[0037] This preferred embodiment design covers the injection scheme for the most common sensor fault types in practical applications, including deviation faults, drift faults, and lock-up faults.

[0038] For the aforementioned deviation fault, the sensor output value deviates from the true value by a fixed amount. This type of fault is commonly caused by sensor zero-point drift or installation errors. Mathematically, it can be expressed as a constant deviation vector superimposed on the true value: ; in: Indicates a deviation fault; Indicates the actual fault value; This represents a fixed bias constant.

[0039] For drift faults, the sensor output error increases linearly with time, a typical scenario being the cumulative error of a gyroscope; this can be simulated by superimposing a drift term proportional to time onto the true value: ; in: Indicates a drift fault; Indicates the actual fault value; , representing the drift term that is proportional to time. Indicates the drift rate, Indicates time.

[0040] For a lock-up fault, the sensor output remains at a certain value and stops updating. This situation may be caused by a sensor crash or communication interruption. In the model, all outputs after the fault occur are fixed at the value at the time of the fault. ; in: Indicates a lock-up fault; It represents a constant.

[0041] The actuator failure types designed in this preferred embodiment mainly consider the decrease in motor efficiency, which is described by an efficiency matrix: ; in: This indicates the actual output produced by the actuator; This represents the efficiency coefficient, with a value between 0 and 1, where 1 indicates complete normal operation and 0 indicates complete failure. This refers to the control commands calculated and issued by the flight controller. These are the theoretical output values ​​(such as desired speed or lift) that each motor is expected to achieve, calculated by a control algorithm (such as PID) based on the desired flight attitude, position, or speed.

[0042] In practice, by combining different working conditions, fault types and fault occurrence times, the simulation is automatically run thousands of times to generate a large-scale labeled dataset containing time-series data from multiple sources such as IMU, GPS, magnetometer, barometer, and motor ESC feedback. Each data point is labeled with either "normal" or specific fault type and severity.

[0043] Example 2: Design and Training of a Time-Graph Fusion Fault Diagnosis Network To achieve high-precision diagnosis, such as Figure 2 As shown, this embodiment designs and trains a lightweight time-graph fusion neural network model. This network fully considers the time-series correlation of fault data from substation inspection drones, the topological coupling relationship between multiple sensors, and the differences in data distribution under different operating conditions.

[0044] Specifically, the temporal-graph fusion neural network model uses a dual-branch encoder-fusionr structure: The temporal feature branch utilizes a Transformer to capture long-range temporal dependencies and dynamic evolution patterns in sensor data; the topological relationship branch, based on a causal topology matrix learned through a Bayesian network, mines the coupling correlations between multiple sensors using a graph convolutional neural network. Finally, the complementary features extracted by the two branches are probabilistically modeled using flow normalization to accurately determine the fault type and severity. The specific implementation process of this network is described in detail below: The network first performs conditionalization processing sequentially along the time and feature dimensions, thereby... The density decomposes into: ; in: This represents a complete multivariate time series dataset with dimensions of 1. ,in It is the number of sensors (variables). It is the total number of time steps; Represents the entire dataset The joint probability density; Indicates the time step index. ; Indicates at time step At that time, the vector composed of the observations from all sensors, i.e. ; Indicates from time step 1 to All historical observation data, i.e. ; This means that given all historical data Under these conditions, current observations The conditional probability density; Indicates the sensor (variable) index. ; Indicates the first Each sensor at time step Observed values; Indicates the first Each sensor from time step 1 to Its own historical observation sequence; : indicates the first Each sensor at time step The set of parent nodes and their historical information are defined as follows: Represents a node At time step The previous set of parent nodes and their historical information, including: Represent a The adjacency matrix (usually learned through a Bayesian network) represents the causal topological relationships between sensors; Indicates sensor It is a sensor The parent node (i.e., there exists a parent node from) arrive (directed causal dependence) Includes time step And before that, all sensors Sensors with direct causal effects Observational data; This represents the historical and current information of a sensor at a given parent node. (Reflecting cross-sensor coupling) and its own historical information (Reflecting its own time-series dependence) Under the two-part condition, the sensor In time Observations The conditional probability density.

[0045] By accurately characterizing the probability distribution of each sensor reading influenced by its own history and its causal parent nodes, the model can not only detect anomalies in individual sensors, but also keenly capture anomalies in the causal relationships between sensors (e.g., a change in a sensor reading does not elicit the expected response from its child nodes), thereby achieving more accurate and robust fault diagnosis.

[0046] A key characteristic of time series data is its temporal dynamics, and modeling such dependencies is crucial for accurate anomaly detection. To integrate temporal information... Encoding as latent temporal representation This preferred embodiment uses the Transformer model: ; in: This represents the Transformer encoder model; This indicates a transpose operation to adapt to the input shape requirements of the Transformer encoder; This indicates an index operation.

[0047] Then, the temporal representations learned from each feature dimension are stacked as input. And use the node update function of graph convolutional neural network to obtain temporal and cross-feature representations. : ; in: This represents the forward propagation function of a graph convolutional neural network. This represents the sigmoid activation function; , represents the normalized adjacency matrix of the Bayesian network; , Represents the parameter matrix to be learned; the final matrix Encoded the time series cutoff time The temporal and feature information will be used to calculate anomaly scores.

[0048] In specific operations, Encoded as Then, the flow normalization mapping function can be used. ,Will The density is expressed as:

[0049] ; in: The mapping function represents the probability density in the latent space after the transformation, which follows an isotropic Gaussian distribution. Possible options include real nonvariable stream (NVP) or masked autoregressive stream (ARF); Representing the Jacobian matrix The determinant; This represents the partial derivative.

[0050] Furthermore, multivariate time series The log-likelihood function can be rewritten as: ; in: This represents all the parameters in the model that need to be optimized through training; Multivariate time series The log-likelihood function represents the probability of a given topological structure. and model parameters Under these conditions, the observed data The logarithm of the probability (likelihood); Represents normalized flow The parameters.

[0051] This preferred embodiment utilizes a temporal-graph fusion fault diagnosis network with a temporal-topology dual-branch architecture to achieve collaborative modeling of the temporal dependencies and sensor coupling relationships in UAV fault data. The temporal branch captures dynamic evolution features through a Transformer, while the topology branch mines causal relationships among multiple sensors using a graph convolutional network. Finally, flow normalization achieves feature fusion and probabilistic modeling. This end-to-end network design provides an effective technical means for high-precision fault diagnosis of substation inspection UAVs.

[0052] Meanwhile, to reduce the difference between the simulation and the real domain, this preferred embodiment introduces domain adversarial loss during model training; the overall loss function is: ; in: Classify task losses based on fault classification; For domain adversarial loss based on MMD, These are the weighting coefficients; The overall loss function is defined as follows: Task loss ensures the model can accurately identify faults on simulation data, while domain adversarial loss uses the Maximum Mean Discrepancy (MMD) metric to narrow the feature distributions of the two domains. This kernel-based distribution distance metric measures the degree of distribution difference by calculating the difference in the mean of samples from the two domains in a high-dimensional feature space. By jointly optimizing these two objectives, the features learned by the model are both discriminative for fault detection tasks and invariant between simulation and real-world environments, thereby improving the model's cross-domain generalization ability.

[0053] Example 3: Continuous Optimization of Edge-Cloud Collaboration: The aforementioned domain adaptation techniques can alleviate the distribution discrepancies between the simulation and real domains to some extent. However, in actual deployment, due to the complexity and diversity of real substation environments, the model may still face distribution offset and performance degradation issues. In particular, when encountering special operating conditions or novel fault modes not fully covered by the simulation environment, the model's detection accuracy will significantly decrease.

[0054] like Figure 3 As shown in the preferred embodiment, this method proposes an edge-cloud collaborative model continuous optimization approach. Through online data collection, intelligent sample selection, and incremental learning, it achieves adaptive model evolution and long-term performance assurance. The specific implementation steps are as follows: Edge-based online data acquisition and preliminary screening: A lightweight data acquisition module is deployed at the edge of the drone to record sensor data, model prediction results, and confidence levels in real time during flight. Samples with model output confidence levels below a threshold (indicating model uncertainty) or suspected faulty samples are automatically marked as pending review samples and uploaded to the cloud. This uncertainty-based preliminary screening mechanism can significantly reduce data transmission volume and lower communication costs.

[0055] Cloud-based sample value assessment based on information entropy: After receiving candidate samples uploaded from the edge, the cloud uses information entropy to quantitatively evaluate the learning value of the samples. For classification tasks, the information entropy of a sample is defined as: ; in: Indicates the model predicts samples Belongs to the The probability of a class; Indicates the total number of fault categories; The information entropy of a sample represents its value. Higher information entropy indicates less uncertainty in the model's classification of that sample, meaning the sample contains more new information and thus offers greater value for model improvement. Simultaneously, to avoid selecting too many similar samples, a sample diversity metric is introduced, calculating the feature space distance between candidate samples and the existing training set. ; in: Represents the feature extraction function; For the current training set, Represents any one of these samples; Represents two eigenvectors and of Norm, also known as Euclidean distance (distance in the feature space); Represents the minimum operator; Indicates candidate samples With the entire training set The minimum Euclidean distance between all samples in the feature space. Taking into account both information entropy and diversity, the overall value score of the sample is: ; in: Information entropy is calculated based on the probability distribution predicted by the model; This is the minimum Euclidean distance between the sample and the existing training set in the feature space; and These are preset weighting coefficients; The comprehensive score is used to assess the value of the sample; the samples are ranked according to the value score, and high-value samples are selected for manual annotation or expert review.

[0056] Incremental Learning and Model Update: High-value samples are manually labeled and added to the training set. Knowledge distillation is used for incremental learning to avoid catastrophic forgetting. The loss function for incremental learning is designed as follows: ; in: For newly labeled samples Cross-entropy loss; The knowledge distillation loss is used to constrain the new model on old data. The output remains consistent with the original model; Weighting for distillation loss; This is the loss function for incremental learning. By constraining the consistency of the new model's output with the original model on old data, it maintains the retention of learned knowledge. ; in: Indicates KL divergence; and These are the output probability distributions of the original model and the new model, respectively. The updated model is evaluated on the validation set, and once the performance improvement is confirmed, it is deployed to the edge to complete the model iteration.

[0057] Edge-based model hot updates and continuous monitoring: New models trained in the cloud are pushed to the drone edge via Over-The-Air (OTA) updates, enabling seamless hot updates. The edge continuously monitors model performance metrics (such as detection accuracy and false alarm rate). When performance metrics show a downward trend, a new round of data collection and model optimization is triggered, forming a closed-loop continuous learning mechanism.

[0058] Example 4: Verification of Technical Effects In the three-month field test of this embodiment, the system demonstrated significant advantages: Detection performance: In 127 real simulated faults (such as magnetometer disturbance and single motor efficiency reduction), the system successfully detected and accurately classified 119 faults, with a detection rate of 93.7% and an average false alarm rate of less than 0.5 times / flight hour, which is an order of magnitude improvement over the traditional threshold method (false alarm rate > 3 times / flight hour).

[0059] Real-time performance: The average time for a single inference run of the airborne model is 100ms, which meets the requirements for online real-time performance, and the CPU utilization rate is less than 40%.

[0060] Continuous optimization capability: Through incremental learning in the cloud, the system autonomously discovered two intermittent abnormal sensor patterns caused by electromagnetic interference from specific device combinations that were not defined in the simulation during subsequent operation, and successfully included them in the model diagnosis scope, demonstrating its adaptive evolution capability.

[0061] Security: Real-time edge diagnostics ensures that even during a 120-second communication interruption, the drone still has complete fault perception and basic emergency response capabilities, significantly improving the overall security level of the system.

[0062] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for real-time online fault detection on UAVs based on deep learning, characterized in that, Includes the following steps: S1. Establish a simulation environment for substation inspection scenarios, and define and execute various flight conditions covering hovering, constant speed cruise, and close-range maneuvering in the simulation environment; S2. Inject sensor failure modes and actuator failure modes into the simulation environment to generate a large-scale labeled multi-condition fault dataset. S3. Based on the multi-condition fault dataset, an initial fault diagnosis model is trained using domain adaptation technology, and the fault diagnosis model is deployed on the UAV's onboard computing unit. S4. During the real-time flight of the UAV, collect sensor time-series data and input it into the fault diagnosis model. Extract and fuse time-series features and topological features through the fault diagnosis model, perform probability modeling through flow normalization, and output the fault diagnosis result and confidence level of the current state. S5. Based on the confidence level or fault detection results output by the model, the edge device of the drone filters out sample data of suspected faults and uploads them to the cloud. S6. The cloud performs value assessment on the sample data based on information entropy and sample diversity measurement, selects samples with values ​​higher than a preset value threshold for labeling, and uses knowledge distillation technology to perform incremental learning of the model and update the model parameters. S7. The updated model in the cloud is wirelessly distributed to the edge of the drone to replace the original model, realizing hot updates and continuous optimization of the model.

2. The method for real-time online fault detection of UAVs based on deep learning according to claim 1, characterized in that, The simulation environment for the substation inspection scenario includes a three-dimensional model of the substation equipment layout, a six-degree-of-freedom dynamic model of the UAV, and an environmental model characterizing electromagnetic interference and aerodynamic disturbances.

3. The method for real-time online fault detection of UAVs based on deep learning according to claim 1, characterized in that, The sensor fault modes include deviation fault, drift fault, and lock-up fault; the actuator fault modes simulate motor performance degradation through an efficiency matrix.

4. The method for real-time online fault detection of UAVs based on deep learning according to claim 1, characterized in that, The fault diagnosis model adopts a dual-branch fusion architecture of temporal feature extraction branch and topology modeling branch: the temporal feature extraction branch is used to capture the long-range dependencies of multi-source sensor temporal data using the Transformer model; the topology modeling branch is used to mine the coupling correlation between sensors based on the predefined sensor causal topology matrix through graph convolutional neural network.

5. The method for real-time online fault detection of UAVs based on deep learning according to claim 1, characterized in that, The domain adaptation technique uses maximum mean difference (MMD) as a measure of distribution difference, and is jointly trained by minimizing the weighted sum of task loss and domain adversarial loss. The overall loss function is: ； in: Classify the task loss according to the fault classification; For domain adversarial loss based on MMD, These are the weighting coefficients; This is the overall loss function.

6. The method for real-time online fault detection of UAVs based on deep learning according to claim 1, characterized in that, The sensor causal topology matrix is ​​obtained through Bayesian network learning and represents the conditional dependencies between different sensor measurements.

7. The method for real-time online fault detection of UAVs based on deep learning according to claim 1, characterized in that, The stream normalization uses real-number invariant stream or masked autoregressive stream to map the input data to the latent space to calculate the log-likelihood, which serves as the probabilistic basis for fault detection.

8. The method for real-time online fault detection of UAVs based on deep learning according to claim 1, characterized in that, The comprehensive scoring formula for the sample value assessment is as follows: ； in: Information entropy is calculated based on the probability distribution predicted by the model; This is the minimum Euclidean distance between the sample and the existing training set in the feature space; and These are preset weighting coefficients; This is the overall score for assessing the value of the sample.

9. The method for real-time online fault detection of UAVs based on deep learning according to claim 1, characterized in that, The loss function for incremental learning is: ； in: For newly labeled samples Cross-entropy loss; The knowledge distillation loss is used to constrain the new model on old data. The output remains consistent with the original model; Weighting for distillation loss; This is the loss function for incremental learning.

10. A deep learning-based real-time online fault detection system for unmanned aerial vehicles (UAVs), used to implement the deep learning-based real-time online fault detection method for UAVs as described in any one of claims 1-9, characterized in that, The system includes: The simulation and data generation module is used to build a high-fidelity simulation environment and generate multi-condition fault datasets. The model training and transfer module is used to train and optimize the initial fault diagnosis model using domain adaptation techniques. An airborne real-time diagnostic module, deployed on an embedded platform of an unmanned aerial vehicle, includes the aforementioned time-series-graph fusion fault diagnosis network for online fault detection; The edge-cloud collaborative optimization module includes an edge data acquisition and screening unit located locally on the drone, a cloud-based sample evaluation and model training unit located remotely, and a communication unit for model distribution, which enables continuous incremental learning and hot updates of the model.

Images