Cloud edge collaborative perception and autonomous intervention method and system for abnormal state of unmanned aerial vehicle

Abstract

The invention belongs to the technical field of intelligent unmanned system safety monitoring and edge artificial intelligence. According to the method, real-time analysis and lightweight anomaly detection are carried out on time sequence data of a multi-source sensor of an unmanned aerial vehicle through an edge side based on a cloud edge collaborative architecture, a control instruction is generated immediately to intervene the unmanned aerial vehicle when an anomaly is detected, and meanwhile, deep learning model training is carried out on historical flight data by relying on a cloud end; and a knowledge distillation or model pruning technology is utilized to generate a lightweight model suitable for edge equipment, so that continuous self-adaptive evolution of the anomaly detection capability is realized. The invention provides a new thought for detecting the abnormal state of the unmanned aerial vehicle, and realizes the abnormal state sensing and autonomous intervention in the flight process of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the fields of intelligent unmanned system safety monitoring and edge artificial intelligence technology, and in particular to a method and system for abnormal state perception and autonomous intervention during the flight process of unmanned aerial vehicles. Background Technology

[0002] With the rapid development of the low-altitude economy, drones are being deployed on a large scale in scenarios such as power line inspection, logistics delivery, emergency rescue, and agricultural plant protection. However, their flight safety is highly dependent on their ability to perceive their own status in real time. In complex environments, factors such as sensor drift, motor failure, communication interruption, and external interference can all lead to flight loss of control or even crashes. Therefore, building a highly reliable, low-latency, and evolvable abnormal state detection and response mechanism has become a key technical requirement for ensuring the safe operation of drones. Summary of the Invention

[0003] This invention provides a cloud-edge collaborative perception and autonomous intervention method and system for abnormal states of unmanned aerial vehicles (UAVs), which can improve the ability of UAVs to detect anomalies and ensure the safe flight of UAVs.

[0004] This invention provides a cloud-edge collaborative sensing and autonomous intervention method and system for abnormal states of unmanned aerial vehicles (UAVs). The specific technical solution is as follows: The method of this invention mainly includes four core steps to construct a closed-loop sensing system that coordinates cloud, edge, and device: Edge-side data acquisition: The UAV collects flight data and flight control logs in real time through multi-source sensors (IMU, GPS, etc.) and sends them to the edge-side ground station; Edge-side inference and synchronization: After the edge-side ground station preprocesses the data, it uses a lightweight model to detect abnormal states of the UAV in real time, and intervenes immediately when an anomaly is detected, while synchronizing the UAV data to the cloud. Cloud-based training: High-precision deep learning anomaly detection models are trained and updated using massive amounts of historical data in the cloud. Model update: The cloud generates a lightweight anomaly detection model through knowledge distillation or pruning, and sends it to the edge via downlink to replace the old model, thus achieving system evolution.

[0005] Achieving millisecond-level autonomous intervention at the edge, the lightweight anomaly detection model in the edge module can complete anomaly detection within milliseconds. Once an anomaly is detected, the edge node does not rely on cloud commands but makes immediate decisions directly based on preset strategies (including returning to home, hovering, forced landing, changing course, etc.), thereby ensuring flight safety in extreme situations such as communication interruptions.

[0006] Building a high-precision unsupervised model in the cloud: A cloud server constructs a multivariate time series dataset from historical data and trains a high-precision deep anomaly detection model. This model employs an unsupervised autoencoder structure (such as LSTM-Autoencoder or Transformer-Autoencoder) to learn the normal behavior pattern of the system by minimizing the reconstruction error of normal flight data, thereby effectively identifying anomalous data that deviates from this pattern.

[0007] Lightweight model design adapted to edge computing power: The lightweight anomaly detection model sent to the edge is an autoencoder model obtained through knowledge distillation (such as an autoencoder based on a fully connected network or LSTM). Its model parameters are compressed to less than 100KB, which can perfectly adapt to edge computing devices with limited computing resources and ensure operation under low power consumption.

[0008] This invention employs a knowledge distillation method based on a teacher-student network: During model lightweighting, a heterogeneous model architecture is constructed: a high-precision "teacher model" in the cloud and a lightweight "student model" at the edge. During training, the teacher model parameters are fixed, and its output soft labels and true labels are used as supervision signals. By minimizing the KL divergence between the probability distributions of the two outputs, the student model is driven to simulate the decision boundary of the teacher model, enabling it to maintain high-precision feature extraction capabilities even with low computational power.

[0009] Establish a "difficult example mining" and "collaborative training" mechanism: The system has continuous learning capabilities. The edge side automatically selects "difficult samples" with low confidence and actively uploads them to the cloud; the cloud aggregates these difficult example samples and historical data for retraining, and transfers new knowledge to the next-generation model through knowledge distillation, realizing adaptive optimization of the model for complex scenarios. Attached Figure Description

[0010] The accompanying drawings provide a further understanding of the invention and explain the invention through specific embodiments.

[0011] Figure 1 This is a flowchart of cloud-edge collaborative perception and autonomous intervention for abnormal states of drones provided by the present invention.

[0012] Figure 2 This is a schematic diagram of the "cloud-edge-device" three-level collaborative system architecture provided by the present invention. Specific implementation methods

[0013] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0014] Figure 1 This is a flowchart illustrating a method for perceiving and analyzing abnormal states of unmanned aerial vehicles (UAVs) in a cloud-edge-device collaborative manner, as provided in an embodiment of the present invention. Figure 1 As shown, the method mainly includes the following steps: Step 101: Collect multi-source sensor data from the UAV. In this embodiment, the end-device (UAV) serves as the data source and operates continuously during flight. The flight control system onboard the UAV (e.g., a PX4 or ArduPilot system based on Pixhawk hardware) reads data from the onboard sensor array in real time via its internal bus. The multi-source sensor data specifically includes, but is not limited to: Inertial Measurement Unit (IMU) data: This includes angular velocity and acceleration information collected by a three-axis accelerometer and a three-axis gyroscope, used to reflect the aircraft's attitude; Location data: latitude, longitude, altitude, and ground speed information from GPS / RTK modules; Status data: This includes battery voltage, current, motor speed (RPM), and magnetic compass data. This data is sampled at high frequencies (e.g., 50Hz-100Hz) to form the original time-series data stream.

[0015] Step 102: Send sensor data to the edge ground station. The UAV uses an onboard data radio or 4G / 5G communication module to establish an uplink data link using the MAVLink communication protocol, encapsulates the real-time data stream collected in step S101 into data packets, and sends them to the edge ground station.

[0016] Step 103: The edge-side ground station parses and preprocesses the data. After receiving the MAVLink data packet, the edge-side ground station (e.g., a smart gateway running on a high-performance laptop or edge computing box) performs the following operations: Protocol parsing: Decode the MAVLink message and extract the specific sensor values; Data cleaning: Removing null values ​​or malformed data that may have been generated during transmission; Preprocessing: Timestamp alignment is performed on the multi-source heterogeneous data, and normalization or standardization methods are used to map the data to a unified interval (e.g., [0, 1]) to form standardized feature vectors for easy model input. After this step, the data stream is split into two paths: one for local real-time detection (step 104), and the other for cloud synchronization (step 107).

[0017] Step 104: Real-time detection using a lightweight edge-side model. The edge-side ground station inputs the pre-processed feature vector into a pre-deployed lightweight anomaly detection model. This lightweight model is a deep learning model compressed through cloud-based knowledge distillation (e.g., a compressed version of LSTM-Autoencoder). This model consumes extremely low computational resources and can perform millisecond-level inference on edge devices, calculating the reconstruction error or anomaly probability of the current flight state.

[0018] Step 105: Detecting Data Anomalies and Generating Intervention Commands. When the anomaly score calculated in Step 104 exceeds a preset safety threshold, the system determines that it is currently in an abnormal state (e.g., detecting drift caused by GPS signal spoofing, or abnormal motor vibration). The edge-side system queries a pre-set safety policy library based on the anomaly type and automatically generates corresponding intervention commands. For example, if a serious sensor malfunction is detected, a "Return to Home Immediately" or "Emergency Hover" command is generated.

[0019] Step 106: Issue intervention commands to control the UAV. The edge-side ground station sends the intervention commands generated in Step 105 to the UAV flight control system with the highest priority via the downlink control link of the MAVLink protocol. The UAV actuators respond to the commands and perform avoidance maneuvers, thereby ensuring flight safety without relying on cloud intervention.

[0020] Step 107: Synchronize preprocessed data to the cloud. While performing edge detection, the edge-side ground station synchronously uploads the preprocessed data to the cloud via the Internet using the MQTT protocol.

[0021] Step 108: Save the preprocessed data to the cloud database. The cloud server receives MQTT messages and uses a subscription service to persistently store massive amounts of flight time-series data in a distributed time-series database (such as InfluxDB or TimescaleDB), building a historical flight data warehouse.

[0022] Step 109: Training the deep learning model in the cloud. The cloud utilizes a high-performance GPU cluster to periodically extract historical data from the database (especially data containing newly uploaded anomalous samples). In this step, a complex and highly accurate teacher model (e.g., a deep Transformer model) is constructed and trained using massive amounts of data in a fully supervised or semi-supervised manner until the model converges, giving it extremely strong feature extraction and generalization capabilities.

[0023] Step 110: Distillation / Pruning to Generate a Lightweight Model. To migrate cloud capabilities to the edge, knowledge distillation is utilized. Using the teacher model trained in Step 109 as guidance, a student model with a small number of parameters (i.e., a lightweight model) is trained. By minimizing the KL divergence between the output distributions of the teacher and student models, the student model "mimics" the decision-making logic of the teacher model. Simultaneously, model pruning techniques can be combined to remove redundant neural connections, further compressing the model size (e.g., to within 100KB).

[0024] Step 111: Distribute updated model to edge nodes. The cloud pushes the latest lightweight model file generated in Step 110 to the edge ground station via the downlink network. After receiving the new model, the edge ground station uses a hot update mechanism to replace the old inference engine (as shown by the dotted arrow in the figure, the updated model is reapplied to Step 104), thereby completing the closed-loop evolution of the system and enabling the edge to identify new anomalies.

[0025] Figure 2 This is a schematic diagram of a three-level collaborative system architecture of "cloud-edge-device" provided in an embodiment of the present invention. Figure 2 As shown, this system is logically and physically divided into three core layers: the terminal layer, the edge layer, and the cloud layer. The following sections provide a detailed explanation of each level and its collaborative relationships: End-side layer (UAV execution layer): The end-side layer is the physical execution and perception front end of the system, mainly composed of sensor groups and flight control units. Sensor group: Includes GPS positioning module, IMU (Inertial Measurement Unit), pitot tube, altimeter, and ESC feedback module. Its function is to collect physical state data of the UAV from all directions. Flight Control Unit: Employs a flight controller based on Pixhawk or similar hardware, running ArduPilot or PX4 firmware. The flight control unit is responsible for low-level attitude calculations and motor control. Data Interaction: The edge layer establishes a wireless communication link through a data transmission radio or an onboard 4G / 5G module, and is responsible for reporting the raw sensor data and MAVLink status packets to the edge layer in real time, and receiving control commands from the edge layer. Edge layer (ground station collaboration layer): The edge layer is the hub connecting the edge and the cloud, and is typically deployed in high-performance portable ground stations or edge computing boxes. For example... Figure 2 As shown in the middle section, this layer contains the following key modules: Data cleaning and caching module ( Figure 2 (The cylindrical icon on the left in the middle): It is responsible for receiving the high-frequency data stream uploaded by the receiving end, performing preprocessing such as noise reduction and packet loss filling, and establishing a local circular buffer to meet the data temporary storage needs in weak network environments. Lightweight Engine Inference Module: Deploys a lightweight deep learning model (i.e., the student model) that has been distilled in the cloud. This module performs real-time inference on the preprocessed data and calculates the probability of anomalies. Anomaly Detection and Feedback Module: Determines the current state based on the reasoning results. As shown in the figure, this module focuses on identifying typical faults such as GPS drift, motor stall, and abnormal vibration. Once an anomaly is identified, an alarm signal is immediately generated. Control Center: Receives abnormal feedback signals, converts decisions into standard MAVLink protocol control commands based on preset safety policies (such as automatic return to home, landing in place), and sends them to the end-side flight controller via downlink. Cloud layer (model training and global analysis layer): The cloud layer is deployed on a remote server cluster and has high-performance computing resources (GPU cluster) and large-capacity storage resources. Distributed databases: such as Figure 2 As shown at the top, a large-scale time-series database is maintained in the cloud to persistently store all historical flight log data, forming a data warehouse. Deep learning model training module: Utilizes massive samples (including normal flight data and labeled abnormal data) from historical data warehouses to perform offline training and hyperparameter optimization of complex models (teacher models). Model optimization and deployment: such as Figure 2 As shown by the dashed line "Model Optimization", the cloud uses knowledge distillation and model pruning techniques to transform large models into lightweight models, and then distributes updates to the edge layer inference engine via the network to achieve algorithm iteration.

[0026] Inter-level collaboration mechanisms, such as Figure 2 As shown, two core closed-loop links are formed between the cloud, edge, and endpoint layers: Data uplink path (solid line / thick dashed line): Edge sensor data -> Edge data cleaning -> Anomaly detection -> Cloud data synchronization -> Storage in distributed database. This path ensures real-time data monitoring and historical data retention. Downlink control and updates (thin dashed line): Short loop (real-time control): Side control center -> MAVLink protocol -> End-side flight controller. This loop has extremely low latency (milliseconds) and is used for emergency avoidance. Long Loop (Model Evolution): Cloud-based model training -> Model optimization -> Edge-based lightweight engine. This loop has a long cycle and is used to improve the system's intelligence level.

[0027] Through the above three-tier architecture, this system effectively solves the pain points of insufficient computing power on the edge and poor real-time performance in the cloud, and achieves optimal resource allocation.

Claims

1. A cloud-edge collaborative perception and autonomous intervention method and system for abnormal states of unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: End-side data acquisition steps: During the flight of the UAV, flight status data is collected in real time through multi-source sensors. The data includes inertial measurement unit (IMU) data, GPS positioning data and flight control log data, and is sent to the edge-side ground station in real time. Edge-side real-time inference and synchronization steps: Edge nodes receive and parse the flight status data, perform data preprocessing, and then use a lightweight anomaly detection model for real-time analysis; when an anomaly is detected, the edge node makes an immediate decision based on a preset strategy and directly intervenes in the behavior of the drone through instructions; at the same time, the preprocessed data is synchronized to the cloud server; Cloud training and global analysis steps: The cloud server receives and stores data from the edge side to build a historical flight database; the cloud computing power is used to train a deep learning model on the historical data to generate an updated anomaly detection model; Model closed-loop update steps: The cloud can generate a lightweight anomaly detection model suitable for edge computing resource limitations through knowledge distillation or model pruning techniques, and send it to the edge ground station through the downlink to replace the old model, thereby realizing the adaptive evolution of the anomaly detection system.

2. According to the method described in claim 1, the lightweight anomaly detection model in the edge-side module can complete anomaly detection and feed it back to the edge-side node within milliseconds. The edge-side node makes an immediate decision, which includes, but is not limited to, returning to home, hovering, forced landing, changing course, etc., to ensure flight safety.

3. The method according to claim 1, characterized in that, The cloud server preprocesses the stored historical flight data to construct a multivariate time series dataset, and trains a high-precision deep anomaly detection model based on the dataset. The deep anomaly detection model is an unsupervised autoencoder structure, including LSTM-Autoencoder or Transformer-Autoencoder. It learns the normal behavior pattern of the system by minimizing the reconstruction error of normal flight data, and judges data that deviates from the pattern as anomalies.

4. The method according to claim 1, characterized in that, The lightweight anomaly detection model is an autoencoder model obtained through knowledge distillation, including autoencoders based on fully connected networks or autoencoders based on long short-term memory networks (LSTM). Its model parameter size is less than 100KB, making it suitable for resource-constrained edge computing devices.

5. The method according to claim 1, characterized in that, The process of transforming the trained and updated model into a lightweight model specifically employs a knowledge distillation method based on a teacher-student network to construct a heterogeneous model architecture: a high-precision teacher model is trained in the cloud using all historical data, and a lightweight student model adapted to edge computing power is constructed; joint distillation training: the parameters of the teacher model are fixed, and its output soft targets are used as supervision signals. By minimizing the KL divergence between the output probability distributions of the teacher model and the student model, the student model is driven to simulate the decision boundary and feature extraction capability of the teacher model.

6. The method according to claim 1, characterized in that, The closed-loop update of the model includes difficult example mining: edge-side ground stations screen out difficult samples with low confidence or uncertainties, mark them, and actively upload them to the cloud; collaborative training: the cloud aggregates difficult example samples and historical data uploaded by multiple machines to retrain the deep learning model; knowledge distillation: based on the teacher-student network architecture, the knowledge of the high-precision teacher model in the cloud is transferred to the student model on the edge side to generate a lightweight model with a small number of parameters and maintain high accuracy, and then distribute it.

Images