Unmanned Aerial Vehicle Disturbance Resistant Flight System and Method Based on Multi-Source Fusion Sensing

Abstract

This invention relates to the field of autonomous flight control and intelligent decision-making for unmanned aerial vehicles (UAVs), and discloses a UAV anti-disturbance flight system and method based on multi-source fusion perception. The system includes an airborne system and a ground system. The airborne system is equipped with a three-source module (visual, ultrasonic, and inertial), a fusion computing module connected to the three-source modules, a robust control module connected to the fusion computing module, and a degradation decision-making module connected to both the fusion computing module and the robust control module. This invention improves the anti-disturbance capability of UAVs during forest fire prevention and provides reliable perception support for autonomous flight and mission execution.

Description

Technical Field

[0001] This invention relates to the field of autonomous flight technology for unmanned aerial vehicles (UAVs), specifically including an anti-disturbance flight system and method for UAVs based on multi-source fusion perception. Background Technology

[0002] Forest fires are characterized by their sudden onset, rapid spread, and complex fire environments. In recent years, fixed-wing and multi-rotor drones have been widely used for tasks such as fire reconnaissance, fire line mapping, and residual fire inspection. However, fire sites and their surrounding areas often experience severe environmental factors such as strong gusts of wind, dense smoke, rapid temperature gradients, and electromagnetic interference, causing drones to experience positioning drift, image blurring, obstacle avoidance failure, and even crashes during missions, severely reducing fire monitoring efficiency and flight safety.

[0003] Specifically, dense smoke and backlighting blur the images captured by visual sensors, making fire point identification difficult; gusts of wind cause the aircraft to vibrate, deviating its flight path from the intended route; temperature gradient changes increase ultrasonic ranging errors; and electromagnetic clutter interference can cause the navigation system to lose lock. These interfering factors not only reduce the positioning accuracy and image quality of the UAV, but also increase the risk of flight loss of control, severely limiting the efficiency and reliability of UAVs in forest fire prevention. Therefore, there is an urgent need for a UAV flight system and method that can effectively resist interference to ensure stable flight and accurate fire monitoring in complex environments. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a UAV anti-disturbance flight system based on multi-source fusion perception, including a processor, and an airborne system and a ground system connected to the processor via data communication. The ground system is used to send the cruise area and fire threshold to the airborne system, and to issue hovering or return-to-home commands to the airborne processor when abnormal disturbances occur. Further, the airborne system includes:

[0005] The visual imaging module is used to capture environmental images, calculate and output visual degradation in real time;

[0006] The ultrasonic processing module is used to transmit ultrasonic pulses to the target area and receive echoes, measure distances in real time, and output ultrasonic errors.

[0007] The inertial sensing module is used to acquire the angular velocity and acceleration measurement data of the body and output inertial zero bias in real time;

[0008] The fusion computing module is connected to the visual image module, ultrasonic processing module and inertial sensing module for data connection. It is used to determine the disturbance level L1-L4 based on the real-time visual degradation, ultrasonic error and inertial zero bias, and then generate and output the disturbance compensation vector and disturbance state quantity.

[0009] The robust control module is connected to the fusion computing module and is used to replan the trajectory in real time and feed forward compensation based on the disturbance compensation vector and disturbance state variables, and then output the mission feedback packet to the ground system.

[0010] The degradation decision module is data-connected to the fusion computing module and the robust control module. It is used to trigger the degradation path and output an eight-bit degradation code when any sensing source fails.

[0011] Furthermore, the ground system includes:

[0012] The ground mission planning station is used to receive external inputs such as the cruise area, weather wind field, no-fly zone, and fire prior threshold, automatically generate flight path templates and mark potential disturbance sources, and output mission packages to guide the airborne system to resist disturbances during flight.

[0013] The ground real-time monitoring station is used to receive task packages output by the ground planning task station in real time, and overlay them on the electronic map and video stream, so that the operator can monitor the disturbance status in real time and issue manual intervention instructions.

[0014] The ground data post-processing center is used to store raw sensor streams and mission logs, automatically compare four types of indicators: wind disturbance attitude error, visual availability, ultrasonic error, and GNSS lockout duration, generate disturbance assessment reports and push updated parameters via OTA, and synchronize the data transmitted back from the ground real-time monitoring station.

[0015] The ground-based emergency intervention terminal is used to send commands to the drone for forced hovering, forced return to home, or emergency landing when disturbance indicators exceed limits or airborne degradation is triggered. It also displays the remaining battery power, the nearest safe point, and the predicted wind speed in real time to reduce the risk of uncontrolled disturbances.

[0016] Furthermore, the visual imaging module includes an RGB global shutter camera, an event camera, an image processing unit, and a feature extraction and sharpness evaluation unit;

[0017] The ultrasonic processing module includes an ultrasonic transmitter, an ultrasonic receiver, a signal processing unit, and a ranging calculation unit.

[0018] The inertial sensing module includes a three-axis accelerometer, a three-axis gyroscope, a signal processing unit, and a data fusion unit;

[0019] The fusion computing module includes a data interface unit, a fault detection unit, a disturbance observer, an adaptive fusion unit, and a disturbance rejection compensation vector generation unit.

[0020] The robust control module includes a data receiving unit, a trajectory replanning unit, a feedforward compensation unit, a redundant action allocation unit, a control command generation unit, and a communication interface.

[0021] The degradation decision module includes a sensor health detection unit, a fault determination unit, a degradation strategy unit, and an instruction output unit.

[0022] Furthermore, the formula for calculating the visual degradation amount is as follows:

[0023]

[0024] Among them, D vis This refers to the amount of visual degradation; Laplacian (I gray ) is a grayscale image I gray High-frequency energy after Laplace kernel convolution; E ref This serves as the high-frequency energy reference for the static frame before takeoff.

[0025] Furthermore, the formula for calculating the ultrasonic error is as follows:

[0026]

[0027] Where ∈ represents the ultrasonic error; c(T) is the temperature-corrected velocity of sound. Δt is the time difference between transmission and reception, d ref This serves as the previous effective distance reference.

[0028] Furthermore, the formula for calculating the zero inertial bias is:

[0029]

[0030] Where the difference b is the inertial zero bias; N is the number of sliding window samples; ω k ω is the current angular velocity. ref The static reference is given; K is the combined visual-ultrasound constrained gain; x vis x us These are the attitude / velocity residuals provided by visual odometry and the ultrasonic ranging residuals, respectively.

[0031] Furthermore, the fusion computing module outputs the disturbance level to the robust control module, the degradation decision module, and the ground system. The disturbance levels L1-L4 are risk scales that the system classifies from low to high based on the total real-time residual variance: L1 has full weights and loose MPC when there is no disturbance in light wind; L2-L3 decrease visual weights and tighten control as the disturbance increases; L4 triggers degradation or return to base immediately when there is an extreme disturbance to ensure flight safety.

[0032] Furthermore, the feature is that the fusion computing module generates an anti-interference compensation vector and an anti-interference state variable in real time, and the calculation formula for the anti-interference compensation vector is:

[0033] Δ=W vis ·D vis +W us ·∈+Wimu ·b

[0034] Where Δ is the disturbance rejection compensation vector; W vis W us W imu These are the adaptive weights of the inverse variance of the output residuals from the vision, ultrasound, and inertial sensors, respectively; D vis ∈ and b represent visual degradation, ultrasonic error, and inertial zero bias, respectively.

[0035] The disturbance rejection state variables are obtained by the fusion calculation module through one-step updating via Extended Kalman Filter (EKF). The calculation formula for the disturbance rejection state variables is as follows:

[0036] x k =Fx k-1 +Bu k-1 +K k [z k -h(x k-1 )]

[0037] Where, x k For disturbance rejection state variables; F is the system transition matrix; B is the control matrix; u k-1 K is the previous control input. k For Kalman gain; z k h is the observation vector; h(·) is the observation function.

[0038] Furthermore, the robust control module outputs a mission feedback packet to the ground system, including fire point coordinates, a fire spread grid, and disturbance confidence labels.

[0039] According to another aspect of the present invention, a method for unmanned aerial vehicle (UAV) anti-disturbance flight based on fused perception is provided, comprising:

[0040] The ground mission planning station receives external inputs such as the cruise area, weather wind field, no-fly zone, and fire prior threshold, generates a flight path template and marks potential disturbance sources, and encapsulates the generated template and disturbance sources into a mission package and sends it to the airborne system.

[0041] After receiving the mission packet, the airborne system initiates the flight mission. The visual imaging module, ultrasonic processing module, and inertial sensing module output the visual degradation, ultrasonic error, and inertial zero bias, respectively.

[0042] The fusion computing module receives visual degradation, ultrasonic error and inertial zero bias, and performs fault detection, disturbance observer estimation and adaptive fusion to generate disturbance immunity compensation vector and disturbance immunity state variables, and determines disturbance levels L1-L4. The fusion computing module then outputs the disturbance level, disturbance immunity compensation vector and disturbance immunity state variables to the robust control module, degradation decision module and ground system.

[0043] The degradation decision module monitors the disturbance level in real time. When any sensor data exceeds the threshold, the degradation decision module triggers the degradation path and outputs an eight-bit degradation code.

[0044] After receiving the degradation code, the robust control module adjusts the sensor weights and then replans the trajectory based on the disturbance level, disturbance rejection compensation vector, and disturbance rejection state quantity output by the fusion calculation module.

[0045] The fusion computing module maps the fire point pixels detected by the visual image module and the anti-disturbance state quantity into fire point coordinates in real time, generates a fire line spread grid and disturbance confidence label, and transmits them back to the ground real-time monitoring station through the robust control module.

[0046] The ground real-time monitoring station receives mission feedback packets in real time and displays the coordinates of the fire point and the fire spread grid. The ground data post-processing center stores the raw sensor stream and mission log, compares various indicators, and if any indicator exceeds the limit, it pushes the updated parameters to the airborne system via OTA.

[0047] The ground data post-processing center pushes updated parameters to the airborne system via OTA. The airborne system reassesses the disturbance level and adjusts the control parameters every 60 seconds before entering the next mission cycle.

[0048] According to one or more technical solutions of the present invention, the present invention has the following beneficial effects:

[0049] 1. Significantly improves anti-disturbance capability

[0050] This invention utilizes multi-source fusion sensing technology (visual, ultrasonic, and inertial) to calculate visual degradation, ultrasonic error, and inertial bias in real time, and generates an anti-disturbance compensation vector. This effectively suppresses common issues in forest fire prevention, such as visual degradation, ultrasonic ranging error, inertial measurement error, and dynamic disturbances, significantly improving the anti-disturbance capability of UAVs in complex environments.

[0051] 2. Improve the accuracy and real-time performance of fire monitoring.

[0052] The fusion computing module maps the fire point pixels detected by the visual image module to fire point coordinates in real time, generating a fire line spread grid and disturbance confidence labels. This data is then transmitted back to the ground real-time monitoring station via the robust control module. Operators can monitor the disturbance status instantly and issue manual intervention commands, improving the accuracy and real-time performance of fire monitoring. Attached Figure Description

[0053] Figure 1 This is a diagram of the airborne system of the present invention.

[0054] Figure 2 This is a ground system diagram of the present invention.

[0055] Figure 3This is a flowchart of the method of the present invention.

[0056] Figure 4 This is a diagram showing the relationship between the airborne system and the ground system of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] Example 1

[0059] A multi-source fusion sensing-based unmanned aerial vehicle (UAV) anti-disturbance flight system includes a processor. The processor comprises a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), a storage unit, and a communication interface. As the system's central control unit, the processor synchronizes and transmits data between the visual imaging module, ultrasonic sensing module, inertial sensing module, and fusion computing module via an internal bus or on-chip network (SoC), ensuring the real-time performance and consistency of data from each module.

[0060] The system also includes airborne and ground systems that are connected to the processor via data links. The airborne systems include:

[0061] The visual imaging module is used to capture environmental images, calculate and output visual degradation in real time;

[0062] The ultrasonic processing module is used to transmit ultrasonic pulses to the target area and receive echoes, measure distances in real time, and output ultrasonic errors.

[0063] The inertial sensing module is used to acquire the angular velocity and acceleration measurement data of the body and output inertial zero bias in real time;

[0064] The fusion computing module is connected to the visual image module, ultrasonic processing module and inertial sensing module for data connection. It is used to determine the disturbance level L1-L4 based on the real-time visual degradation, ultrasonic error and inertial zero bias, and then generate and output the disturbance compensation vector and disturbance state quantity.

[0065] It should be noted that the disturbance rejection compensation vector (Δ) is calculated by the fusion calculation module based on the visual degradation amount D.vis Ultrasonic error ∈ and inertial zero bias b, calculated as the reciprocal of the variance of their respective residuals. A vector synthesized after dynamically adjusting weights. It quantifies the reliability of current sensor data in real time and, through a feedforward compensation mechanism, offsets the impact of these errors on flight control, thereby enhancing the flight stability of the UAV in complex environments. Disturbance-resistant state variable (x) k The fusion computing module uses the Extended Kalman Filter (EKF) algorithm, combined with fused data from visual, ultrasonic, and inertial sensors, to generate an accurate flight state estimate containing information such as the UAV's position, velocity, and attitude. This state variable provides a reliable control basis for the robust control module, ensuring that the UAV can fly stably along the planned trajectory and maintain high flight accuracy even when sensor data is disturbed.

[0066] The robust control module is connected to the fusion computing module and is used to replan the trajectory in real time and feed forward compensation based on the disturbance compensation vector and disturbance state variables, and then output the mission feedback packet to the ground system.

[0067] It should be noted that feedforward compensation is a control strategy used to compensate for known disturbances or errors in advance in the control system. By adjusting the control input in real time, it can counteract the impact of disturbances on the flight state and ensure that the UAV can complete its mission stably and efficiently.

[0068] The degradation decision module is connected to the fusion computing module and the robust control module. It is used to trigger the degradation path and output an eight-bit degradation code when any sensing source fails.

[0069] It should be noted that the 8-bit degradation code is an encoding mechanism used by UAV systems to trigger degradation operations in the event of sensor failure or extreme disturbances. It uses an 8-bit binary number (ranging from 0 to 255) to represent different fault types and degradation strategies, facilitating rapid identification and response. The specific encoding rules are as follows: The first bit (highest bit) identifies a visual sensor failure (1 indicates a failure, switching to the event camera; 0 indicates normal); the second bit identifies an ultrasonic sensor failure (1 indicates a failure, switching to visual-inertial altitude hold mode; 0 indicates normal); the third bit identifies an inertial sensor failure (1 indicates a failure, freezing inertial data and switching to visual-ultrasonic altitude hold mode; 0 indicates normal); the fourth bit identifies a multi-source sensor failure (1 triggers return to home or hovering; 0 indicates no such failure). The remaining bits, 5 to 8, are reserved for future expansion or special purposes. For example, when both the visual and ultrasonic sensors fail simultaneously, the generated degradation code is 11000000 (decimal 192). Upon receiving this code, the robust control module quickly switches to the backup sensor and adjusts the control strategy to ensure flight safety.

[0070] The ground system is used to send the cruise area and fire threshold to the airborne system, and to issue hovering or return-to-base commands to the airborne processor when there is abnormal disturbance.

[0071] It should be noted that when the ground data post-processing center detects that indicators such as wind disturbance attitude error, visual availability, ultrasonic error, or GNSS lock-off duration exceed limits, or when the degradation decision module determines that a sensor malfunction or extreme disturbance has occurred, it will immediately generate corresponding hovering or return-to-home commands. These commands are sent to the robust control module of the airborne system via a wireless link (such as 4G or LoRa) or internal bus. After receiving and parsing the commands, the robust control module quickly executes a hovering operation to keep the UAV stationary at its current position, or plans and executes the optimal path back to the takeoff point, ensuring that the UAV can safely and stably stop flying or return in abnormal situations, thus guaranteeing the safety of the flight mission.

[0072] Furthermore, the ground system includes:

[0073] The ground mission planning station is used to receive external inputs such as the cruise area, weather wind field, no-fly zone, and fire prior threshold, automatically generate flight path templates and mark potential disturbance sources, and output mission packages to guide the airborne system to resist disturbances during flight.

[0074] It should be noted that the ground mission planning station includes a mission input interface, a meteorological / no-fly zone database, a flight path generation engine, and a disturbance source labeling module. After the operator inputs the cruise area, meteorological wind field, no-fly zone, and fire threshold through the interface, the system immediately calls the meteorological / no-fly zone database to complete the conflict detection. The flight path generation engine generates feasible tracks based on the three-dimensional terrain and automatically inserts backup waypoints in high-risk areas such as canyon winds, smoke areas, and electromagnetic stations. The disturbance source labeling module overlays potential disturbance sources with color codes on the electronic map and outputs a mission package containing flight paths, waypoints, and disturbance level thresholds, which can be issued by the real-time monitoring station with one click.

[0075] The ground real-time monitoring station is used to receive task packages output by the ground planning task station in real time, and overlay them on the electronic map and video stream, so that the operator can monitor the disturbance status in real time and issue manual intervention instructions.

[0076] It should be noted that the ground real-time monitoring station includes a LoRa / 4G dual-link transceiver, a GIS overlay engine, an H.265 video decoder, and an intervention command panel (including emergency stop, return to base, and hover three-button operation). The transceiver completes airborne data packet parsing within 500ms, and the GIS engine projects the fire point coordinates onto a 1m resolution electronic map in real time, automatically generating a fire line spread raster animation. The video decoder synchronously plays 1080p fire scene video and overlays disturbance confidence labels. The intervention panel allows operators to issue hover or return to base commands with a single button within 1 second, and the system automatically records the intervention log.

[0077] The ground data post-processing center is used to store raw sensor streams and mission logs, automatically compare four types of indicators: wind disturbance attitude error, visual availability, ultrasonic error, and GNSS lockout duration, generate disturbance assessment reports and push updated parameters via OTA, and synchronize the data transmitted back from the ground real-time monitoring station.

[0078] It should be noted that the ground data post-processing center includes an NVMe storage array, a Python comparison script, and an OTA differential packet generator. The storage array saves the original sensor stream and mission logs at a write rate of 1GB / s. The comparison script calculates four types of indicators every 10 seconds: wind disturbance attitude error, visual availability, ultrasonic error, and GNSS lockout duration. If any of these indicators are exceeded, the differential packet generator is invoked to generate an OTA update file of only 50kB within 30 seconds. The OTA channel transmits the data back to the airborne system via 4G to complete online parameter calibration.

[0079] It should also be noted that OTA (Over-The-Air) technology is a wireless update method that allows ground stations to remotely push software updates, parameter adjustments, and other data to the UAV via wireless networks (such as 4G, 5G, and Wi-Fi). Wind disturbance attitude error, visual availability, ultrasonic error, and GNSS lock-off duration are four key parameters for assessing flight status and sensor health. Wind disturbance attitude error reflects the impact of wind on the UAV's attitude stability; visual availability measures the effectiveness of visual sensor data in complex environments; ultrasonic error reflects the accuracy of ultrasonic ranging; and GNSS lock-off duration indicates the continuity of satellite signals. The ground data post-processing center compares these indicators in real time. If any indicator exceeds the limit—such as wind disturbance attitude error exceeding 10°, visual availability below 60%, ultrasonic error exceeding 0.1 meters, or GNSS lock-off exceeding 10 seconds—updated parameters are pushed to the airborne system via OTA to optimize control strategies and ensure flight safety and mission continuity.

[0080] The ground-based emergency intervention terminal is used to send commands to the drone for forced hovering, forced return to home, or emergency landing when disturbance indicators exceed limits or airborne degradation is triggered. It also displays the remaining battery power, the nearest safe point, and the predicted wind speed in real time to reduce the risk of uncontrolled disturbances.

[0081] It should be noted that the ground emergency intervention terminal includes an emergency stop button, a voice broadcaster, a remaining battery / safety point display, and a LoRa emergency link. When disturbance indicators exceed limits or airborne degradation is triggered, the emergency stop button generates a mandatory command within 200ms; the voice broadcaster repeatedly broadcasts the "return to base - hover" command; the display refreshes the remaining battery, the coordinates of the nearest safe point, and the 5-minute wind speed forecast in real time; and the LoRa link can still issue commands within 1km even without 4G signal.

[0082] Furthermore, the visual imaging module includes an RGB global shutter camera, an event camera, an image processing unit, and a feature extraction and sharpness evaluation unit;

[0083] In this embodiment, the visual imaging module consists of a SONY IMX-264RGB global shutter camera, a PROPHESEEEVK4 event camera, a Xilinx Artix-7XC7A35T FPGA, and 256MB of DDR3 cache. The RGB camera captures environmental images at a resolution of 2048×1536 at 60fps, while the event camera captures dynamic changes at 1kHz. The FPGA receives the RGB images, runs the CLAHE algorithm for contrast enhancement, and then applies the Laplacian operator to calculate the high-frequency energy of the image, outputting a sharpness index E. The feature extraction unit extracts key feature points, such as corners and edges, from the RGB images for subsequent visual odometry calculations. The sharpness evaluation unit calculates the ratio of the current frame sharpness E to the pre-takeoff baseline E_ref to obtain the visual degradation amount. When D vis If the threshold value is exceeded (e.g., 0.3), the FPGA switches to the event camera data stream to maintain the continuity of the visual odometry. In a dense smoke environment with 30m visibility, the visual degradation D... vis The calculation accuracy reaches ±0.02, ensuring that the fire point coordinate error is less than 2px. The event camera seamlessly takes over when vision degrades, ensuring continuous monitoring capability under low visibility conditions, and reducing the false alarm rate of fire point detection to below 2%.

[0084] The ultrasonic processing module includes an ultrasonic transmitter, an ultrasonic receiver, a signal processing unit, and a ranging calculation unit;

[0085] In this embodiment, the ultrasonic processing module consists of a TDKMA40S4R ultrasonic transmitter, an MA40S4S ultrasonic receiver, a TIPGA460 signal processing unit, and an STM32L432 MCU. The STM32L432 configures the PGA460 via SPI to alternately transmit 40kHz and 60kHz ultrasonic pulses with a period of 5ms. The PGA460 captures the echo signal, performs automatic gain control and correlation detection, and calculates the echo time difference Δt with an accuracy of 0.1μs. The MCU reads data from its internal NTC temperature sensor and calculates the temperature-corrected velocity of sound. It also outputs ultrasonic errors in real time. When the flame temperature difference is 60℃, the ranging error is reduced from ±0.25m to ±0.05m, improving ranging accuracy and reliability.

[0086] The inertial sensing module includes a three-axis accelerometer, a three-axis gyroscope, a signal processing unit, and a data fusion unit;

[0087] In this embodiment, the inertial sensing module consists of a Bosch BMI088 triaxial accelerometer and a triaxial gyroscope, and an STM32L432 MCU. The MCU receives acceleration and angular velocity data at a sampling frequency of 400Hz, calculates the Allan variance through a 300-point sliding window, and estimates the inertial zero bias b. Combining the attitude / velocity residuals provided by vision and ultrasound, the MCU runs the ZUPT algorithm to correct inertial drift. Within 30 minutes, the position drift is less than 0.8m, and the attitude error RMS is less than 1°, reducing the cumulative error during long-term flight and improving the positioning accuracy and flight stability of the UAV.

[0088] The fusion computing module includes a data interface unit, a fault detection unit, a disturbance observer, an adaptive fusion unit, and a disturbance rejection compensation vector generation unit.

[0089] In this embodiment, the fusion computing module consists of an NVIDIA Jetson OrinNano, a Lattice ECP5 FPGA, and 8GB of LPDDR5 memory. The data interface unit receives real-time data from the visual imaging module, ultrasonic processing module, and inertial sensing module. The fault detection unit continuously monitors the integrity and accuracy of data from each sensor, promptly identifying potential faults. The disturbance observer estimates visual degradation, ultrasonic error, and inertial bias based on real-time data, providing accurate disturbance information for subsequent processing. The adaptive fusion unit dynamically adjusts the weights according to the residual variance of each sensor, generating an anti-disturbance compensation vector Δ and an anti-disturbance state variable xk, while simultaneously determining the L1-L4 disturbance levels. The anti-disturbance compensation vector generation unit integrates and outputs this information, providing necessary data support for the robust control module. Every 60 seconds, the fusion computing module recalculates the weights and updates them via OTA, ensuring the system can dynamically adjust according to real-time environmental changes, maintaining efficient operation and high-precision fusion. The position error after fusion is less than 0.3m, and the CPU load is less than 25%, significantly improving the overall performance and reliability of the system.

[0090] The robust control module includes a data receiving unit, a trajectory replanning unit, a feedforward compensation unit, a redundant action allocation unit, a control command generation unit, and a communication interface.

[0091] In this embodiment, the robust control module consists of a TITMS320F28379D MCU and a six-redundant BLDC motor. The data receiving unit receives the disturbance rejection compensation vector Δ and disturbance rejection state variable x from the fusion calculation module every 20ms. kBased on this data, the trajectory replanning unit runs the Tube-MPC algorithm for 1-second rolling optimization, generating new control commands. The feedforward compensation unit uses the disturbance rejection compensation vector Δ as a feedforward term to compensate for the impact of disturbances on flight in real time. When a single motor failure is detected, the redundancy action allocation unit reallocates the thrust of the remaining motors within 5ms to ensure flight stability. The control command generation unit generates the final control commands based on the optimization results and sends them to the UAV's actuators via the communication interface. Upon receiving the degradation decision module's degradation command, it switches to the backup sensor or triggers return-to-home within 3ms. Through this series of mechanisms, the robust control module ensures that the UAV's attitude overshoot is less than 2° in strong winds (gusts of 12m / s), significantly improving flight stability and redundancy.

[0092] The degradation decision module includes a sensor health detection unit, a fault determination unit, a degradation strategy unit, and an instruction output unit.

[0093] In this embodiment, the degradation decision module consists of an STM8L151 MCU and dual-channel interrupts. The sensor health detection unit calculates and updates the visual degradation amount D every 100ms. vis The residual variance of ultrasonic error ∈ and inertial zero bias b The fault determination unit monitors these residual variances in real time. If any residual variance exceeds a set threshold (such as visual degradation D), the fault determination unit will detect the fault. vis If the error exceeds 0.3 m, the ultrasonic error exceeds 0.1 m, or the inertial zero bias (b) exceeds 0.05° / s, the corresponding sensor is considered faulty. The degradation strategy unit selects and executes the appropriate degradation strategy based on the fault type, such as switching to the event camera when the visual sensor fails, and switching to the visual-inertial altitude hold mode when the ultrasonic sensor fails. Upon detecting a fault, the command output unit immediately outputs an 8-bit degradation code, and an interrupt signal is sent to the robust control module and the ground station within 200 ns. Through this mechanism, sensor failure is degraded within 150 ms, reducing the failure rate by 90% and significantly improving the system's fault tolerance.

[0094] Furthermore, the formula for calculating visual degradation is:

[0095]

[0096] Among them, D vis This refers to the amount of visual degradation; Laplacian (I gray ) is a grayscale image I gray High-frequency energy after Laplace kernel convolution; E ref This serves as the high-frequency energy reference for the static frame before takeoff.

[0097] It should be noted that this formula quantifies the degree of image degradation by calculating the ratio of the high-frequency energy of the current image to the reference energy. Lower high-frequency energy indicates a more blurred image, and a higher degree of visual degradation (D). vis The larger.

[0098] Furthermore, the formula for calculating ultrasonic error is as follows:

[0099]

[0100] Where ∈ represents the ultrasonic error; c(T) is the temperature-corrected velocity of sound. T represents the current ambient temperature in degrees Celsius; Δt represents the time difference between transmission and reception, expressed as d. ref This serves as the previous effective distance reference.

[0101] It should be noted that this formula calculates the current measurement distance by measuring the round-trip time Δt of the ultrasonic pulse, combining it with the sound velocity c(T) corrected for the current ambient temperature T, and comparing it with the previous effective distance reference d. ref By comparison, the ultrasonic error ∈ is obtained.

[0102] Furthermore, the formula for calculating zero inertial bias is:

[0103]

[0104] Where the difference b is the inertial zero bias; N is the number of sliding window samples; ω k ω is the current angular velocity. ref The static reference is given; K is the combined visual-ultrasound constrained gain; x vis x us These are the attitude / velocity residuals provided by visual odometry and the ultrasonic ranging residuals, respectively.

[0105] It should be noted that the inertial bias formula quantifies the inertial error by calculating the difference between the current IMU drift and the combined visual-ultrasound correction. The larger the drift, the larger the bias b, and the system immediately corrects using visual-ultrasound constraints.

[0106] It should also be noted that visual-ultrasonic constraint is a cross-sensor error correction strategy: in inertial zero-bias estimation, the attitude / velocity residuals x provided by visual odometry are utilized. vis Compared with the residual of ultrasonic ranging x us As an external zero-velocity or zero-bias reference, IMU drift is corrected in real time through Kalman filtering and joint constraints, ensuring that inertial error is always "anchored" by visual and ultrasonic data, thereby compressing long-term positioning drift to the centimeter level.

[0107] Furthermore, the fusion computing module outputs the disturbance level to the robust control module, the degradation decision module, and the ground system. The disturbance levels L1-L4 are risk scales that the system classifies from low to high based on the total real-time residual variance: L1 has full weights and loose MPC when there is no disturbance in light wind; L2-L3 decrease visual weights and tighten control as the disturbance increases; L4 triggers degradation or return to base immediately when there is an extreme disturbance to ensure flight safety.

[0108] Furthermore, the feature is that the fusion computing module generates the anti-disturbance compensation vector and the anti-disturbance state variables in real time, and the calculation formula for the anti-disturbance compensation vector is:

[0109] Δ=W vis ·D vis +W us ·∈+W imu ·b

[0110] Where Δ is the disturbance rejection compensation vector; W vis W us W imu These are the adaptive weights of the inverse variance of the output residuals from the vision, ultrasound, and inertial sensors, respectively; D vis ∈ and b represent visual degradation, ultrasonic error, and inertial zero bias, respectively. This formula quantifies the overall disturbance intensity in real time by weighting and summing the three source errors of vision, ultrasound, and inertia according to the inverse of their respective residual variances, and uses it as the input for feedforward compensation.

[0111] The disturbance immunity state variables are obtained by the fusion calculation module through one-step updating via Extended Kalman Filter (EKF). The calculation formula for the disturbance immunity state variables is as follows:

[0112] x k =Fx k-1 +Bu k-1 +K k [z k -h(x k-1 )]

[0113] Where, x k For disturbance rejection state variables; F is the system transition matrix; B is the control matrix; u k-1 K is the previous control input. k For Kalman gain; z k Let be the observation vector; h(·) be the observation function. The result x k It includes the drone's real-time position, speed, attitude, and other flight status data, which are used for Tube-MPC feedforward compensation and trajectory replanning in the robust control module.

[0114] Furthermore, the robust control module outputs mission feedback packets to the ground system, including fire point coordinates, generating a fire spread grid and disturbance confidence labels;

[0115] The formula for generating fire point coordinates is:

[0116]

[0117] Among them, P geo That is, the coordinates of the fire point; (u,v) are the pixel coordinates of the fire point detected by the visual imaging module; d is the corresponding pixel depth provided by the ultrasonic sensing module; π -1 It is the inverse projection function; Let be the transformation matrix from the body to the world coordinate system given by the disturbance rejection state variables;

[0118] Accumulate fire point coordinates and predict fire line spread using ConvLSTM, outputting a 1m resolution GeoTIFF raster;

[0119] It's important to note that ConvLSTM is a hybrid model combining convolutional operations and LSTM, specifically designed for spatiotemporal sequence data (such as video, weather forecasts, and radar echoes). Building upon traditional LSTM, it replaces fully connected layers with convolutional layers, enabling the model to directly extract spatial local features of the input data (such as the relationship between adjacent pixels) while analyzing temporal dependencies. Both input and output are 3D tensors (e.g., H×W×C), and gating (input gate, forget gate, output gate) and state updates are computed through sliding convolutional kernels, thus simultaneously modeling spatiotemporal dynamics. Compared to ordinary LSTM, ConvLSTM more efficiently preserves spatial structure and is suitable for tasks requiring joint spatiotemporal modeling (such as future frame prediction and dynamic scene analysis).

[0120] Confidence labels are calculated based on the moving variance of visual, ultrasonic, and inertial residuals.

[0121]

[0122] Where, Perturbation Confidence Label is the confidence level label; N is the total number of sensor modules; Confidence i It is the confidence level of the i-th sensor module; Weight i The weight of the i-th sensor module is used to reflect the importance of different sensors in disturbance estimation; Accuracy is the estimation accuracy of the fusion calculation module, which is obtained by multiplying the confidence of each sensor by its weight and summing them to obtain a comprehensive confidence index.

[0123] The formula for calculating adaptive weights is:

[0124]

[0125] Among them, W vis W us W imuThese are adaptive weights, calculated from the inverse variance of the output residuals of the vision, ultrasound, and inertial sensors, respectively. These are the sliding variance estimates for the visual, ultrasonic, and inertial residuals of the current frame, respectively. The larger the value, the smaller the weight, thus achieving dynamic disturbance rejection.

[0126] Furthermore, the trajectory replanning uses the Tube-MPC algorithm, as follows:

[0127] x k+1 =Ax k +Bu k +w k

[0128] Where, x k It is the system state, u k It is a control input, w k It is a disturbance term.

[0129] It should be noted that the algorithm, based on the disturbance rejection state variables (including position, velocity, and attitude) provided by the fusion computing module and the real-time determined disturbance levels (L1-L4), performs rolling time-domain optimization with a period of 20ms to achieve online replanning of the flight path. Secondly, through a unique "Tube" constraint mechanism, the system state deviation is strictly limited within a preset safety boundary, ensuring flight safety even in the face of sudden disturbances such as L4-level extreme wind shear. Finally, the algorithm forms a closed loop with the multi-source perception system: the disturbance rejection compensation vector (Δ) serves as a feedforward input to directly cancel known disturbances, while the state variables estimated by the extended Kalman filter (EKF) are used for feedback correction. This feedforward-feedback composite control architecture enables the UAV to maintain an attitude overshoot of less than 2° even under the 12m / s gusts common in forest fires. Furthermore, the Tube-MPC can complete the control strategy switch within 150ms based on the eight-bit degradation code output by the degradation decision module, such as automatically increasing the inertial navigation weight when the visual sensor fails, or reallocating thrust when the motor fails.

[0130] Example 2

[0131] Based on the same inventive concept as the foregoing embodiments, this embodiment provides a method for unmanned aerial vehicle (UAV) anti-disturbance flight based on fused perception, as follows:

[0132] The ground mission planning station receives external inputs regarding the cruise area (3km × 2.1km, center coordinates 25.123°N, 113.456°E, altitude 450–620m), meteorological wind field (westerly wind 12m / s, peak gust 14m / s, wind shear 0.8m / s / 100m), and no-fly zone (7 polygons, total area 0.12km²). 2The system sets the fire prior threshold (confidence 0.8, peak flame front temperature 420℃) and generates 18 flight paths within 25 seconds, marking 4 canyon wind disturbance sources (valley mouth wind speed 18m / s). The generated flight path templates and disturbance sources are encapsulated into a task packet (JSON 3.2kB) and transmitted to the airborne system via a 900MHz LoRa wireless link, with a coverage latency of less than 500ms.

[0133] After receiving the mission packet, the airborne system initiates the flight mission. The RGB global shutter camera of the visual imaging module captures environmental images at 60fps, and the event camera captures dynamic changes at 1kHz, outputting the visual degradation measure D in real time. vis =0.26. The transmitter of the ultrasonic processing module emits 40kHz / 60kHz pulses towards the target area. The receiver captures the echo and calculates the echo time difference Δt = 0.018s. Combined with the sound velocity c(T) corrected for a temperature of 45℃ = 355.3m / s, the output ultrasonic error ∈ = 0.10m. The triaxial accelerometer and gyroscope of the inertial sensing module output the angular velocity and acceleration measurement data of the body at a sampling frequency of 400Hz. The inertial zero bias b = 0.005° / s is calculated through a 300-point sliding window.

[0134] The fusion computing module receives visual degradation D vis Based on the ultrasonic error ∈ and the inertial zero bias b, fault detection, disturbance observer estimation, and adaptive fusion are performed to generate the disturbance rejection compensation vector Δ = 28.6 and the disturbance rejection state variable x. k The disturbance level was determined to be L2(σ). 2 =0.12). The fusion calculation module combines the disturbance level L2, the disturbance immunity compensation vector Δ, and the disturbance immunity state variable x. k The output is sent to the robust control module, the degradation decision module, and the ground system. The weights are recalculated every 60 seconds, the CPU load is 24%, and the fused position error RMS is 0.28m.

[0135] The degradation decision module monitors the disturbance level L2 in real time, and when the visual degradation amount D... vis When the threshold of 0.25 is exceeded, a degradation path is triggered, and an eight-bit degradation code 01 is output. Upon receiving the degradation code 01, the robust control module reduces the visual weight from 100 to 50, and the redundant motor completes thrust reconfiguration within 5ms. The robust control module determines the thrust based on the disturbance level L2, the disturbance rejection compensation vector Δ, and the disturbance rejection state variable x. k The Tube-MPC algorithm is used to replan the flight path for the next second, ensuring that the attitude overshoot of the UAV is less than 2° and the position error is less than 0.3m under a gust of wind of 12m / s.

[0136] The fusion computing module combines the fire point pixels detected by the visual image module with the anti-disturbance state quantity x. kThe data is mapped in real time to the coordinates of the fire point, generating a fire spread grid (1m resolution, 2.1MB file) and a disturbance confidence label (0.87 confidence level), and then transmitted back to the ground real-time monitoring station via a robust control module with a transmission delay of 120ms.

[0137] The ground-based real-time monitoring station receives mission feedback packets in real time and displays the coordinates of fire points and the fire line spread grid, allowing operators to monitor the disturbance status immediately and issue manual intervention commands. The ground-based data post-processing center stores the raw sensor streams and mission logs (1.8GB for 45 minutes of logs) and compares indicators such as wind disturbance attitude error, visual availability, ultrasonic error, and GNSS lock-off duration. If any indicator exceeds the limit (e.g., wind disturbance attitude error > 1.5°), 48kB of updated parameters are pushed to the airborne system via OTA, further reducing the error by 33%.

[0138] The ground data post-processing center pushes updated parameters to the airborne system via OTA. The airborne system reassesses the disturbance level and adjusts control parameters every 60 seconds before entering the next mission cycle, ensuring the continuity and stability of the flight mission. It completes a 6.3km mission within 45 minutes. 2 Fire scene inspections showed a 0% fire spot detection rate, attitude drift of less than 1.5°, and a 100% mission success rate.

[0139] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A UAV anti-disturbance flight system based on multi-source fusion perception, comprising a processor, and further comprising an airborne system and a ground system connected to the processor via data connection, wherein the ground system is used to send the cruise area and fire threshold to the airborne system, and to issue hovering or return-to-home commands to the airborne processor when disturbances are abnormal, characterized in that, The airborne system includes: The visual imaging module is used to capture environmental images, calculate and output visual degradation in real time; The formula for calculating the visual degradation amount is: ; in, This refers to the amount of visual degradation; Laplacian ( (This is a grayscale image) High-frequency energy after Laplace kernel convolution; The high-frequency energy reference is the static frame before takeoff; The ultrasonic processing module is used to transmit ultrasonic pulses to the target area and receive echoes, measure distances in real time, and output ultrasonic errors. The formula for calculating ultrasonic error is as follows: ; in, This is due to ultrasonic error; This is the temperature-corrected speed of sound. ; To account for the time difference between sending and receiving, This serves as the previous effective distance reference. The inertial sensing module is used to acquire the angular velocity and acceleration measurement data of the body and output inertial zero bias in real time; The formula for calculating the zero inertial bias is: ; Where the difference b is the inertial zero bias; N is the number of sliding window samples; The current angular velocity; The static reference is used; K is the combined visual-ultrasound constraint gain. These are the attitude / velocity residuals provided by visual odometry and the ultrasonic ranging residuals, respectively. The fusion computing module connects with the visual image module, ultrasonic processing module, and inertial sensing module to determine the disturbance level L1-L4 based on real-time visual degradation, ultrasonic error, and inertial zero bias, and then generates and outputs the disturbance rejection compensation vector and disturbance rejection state variables; The robust control module is connected to the fusion computing module and is used to replan the trajectory in real time and feed forward compensation based on the disturbance compensation vector and disturbance state variables, and then output the mission feedback packet to the ground system. The degradation decision module is data-connected to the fusion computing module and the robust control module. It is used to trigger the degradation path and output an eight-bit degradation code when any sensing source fails.

2. The system according to claim 1, characterized in that, The ground system includes: The ground mission planning station is used to receive external inputs such as the cruise area, weather wind field, no-fly zone, and fire prior threshold, automatically generate flight path templates, mark potential disturbance sources, and package them into mission packages for output. The ground real-time monitoring station is used to receive task packages output by the ground planning task station in real time, and overlay them on the electronic map and video stream, so that the operator can monitor the disturbance status in real time and issue manual intervention instructions. The ground data post-processing center is used to store raw sensor streams and mission logs, automatically compare four types of indicators: wind disturbance attitude error, visual availability, ultrasonic error, and GNSS lockout duration, generate disturbance assessment reports and push updated parameters via OTA, and synchronize the data transmitted back from the ground real-time monitoring station. The ground-based emergency intervention terminal is used to send commands to the drone for forced hovering, forced return to home, or emergency landing when disturbance indicators exceed limits or airborne degradation is triggered. It also displays the remaining battery power, the nearest safe point, and the predicted wind speed in real time to reduce the risk of uncontrolled disturbances.

3. The system according to claim 1, characterized in that, The visual imaging module includes an RGB global shutter camera, an event camera, an image processing unit, and a feature extraction and sharpness evaluation unit; The ultrasonic processing module includes an ultrasonic transmitter, an ultrasonic receiver, a signal processing unit, and a ranging calculation unit. The inertial sensing module includes a three-axis accelerometer, a three-axis gyroscope, a signal processing unit, and a data fusion unit; The fusion computing module includes a data interface unit, a fault detection unit, a disturbance observer, an adaptive fusion unit, and a disturbance rejection compensation vector generation unit. The robust control module includes a data receiving unit, a trajectory replanning unit, a feedforward compensation unit, a redundant action allocation unit, a control command generation unit, and a communication interface. The degradation decision module includes a sensor health detection unit, a fault determination unit, a degradation strategy unit, and an instruction output unit.

4. The system according to claim 1, characterized in that, The fusion computing module outputs the disturbance level to the robust control module, the degradation decision module, and the ground system. The disturbance levels L1-L4 are risk scales that the system classifies from low to high according to the total real-time residual variance: L1 is when there is no disturbance in a light wind and the weights are fully open, and the MPC is relaxed. From L2 to L3, visual weights are reduced and control is tightened as the disturbance intensifies; at L4, extreme disturbances immediately trigger a downgrade or return to base to ensure flight safety.

5. The system according to claim 1, characterized in that, The fusion computing module generates anti-interference compensation vectors and anti-interference state variables in real time. The calculation formula for the anti-interference compensation vector is as follows: ; Where Δ is the anti-interference compensation vector; These are adaptive weights, which are the inverse variances of the output residuals from the vision, ultrasound, and inertial sensors, respectively. These are visual degradation, ultrasonic error, and inertial zero bias, respectively. The disturbance rejection state variables are obtained by the fusion calculation module through one-step updating via Extended Kalman Filter (EKF), and the calculation formula for the disturbance rejection state variables is as follows: ; in, These are disturbance rejection state variables; The system transition matrix; For control matrix; This is the previous control input; Kalman gain; For observation vectors; For observation functions.

6. The system according to claim 1, characterized in that, The robust control module outputs a mission feedback packet to the ground system, including fire point coordinates, and generates a fire spread grid and disturbance confidence labels.

7. A method for anti-disturbance flight of unmanned aerial vehicles based on multi-source fusion perception, using the system described in any one of claims 1-6, characterized in that, include: The ground mission planning station receives external inputs such as the cruise area, weather wind field, no-fly zone, and fire prior threshold, generates a flight path template and marks potential disturbance sources, and then combines the generated template and disturbance sources into a mission package and sends it to the airborne system. After receiving the mission packet, the airborne system initiates the flight mission. The visual imaging module, ultrasonic processing module, and inertial sensing module output the visual degradation, ultrasonic error, and inertial zero bias, respectively. The fusion computing module receives visual degradation, ultrasonic error and inertial zero bias, and performs fault detection, disturbance observer estimation and adaptive fusion to generate disturbance immunity compensation vector and disturbance immunity state variables, and determines disturbance levels L1-L4. The fusion computing module then outputs the disturbance level, disturbance immunity compensation vector and disturbance immunity state variables to the robust control module, degradation decision module and ground system. The degradation decision module monitors the disturbance level in real time. When any sensor data exceeds the threshold, the degradation decision module triggers the degradation path and outputs an eight-bit degradation code. After receiving the degradation code, the robust control module adjusts the sensor weights and then replans the trajectory based on the disturbance level, disturbance rejection compensation vector, and disturbance rejection state quantity output by the fusion calculation module. The fusion computing module maps the fire point pixels detected by the visual image module and the anti-disturbance state quantity into fire point coordinates in real time, generates a fire line spread grid and disturbance confidence label, and transmits them back to the ground real-time monitoring station through the robust control module. The ground real-time monitoring station receives mission feedback packets in real time and displays the coordinates of the fire point and the fire spread grid. The ground data post-processing center stores the raw sensor stream and mission log, compares various indicators, and if any indicator exceeds the limit, it pushes the updated parameters to the airborne system via OTA. The ground data post-processing center pushes updated parameters to the airborne system via OTA. The airborne system reassesses the disturbance level and adjusts the control parameters every 60 seconds before entering the next mission cycle.

Images