An unmanned aerial vehicle intelligent path planning method and system

Abstract

The application discloses an unmanned aerial vehicle intelligent path planning method and system, which is applied to a human target search and rescue task in a forest environment under rain and snow weather and comprises the following steps: S1, collecting real-time image data through a multi-modal perception module carried on the unmanned aerial vehicle, including collecting visible light images, thermal infrared images and environmental meteorological data, and performing rain and snow degradation removal processing on the visible light images to reconstruct clear images, wherein the thermal infrared images are used for enhancing human body heat source identification capability in a low-temperature rain and snow environment; and the application has the advantages that a dynamic risk weight W changing with time is innovatively introduced, transient risks such as branch swinging and snow shedding are uniformly processed with static obstacles, the three-dimensional grid map has the timeliness at the moment, and the unmanned aerial vehicle can realize real-time path planning in flight in combination with an improved A-star algorithm, so that the collision probability and the planning frequency are obviously reduced.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) path planning technology, specifically to an intelligent path planning method and system for UAVs. Background Technology

[0002] With the rapid development of drone technology, its application in emergency rescue, disaster search and rescue, border patrol and other fields is becoming increasingly widespread. Especially when performing personnel search missions in forest environments, drones have become important auxiliary tools due to their high mobility, wide visual field and rapid deployment. However, in complex forest environments, especially in rainy and snowy weather conditions, drones are prone to more problems when performing search and rescue missions.

[0003] Patent CN117970943A discloses an AI-based path planning method and system for building inspection drones, relating to the field of variable regulation and control. It sets starting and ending coordinates, calculates a set of regional influence levels, and uses a genetic algorithm to determine the optimal path. By comprehensively considering these factors, this invention can minimize the possibility of drone malfunctions due to external influences during flight. By dividing the flight area, it can more accurately predict the occurrence of influencing factors, better ensuring the normal flight of the drone. The use of a genetic algorithm improves the efficiency and accuracy of determining the optimal flight path, thereby increasing the overall efficiency of building inspection. Simultaneously, image analysis algorithms are used during drone flight to identify obstacles, enabling the drone to avoid them in time and preventing collisions. However, the obstacle avoidance in the above-mentioned technical solutions only identifies fixed buildings, and existing path planning algorithms are mostly based on ideal weather conditions, without fully considering the aerodynamic changes in rain and snow, such as lift reduction, wind disturbance, dynamic obstacles of tree canopy, and the impact of flight altitude on perception range and obstacle avoidance ability. Moreover, when applied to search and rescue operations, although thermal imaging can penetrate rain and snow, the temperature ranges of animals and humans overlap, which will result in a certain false alarm rate. Therefore, we propose an intelligent path planning method and system for UAVs. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0005] The present invention addresses the aforementioned technical problems using the following technical solutions: Firstly, it provides an intelligent path planning method for unmanned aerial vehicles (UAVs), applied to human target search and rescue missions in forest environments during rain and snow. The method includes the following steps: S1, using a multimodal perception module mounted on the UAV to collect real-time image data, including visible light images, thermal infrared images, and environmental meteorological data; and performing rain and snow degradation removal processing on the visible light images to reconstruct a clear image, wherein the thermal infrared images are used to enhance the ability to identify human heat sources in low-temperature rain and snow environments; S2, based on the acquired real-time image data of the UAV's flight environment, constructing a dynamic environment model using an onboard visual sensor, and fusing lidar point clouds with visual depth estimation. S3. Generate a 3D raster map M of the forest and assign a dynamic risk weight W to each raster cell N; S4. Based on the improved A-star algorithm, perform path search in combination with the dynamic risk weight W to generate the optimal flight path; S5. During flight, perform pixel-level registration of the acquired visible light image and thermal infrared image to obtain a registration map P; S6. Perform human-animal semantic segmentation on the registration map P to obtain a human probability map H and an animal probability map A, and calculate the human-animal contrast ratio CHA to confirm highly credible human beings; S7. When the highly credible human being is detected, the UAV enters a hovering confirmation mode, and the altitude decreases slightly to the final confirmation altitude. After confirmation, the target coordinates are sent to the ground station by initiating secondary verification of thermal imaging and visible light zoom recognition.

[0006] Preferably, the rain and snow degradation removal process in S1 includes: establishing a rain and snow imaging degradation model, expressed as follows: V = L ⊙ (1 - I) + O where: V is the visible light image, L is the reconstructed clear image, I is the raindrop mask, and O is the snowflake scattering noise; a dual-branch rain and snow removal network is used, including a physical branch based on the atmospheric scattering model and a data-driven branch based on the frequency domain attention module of Transformer, to output a degradation-free image C.

[0007] Preferably, the dynamic risk weight W mentioned in S2 changes over time, reflecting the dynamics of tree branch swaying and snow shedding, and identifying flyable areas and obstacle areas. The expression is as follows: W=g+α·sin(2πf·t+φ)+β·|∇vt| Where: g is the static obstacle weight, such as tree trunks and rocks; α is the sway amplitude coefficient, which is related to the flexibility of tree species; f is the natural frequency of tree branches, which is determined by tree species and wind speed; φ is the phase shift; β is the wind speed change sensitivity coefficient; and ∇vt is the wind speed time gradient.

[0008] Preferably, the expression for the human-animal contrast CHA in S5 is: CHA = (ΣH – ΣA) / (ΣH + ΣA + 0.01). When CHA ≥ 0.3 and the area of ​​the continuous domain H is greater than 0.7m... 2 At that time, the area is marked as a highly reliable human body.

[0009] Preferably, the final confirmed altitude expression in S6 is: Z = T - Δz where: Z is the final confirmed altitude, T is the optimal recognition altitude calculated according to the thermal imaging resolution model, i.e. the optimal flight altitude, and Δz is a small reduction that is manually set or adaptively determined based on experience.

[0010] Preferably, the optimal flight altitude T is dynamically calculated using the thermal imaging resolution model as follows: T = argmax_z [ GSD(z) ≤ GSD_th ] where: the ground sampling distance GSD(z) = (s·z) / (f·p), s is the sensor pixel size, f is the focal length, p is the number of pixels, and GSD_th is the maximum GSD threshold required to identify a human body, set to 0.05m.

[0011] Secondly, a drone intelligent path planning system is provided, comprising: a multimodal perception module, which is mounted on the drone to collect real-time image data, including visible light images, thermal infrared images, and environmental meteorological data, and to perform rain and snow degradation removal processing on the visible light images to reconstruct clear images, wherein the thermal infrared images are used to enhance the ability to identify human heat sources in low-temperature rain and snow environments; and a dynamic weighting module, which constructs a dynamic environment model through an airborne visual sensor based on the real-time image data of the drone's flight environment, fuses lidar point clouds and visual depth estimation to generate a forest 3D raster map M, and assigns a dynamic risk weight W to each raster unit N; The path search module, based on the improved A-satellite algorithm and combined with the dynamic risk weight W, performs path search to generate the optimal flight path; the human-animal contrast module performs pixel-level registration of the acquired visible light image and thermal infrared image during flight to obtain a registration map P, performs human-animal semantic segmentation on the registration map P to obtain a human probability map H and an animal probability map A, and calculates the human-animal contrast ratio CHA to confirm highly credible human beings; the hovering confirmation and target transmission module, when the highly credible human being is detected, the UAV enters a hovering confirmation mode, lowers its altitude slightly to the final confirmation altitude, and initiates secondary verification by thermal imaging and visible light zoom recognition, and after confirmation, sends the target coordinates to the ground station.

[0012] Compared with the prior art, the present invention provides an intelligent path planning method and system for unmanned aerial vehicles (UAVs), which has the following beneficial effects: This invention relates to an intelligent path planning method and system for unmanned aerial vehicles (UAVs). It employs a dual-branch rain and snow removal network and thermal infrared enhancement. The physical branch estimates raindrop and snowflake degradation parameters based on an atmospheric scattering model, while the data-driven branch restores texture details through a Transformer frequency domain attention module, simultaneously outputting a clear, degradation-free image. The thermal infrared branch enhances the human body's heat source signal in low-temperature rain and snow environments, effectively solving the problem of visibility issues. Furthermore, it innovatively introduces a time-varying dynamic risk weight W=g+α·sin(2πf·t+φ)+β·|∇vt| to handle transient risks such as swaying branches and falling snow in a unified manner with static obstacles, giving the 3D raster map real-time visibility. Combined with an improved A-satellite algorithm, the UAV can perform real-time path planning during flight, significantly reducing collision probability and planning frequency.

[0013] In the target confirmation stage, two discrimination methods, human-animal contrast (CHA) and hovering secondary verification, are used to reduce false alarms in thermal imaging and improve search and rescue effectiveness from the outset. Human probability map H and animal probability map A are obtained simultaneously through pixel-level registration and semantic segmentation. Using CHA = (ΣH–ΣA) / (ΣH +ΣA + 0.01), when CHA exceeds the threshold and the continuous domain area meets the condition, the region is marked as a highly reliable human. Subsequently, the UAV automatically enters the hovering confirmation mode. The optimal recognition height T is calculated by T = argmax_z [ GSD(z) ≤ GSD_th ], and the UAV actively descends Δz to obtain a sufficient ground sampling distance, thereby improving thermal imaging resolution and target recognition accuracy while reducing collision risk. Attached Figure Description

[0014] Figure 1 A flowchart of an intelligent path planning method for unmanned aerial vehicles provided by the present invention; Figure 2 The present invention provides a module structure diagram of an intelligent path planning system for unmanned aerial vehicles. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1 Please see Figure 1As shown, this embodiment discloses an intelligent path planning method for unmanned aerial vehicles (UAVs) applied to human target search and rescue missions in forest environments under rain and snow conditions. The method includes the following steps: S1, using a multimodal perception module mounted on the UAV to collect real-time image data, including visible light images, thermal infrared images, and environmental meteorological data. The visible light images are processed to remove rain and snow degradation, reconstructing a clear image. The thermal infrared images are used to enhance the ability to identify human heat sources in low-temperature rain and snow environments. S2, based on the acquired real-time image data of the UAV's flight environment, a dynamic environment model is constructed using an onboard visual sensor. This model is then fused with lidar point clouds and visual depth estimation to generate a three-dimensional forest grid map M, and each grid cell N is assigned a dynamic risk weight W. S3. Based on the improved A-satellite algorithm, combined with dynamic risk weight W, path search is performed to generate the optimal flight path; S4. During flight, the acquired visible light image and thermal infrared image are registered at the pixel level to obtain the registration map P; S5. Human-animal semantic segmentation is performed on the registration map P to obtain the human probability map H and the animal probability map A, and the human-animal contrast ratio CHA is calculated to confirm the highly credible human body; S6. When a highly credible human body is detected, the UAV enters the hovering confirmation mode, and the altitude decreases slightly to the final confirmation altitude. After confirmation, the target coordinates are sent to the ground station by initiating secondary verification of thermal imaging and visible light zoom recognition.

[0016] In this embodiment, the rain and snow degradation removal process in S1 includes: establishing a rain and snow imaging degradation model, with the following expression: V = L ⊙ (1 - I) + O where: V is the visible light image, L is the reconstructed clear image, I is the raindrop mask, and O is the snowflake scattering noise; a dual-branch rain and snow removal network is adopted, including a physical branch based on the atmospheric scattering model and a data-driven branch based on the frequency domain attention module of Transformer, and the output is a degradation-free image C.

[0017] Specifically, the two branches share a 1×1 dimensionality-reduced convolution, reducing the computational load by 30%. After TensorRT INT8 quantization, it achieves a frame rate of ≥28 fps on the NVIDIA Jetson Orin Nano with a power consumption of <12 W, meeting the real-time requirements of airborne systems.

[0018] In this embodiment, the dynamic risk weight W in S2 changes over time, reflecting the dynamics of tree branch swaying and snow shedding, and identifying the flyable area and obstacle area. The expression is as follows: W=g+α·sin(2πf·t+φ)+β·|∇vt| where: g is the static obstacle weight, such as tree trunk and rock; α is the sway amplitude coefficient, which is related to the flexibility of the tree species; f is the natural frequency of the tree branch, which is determined by the tree species and wind speed; φ is the phase shift; β is the wind speed change sensitivity coefficient; and ∇vt is the wind speed time gradient.

[0019] Specifically, the static obstacle weight *g* is obtained by rasterizing the LiDAR point cloud at a resolution of 0.2 m after passing through the PCL library's PassThrough filter. Grids with a height > 0.5 m and a gradient > 45° are assigned *g* = 1.0, while the remaining spatial domains are assigned *g* = 0. Tree species frequencies are calibrated offline using a lookup table: birch *f* = 2.3 Hz, Scots pine *f* = 1.7 Hz, and spruce *f* = 1.4 Hz. The phase *φ* is recorded from the previous frame's grid state machine to ensure temporal continuity. The wind speed time gradient *∇vt* is determined using a 50 ms sliding window difference from the airborne ultrasonic anemometer, with β = 0.8. Experiments show that when |∇vt| > 3 m / s... 2 When the weight is increased by more than 2.4, obstacle avoidance can be effectively triggered 0.8 seconds in advance. The dynamic risk weight W and the grid map M are updated in lockstep on the FPGA with a cycle of 50 ms. It adopts 16-bit half-floating point, and the memory usage per frame is less than 3 MB. It shares with the A-star search thread through zero copy.

[0020] In this embodiment, the human-animal contrast CHA expression in S5 is: CHA = (ΣH – ΣA) / (ΣH + ΣA + 0.01). When CHA ≥ 0.3 and the area of ​​the continuous domain H is greater than 0.7m... 2 At that time, the area is marked as a highly reliable human body.

[0021] Specifically, pixel-level registration employs a mutual information (MI)-based registration method: Parzen window estimation is performed on 256-level histograms of visible light and thermal infrared, MI peak sub-pixel refinement is applied, registration error is <0.6 pixels, processing time is 6ms, GPU CUDA acceleration is used, 8000 rain and snow forest scene images are collected for training data, 1200 human bounding boxes and 3500 animal bounding boxes are manually labeled, and data augmentation includes random rain and snow overlay; after applying 3×3 Gaussian smoothing to the human probability map H and animal probability map A, a threshold of 0.2 is applied for binarization, and connected component analysis preserves an area >0.7m. 2 If a region (corresponding to thermal infrared 20 pixels@GSD=0.18m) has a CHA ≥ 0.3 and satisfies this criterion for 5 consecutive frames, it is marked as a highly reliable human body and proceeds to S6.

[0022] In this embodiment, the final confirmed altitude expression in S6 is: Z = T - Δz, where Z is the final confirmed altitude, T is the optimal recognition altitude calculated according to the thermal imaging resolution model, i.e., the optimal flight altitude, and Δz is a small decrease set manually or adaptively based on experience; the optimal flight altitude T is dynamically calculated through the thermal imaging resolution model and expressed as: T = argmax_z [ GSD(z) ≤ GSD_th ], where the ground sampling distance GSD(z) = (s·z) / (f·p), s is the sensor pixel size, f is the focal length, p is the number of pixels, and GSD_th is the maximum GSD threshold required to identify the human body, set to 0.05m.

[0023] Specifically, the default small descent Δz = 5m. If the current wind speed vs > 8m / s, Δz is adaptively reduced to 3m to ensure redundancy. The descent rate is limited to 0.5m / s. The lidar is pointed downwards for real-time ranging, and the minimum safe height above the ground is 8m. During the secondary verification, after hovering, a narrow FOV sub-window for thermal imaging is activated, with a central area of ​​64×64 pixels. The frame average is accumulated for 8 frames, and the NETD is improved to 35mK. Visible light is synchronously zoomed to the longest focal length, and YOLOv5-face is run for head and shoulder detection. A confidence score > 0.6 and an IoU with the thermal target > 0.5 indicate successful confirmation. The confirmation time is < 3s. Subsequently, the WGS-84 coordinates, confidence score, and thumbnail are transmitted back to the ground station via the 5GSA network to complete the closed loop.

[0024] Example 2 Please see Figure 2 As shown, based on the same inventive concept, this embodiment discloses an intelligent path planning system for unmanned aerial vehicles (UAVs). For details not covered in this embodiment, please refer to the relevant parts of Embodiment 1 above. The system includes: a multimodal perception module, mounted on the UAV, for acquiring real-time image data, including visible light images, thermal infrared images, and environmental meteorological data. The visible light images undergo rain and snow degradation removal processing to reconstruct a clear image. The thermal infrared images are used to enhance the ability to identify human heat sources in low-temperature rain and snow environments. A dynamic weighting module, based on the acquired real-time image data of the UAV's flight environment, constructs a dynamic environment model using an onboard visual sensor, fusing lidar point clouds and visual depth estimation to generate a three-dimensional forest grid. Figure M is generated, and each grid cell N is assigned a dynamic risk weight W. The path search module, based on the improved A-satellite algorithm, combines the dynamic risk weight W to perform path search and generate the optimal flight path. The human-animal contrast module performs pixel-level registration of the acquired visible light image and thermal infrared image during flight to obtain a registration map P. Human-animal semantic segmentation is performed on the registration map P to obtain a human probability map H and an animal probability map A. The human-animal contrast ratio CHA is calculated to confirm highly credible human beings. The hovering confirmation and target transmission module, when a highly credible human being is detected, the UAV enters a hovering confirmation mode, and the altitude decreases slightly to the final confirmation altitude. After confirmation, the target coordinates are sent to the ground station by initiating secondary verification through thermal imaging and visible light zoom recognition.

[0025] Specifically, in the multimodal perception module: Hardware: Visible light IMX678 + thermal infrared Tau2640 + meteorological sensor BME688; Driver: Custom V4L2 and GStreamer pipeline, timestamp hard synchronization <1 ms; Rain and snow degradation removal network resides on Orin Nano in ONNX form, GPU utilization <65%; Dynamic weight module: Point cloud-visual extrinsic parameters are calibrated offline via Kalibr, reprojection error 0.08 pixels; Raster map coverage 80m×80m, ROS2 topic / risk_map published, frequency 20Hz; Path search module: Improved A-star: JPS (Jump Point Search) + risk heuristic; safety radius 1.5m, maximum speed 10m / s, acceleration 2m / s². 2 If no solution is found within 10ms, RRT* replanning is triggered to ensure real-time performance; Human-animal contrast module: TensorRT engine accuracy FP16, batch=1, inference 7ms; output topics / human_prob and / animal_prob, synchronize CHA values; Hover confirmation and target sending module: hovering is achieved by sending SET_POSITION_TARGET_LOCAL_NED to PX4 via MAVROS; target coordinates are achieved using RTK-GPS fusion SLAM pose, with a horizontal error <0.1m.

[0026] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0027] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent path planning of unmanned aerial vehicles (UAVs), characterized in that, The method for searching and rescuing human targets in forest environments during rain and snow includes the following steps: S1. Real-time image data is collected using a multimodal perception module mounted on a drone, including visible light images, thermal infrared images, and environmental meteorological data. The visible light images are processed to remove rain and snow degradation and reconstruct clear images. The thermal infrared images are used to enhance the identification of human heat sources in low-temperature rain and snow environments. S2. Based on the real-time image data of the drone's flight environment, a dynamic environment model is constructed using an airborne visual sensor. A three-dimensional forest grid map M is generated by fusing lidar point clouds and visual depth estimation, and a dynamic risk weight W is assigned to each grid cell N. S3. Based on the improved A-satellite algorithm, a path search is performed using the dynamic risk weight W to generate the optimal flight path. S4. During flight, the acquired visible light images and thermal infrared images are registered at the pixel level to obtain a registration map P. S5. Human-animal semantic segmentation is performed on the registration map P to obtain a human probability map H and an animal probability map A. The human-animal contrast ratio (CHA) is calculated to identify highly reliable human subjects. The expression for the human-animal contrast ratio (CHA) in S5 is: CHA=(ΣH–ΣA) / (ΣH +ΣA + 0.01); When CHA≥0.3 and the area of ​​the continuous domain of H is greater than 0.7m², the region is marked as a highly reliable human body; S6, when the highly reliable human body is detected, the UAV enters the hovering confirmation mode, the altitude decreases by a small amount, which is the final confirmation altitude. After confirmation, the target coordinates are sent to the ground station by starting thermal imaging secondary verification and visible light zoom recognition.

2. The intelligent path planning method for unmanned aerial vehicles according to claim 1, characterized in that, The rain and snow degradation removal process described in S1 includes: establishing a rain and snow imaging degradation model, expressed as follows: V = L ⊙ (1 - I) + O; where: V is the visible light image, L is the reconstructed clear image, I is the raindrop mask, and O is the snowflake scattering noise; a dual-branch rain and snow removal network is used, including a physical branch based on the atmospheric scattering model and a data-driven branch based on the frequency domain attention module of Transformer, outputting a degradation-free image C.

3. The intelligent path planning method for unmanned aerial vehicles according to claim 1, characterized in that, The dynamic risk weight W described in S2 changes over time, reflecting the dynamics of tree branch swaying and snow shedding, and identifying flyable areas and obstacle areas. The expression is as follows: W=g+α·sin(2πf·t+φ)+β·|∇vt|; Where: g is the static obstacle weight, tree trunk, rock; α is the sway amplitude coefficient, which is related to the flexibility of the tree species; f is the natural frequency of the tree branch, which is determined by the tree species and wind speed; φ is the phase shift; β is the wind speed change sensitivity coefficient; and ∇vt is the wind speed time gradient.

4. The intelligent path planning method for unmanned aerial vehicles according to claim 1, characterized in that, The final confirmed altitude expression described in S6 is: Z = T - Δz; where: Z is the final confirmed altitude, T is the optimal recognition altitude calculated according to the thermal imaging resolution model, i.e. the optimal flight altitude, and Δz is a small descent that is manually set or adaptively determined based on experience.

5. The intelligent path planning method for unmanned aerial vehicles according to claim 4, characterized in that, The optimal flight altitude T is dynamically calculated using the thermal imaging resolution model and expressed as: T = argmax_z [ GSD(z) ≤ GSD_th ]; where: the ground sampling distance GSD(z) = (s·z) / (f·p), s is the sensor pixel size, f is the focal length, p is the number of pixels, and GSD_th is the maximum GSD threshold required to identify the human body, set to 0.05m.

6. An intelligent path planning system for unmanned aerial vehicles (UAVs), comprising an intelligent path planning method for UAVs according to any one of claims 1-5, characterized in that, include: A multimodal perception module, mounted on the UAV, is used to collect real-time image data, including visible light images, thermal infrared images, and environmental meteorological data. The visible light images undergo rain and snow degradation removal processing to reconstruct clear images. The thermal infrared images are used to enhance the ability to identify human heat sources in low-temperature rain and snow environments. A dynamic weighting module, based on the acquired real-time image data of the UAV's flight environment, constructs a dynamic environment model using an onboard visual sensor. It fuses LiDAR point clouds and visual depth estimation to generate a 3D forest raster map M, and assigns a dynamic risk weight W to each raster cell N. The path search module, based on the improved A-star algorithm and combined with the dynamic risk weight W, performs path search to generate the optimal flight path. The human-animal contrast module performs pixel-level registration of the acquired visible light and thermal infrared images during flight to obtain a registration map P. Human-animal semantic segmentation is then performed on the registration map P to obtain a human probability map H and an animal probability map A. The human-animal contrast ratio CHA is calculated to confirm highly reliable human subjects. The hovering confirmation and target transmission module, upon detecting the highly reliable human subject, enters a hovering confirmation mode, lowering its altitude slightly to the final confirmation altitude. It then initiates secondary verification using thermal imaging and visible light zoom recognition, and after confirmation, sends the target coordinates to the ground station.

Images