An unmanned aerial vehicle obstacle avoidance method, system, device and storage medium

Abstract

The present application relates to the technical field of unmanned aerial vehicle control, and particularly relates to an unmanned aerial vehicle obstacle avoidance method, system, device and storage medium, comprising obtaining laser radar point cloud data, image data, millimeter wave radar data and environmental thermal imaging data; performing anti-reflection suppression processing on the image data to obtain filtered image data; respectively performing preprocessing on the laser radar point cloud data, the millimeter wave radar data and the environmental thermal imaging data to obtain obstacle point cloud data, distance and speed data of the obstacle and obstacle contour information; performing data fusion on the filtered image data, the obstacle point cloud data, the distance and speed data of the obstacle and the obstacle contour information to obtain obstacle fusion data; constructing a dynamic cost map according to the obstacle fusion data and preset environmental map data; correcting a preset flight path according to the dynamic cost map to obtain a corrected flight path; and eliminating ice and snow reflection interference by using multi-source data to improve the obstacle avoidance accuracy of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control, and more particularly to a UAV obstacle avoidance method, system, device, and storage medium. Background Technology

[0002] Currently, most mainstream drone obstacle avoidance solutions rely on single or combined sensors such as visual sensors, LiDAR, and millimeter-wave radar. In normal environments, these solutions can achieve relatively ideal obstacle avoidance results. However, in icy and snowy environments, the reflection of ice and snow can severely interfere with visual sensors.

[0003] In icy and snowy environments, snow and ice surfaces cause strong specular and diffuse reflections, resulting in severely overexposed images, lost details, and blurred or disappeared feature points. This makes it difficult to effectively distinguish reflective areas from real obstacles, potentially leading to missed or false detections. Meanwhile, while LiDAR-based solutions are unaffected by reflections, in high-density snowfall, snowflakes scatter and interfere with the laser beam, reducing detection accuracy. Therefore, in reflective icy and snowy environments, drones suffer from low obstacle avoidance accuracy and poor reliability, limiting their application in polar exploration, snow disaster relief, and winter power line inspection.

[0004] It is evident that existing technologies still need improvement and enhancement. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a method, system, device and storage medium for drone obstacle avoidance, which utilizes multi-source data to eliminate interference from reflective areas of ice and snow, thereby improving the obstacle avoidance accuracy of drones during flight.

[0006] The first aspect of this invention provides an obstacle avoidance method for unmanned aerial vehicles (UAVs), comprising: acquiring lidar point cloud data, image data, millimeter-wave radar data, and environmental thermal imaging data; performing reflection suppression processing on the image data to obtain filtered image data; preprocessing the lidar point cloud data, millimeter-wave radar data, and environmental thermal imaging data respectively to obtain obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information; fusing the filtered image data, obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information to obtain obstacle fused data; constructing a dynamic cost map based on the obstacle fused data and preset environmental map data; and correcting a preset flight path based on the dynamic cost map to obtain a corrected flight path.

[0007] Optionally, in a first implementation of the first aspect of the present invention, the step of performing reflection suppression processing on the image data to obtain filtered image data includes: dividing the image data into multiple local sub-regions according to a preset size; obtaining the mean gray level and gray level variance of each local sub-region, and calculating an adaptive threshold for each local sub-region based on the mean gray level and gray level variance of each local sub-region; comparing the gray level of each pixel in the local sub-region with the corresponding adaptive threshold to obtain overexposed pixel regions and effective pixel regions; the overexposed pixel regions are pixel regions with gray level values ​​greater than the adaptive threshold; the effective pixel regions are pixel regions with gray level values ​​less than or equal to the adaptive threshold; performing gray level suppression processing on the overexposed pixel regions to obtain repaired pixel regions; adjusting the contrast of the effective pixel regions to obtain enhanced pixel regions; and generating filtered image data based on the repaired pixel regions and enhanced pixel regions.

[0008] Optionally, in a second implementation of the first aspect of the present invention, the preprocessing of the lidar point cloud data, millimeter-wave radar data, and environmental thermal imaging data to obtain obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information includes: using a pass-through filtering algorithm to filter out false point clouds caused by snowflake scattering in the lidar point cloud data to obtain obstacle point cloud data; extracting obstacle distance and velocity data from the millimeter-wave radar data according to preset extraction fields; and using a histogram equalization algorithm to enhance the temperature difference features between the obstacle and the icy and snowy environment in the environmental thermal imaging data to extract obstacle contour information.

[0009] Optionally, in a third implementation of the first aspect of the present invention, the step of fusing filtered image data, obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information to obtain obstacle fusion data includes: determining the center coordinates and contour range of the obstacle based on the filtered image data and obstacle contour information to obtain first obstacle center coordinates and first contour range information; determining the center coordinates and contour range of the obstacle based on the obstacle point cloud data to obtain second obstacle center coordinates and second contour range information; determining the center coordinates of the obstacle based on the obstacle distance and velocity data to obtain third obstacle center coordinates; performing weighted fusion of the first obstacle center coordinates, second obstacle center coordinates, and third obstacle center coordinates based on a preset first confidence weight coefficient to obtain fused center coordinates; performing weighted fusion of the first contour range information and second contour range information based on a preset second confidence weight coefficient to obtain fused contour range information; and determining the obstacle fusion data; wherein the obstacle fusion data includes fused center coordinates and fused contour range information.

[0010] Optionally, in a fourth implementation of the first aspect of the present invention, the step of constructing a dynamic cost map based on obstacle fusion data and preset environmental map data includes: determining the fusion center coordinates and fusion contour range information of obstacles based on obstacle fusion data; marking the corresponding coordinates in the preset environmental map data based on the fusion center coordinates to obtain marked coordinates; and constructing an obstacle warning area on the preset environmental map data based on the fusion contour range information, using the marked coordinates as the origin, to form a dynamic cost map.

[0011] Optionally, in a fifth implementation of the first aspect of the present invention, the step of correcting the preset flight path according to the dynamic cost map to obtain a corrected flight path includes: marking the coordinates and flight direction of each flight node of the UAV on the dynamic cost map according to the preset flight path to form a reference flight path; comparing the reference flight path with the obstacle warning area; if the distance between the reference flight path and the obstacle warning area is less than or equal to a preset safe distance, then using the obstacle warning area as the search origin, searching for a safe area on the dynamic cost map; and correcting the preset flight path according to the safe area to obtain a corrected flight path.

[0012] Optionally, in a sixth implementation of the first aspect of the present invention, the step of correcting the preset flight path according to the safe area to obtain the corrected flight path includes: if the number of safe areas is greater than 1, obtaining the coordinates of two adjacent flight nodes that are close to the obstacle warning area; calculating the total distance between the coordinates of the two flight nodes and the center of each safe area, and taking the safe area with the shortest total distance as the detour area; and correcting the preset flight path according to the detour area to obtain the corrected flight path.

[0013] A second aspect of the present invention provides an obstacle avoidance system for unmanned aerial vehicles (UAVs), comprising: an acquisition module for acquiring lidar point cloud data, image data, millimeter-wave radar data, and environmental thermal imaging data; a suppression module for performing reflection suppression processing on the image data to obtain filtered image data; a preprocessing module for preprocessing the lidar point cloud data, millimeter-wave radar data, and environmental thermal imaging data respectively to obtain obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information; a fusion module for fusing the filtered image data, obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information to obtain obstacle fused data; a construction module for constructing a dynamic cost map based on the obstacle fused data and preset environmental map data; and a correction module for correcting a preset flight path based on the dynamic cost map to obtain a corrected flight path.

[0014] A third aspect of the present invention provides a drone obstacle avoidance device, the drone obstacle avoidance device comprising: a memory and at least one processor, the memory storing instructions; at least one processor calling the instructions in the memory to cause the drone obstacle avoidance device to perform each step of any of the drone obstacle avoidance methods described above.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of any of the above-described unmanned aerial vehicle obstacle avoidance methods.

[0016] In the technical solution of this invention, in environments with reflective ice and snow, visual sensors may malfunction due to overexposure, but lidar and thermal imaging data can still provide reliable information. In snowy weather, lidar may be interfered with by snowflakes, at which point millimeter-wave radar and visual information can serve as important supplements. This greatly enhances the system's perception robustness in complex and harsh environments. It does not simply superimpose information from different sensors, but uses a weighted fusion approach to integrate the advantages of multi-source data to obtain more comprehensive and accurate fused data regarding obstacle positions and contours. This fusion strategy effectively overcomes the limitations of a single sensor in specific scenarios, thus significantly improving the accuracy and reliability of obstacle detection. The obstacle fusion data is transformed into a navigation map marked with areas of varying danger levels. This allows for clear identification of safe areas in the environment during path planning, providing a good reference for safe obstacle avoidance. When the original path is found to conflict with or be too close to a high-risk area, a new path can be planned that safely bypasses the obstacle and gets as close as possible to the mission objective. This achieves a complete closed loop from environmental perception to flight control, enabling drones to adjust their flight trajectory according to the real-time environment, ultimately effectively avoiding collision risks and ensuring flight safety. Attached Figure Description

[0017] Figure 1 A flowchart of an obstacle avoidance method for unmanned aerial vehicles provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the unmanned aerial vehicle obstacle avoidance system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the drone obstacle avoidance device provided in an embodiment of the present invention. Detailed Implementation

[0018] This invention provides a method, system, device, and storage medium for obstacle avoidance in unmanned aerial vehicles (UAVs). In reflective environments like ice and snow, visual sensors may fail due to overexposure, but LiDAR and thermal imaging data can provide reliable information. During snowfall, LiDAR may be affected by snowflakes, allowing millimeter-wave radar and visual information to supplement it. This enhances the system's robustness in complex and harsh environments. The system does not simply superimpose information from different sensors; instead, it employs a weighted fusion approach, integrating the advantages of multi-source data to obtain more comprehensive and accurate fused data regarding obstacle positions and outlines. This fusion strategy overcomes the limitations of single sensors, improving the accuracy and reliability of obstacle detection. The fused obstacle data is transformed into a navigation map marking hazardous areas, facilitating the identification of safe zones during path planning and providing a basis for obstacle avoidance. If the original path conflicts with or is too close to a high-risk area, a new path can be planned to safely bypass the obstacle and approach the mission objective. This achieves a complete closed loop from environmental perception to flight control, enabling the UAV to adjust its trajectory according to the real-time environment, avoid collision risks, and ensure flight safety.

[0019] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the drone obstacle avoidance method in this invention includes: 101. Acquire lidar point cloud data, image data, millimeter-wave radar data, and environmental thermal imaging data; In this embodiment, the lidar directly acquires high-precision three-dimensional distance and shape information of surrounding objects by emitting laser beams and measuring their return time, forming point cloud data. The visual camera captures rich two-dimensional texture and color information to identify the specific type of obstacle. The millimeter-wave radar effectively measures distance and velocity by emitting radio waves and analyzing the echoes, and has strong penetration in adverse weather conditions such as rain, snow, and fog. The environmental thermal imager detects infrared radiation from object surfaces, generating temperature distribution images, independent of ambient lighting conditions. This multi-source data acquisition strategy achieves data complementarity. In reflective environments like ice and snow, the visual sensor may fail due to overexposure, but lidar and thermal imaging data can still provide reliable information; while in snowy weather, lidar may be affected by snowflake interference, making millimeter-wave radar and visual information important supplements. This greatly improves the system's perception robustness in complex and harsh environments.

[0021] 102. Perform reflection suppression processing on the image data to obtain filtered image data; In this embodiment, regions with different brightness levels in the image data are first identified to determine overexposed areas. By performing grayscale compression or repair on the overexposed areas, the negative impacts of detail loss and feature blurring caused by overexposure are mitigated, providing a more reliable data foundation for subsequent image-based obstacle recognition and reducing the risk of false detection and missed detection due to image quality issues.

[0022] 103. Preprocess the lidar point cloud data, millimeter-wave radar data, and environmental thermal imaging data respectively to obtain obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information; In this embodiment, scattered point clouds generated by temporary tiny particles such as snowflakes falling from the air are identified and filtered out from the lidar point cloud data, retaining the stable point cloud of the real obstacles. For millimeter-wave radar data, the distance and radial velocity of the obstacle relative to the UAV are directly extracted by analyzing its echo signal. For environmental thermal imaging data, algorithms such as histogram equalization are used to enhance the temperature difference contrast of different objects in the image, making the outlines of obstacles (especially living objects or objects with temperature differences from the snowy background) clearer and more distinguishable. Preprocessing effectively reduces noise interference in the raw data and highlights the most valuable obstacle avoidance information (such as the shape, distance, velocity, and outline of the object) in various types of data, improving the purity and usability of the information.

[0023] 104. Perform data fusion on filtered image data, obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information to obtain obstacle fusion data; In this embodiment, data fusion is performed at different levels. For example, the position and outline of an obstacle can be initially determined based on visual and thermal imaging data, while its three-dimensional shape and position can be confirmed based on LiDAR point cloud data. Then, its distance and speed can be verified using millimeter-wave radar data. The system assigns a confidence weight to the reliability of different sensors under different environmental conditions, and then performs weighted fusion. Through fusion, the accuracy and reliability of obstacle detection can be significantly improved. It effectively overcomes the limitations of a single sensor in a specific scenario, achieving a more accurate estimation of the obstacle's position and range, and providing a more reliable basis for safe path planning.

[0024] 105. Construct a dynamic cost map based on obstacle fusion data and preset environmental map data; In this embodiment, obstacle fusion data is overlaid onto a preset environmental map (such as a global grid map or elevation map). Centered on the obstacle, areas with different levels of danger are marked on the grid map according to its outline and a preset safety margin, thus forming a cost map that dynamically changes over time or with sensor updates. The areas containing obstacles are typically assigned extremely high passage costs. This dynamic cost map transforms abstract obstacle information into intuitive spatial cost information that is easy for path planning algorithms to process. It enables the UAV to identify which areas in the environment are safe and which need to be avoided, providing a direct basis for real-time, safe path decision-making.

[0025] 106. Correct the preset flight path according to the dynamic cost map to obtain the corrected flight path.

[0026] In this embodiment, the UAV compares its preset flight path with a dynamic cost map. If a conflict or excessive proximity between the planned path and a high-cost obstacle warning area is detected, the flight path will be corrected. Using the current path point as a reference, it searches for nearby safe areas on the cost map and replans a new path that bypasses the obstacle warning area while remaining as close as possible to the original planned route or mission objective. This ultimately achieves a closed loop from perception to action. It enables the UAV to adjust its flight trajectory based on real-time environmental perception, effectively avoiding collision risks, ensuring flight safety, and successfully completing flight missions in complex environments.

[0027] In this embodiment of the invention, in icy and reflective environments, visual sensors may lose functionality due to overexposure; however, lidar and thermal imaging data can still provide reliable information. In snowy weather conditions, lidar may be interfered with by snowflakes, at which point millimeter-wave radar and visual information can serve as important supplements. This significantly enhances the system's perception robustness in complex and harsh environments. It is not a simple superposition of information from different sensors, but rather a weighted fusion approach that integrates the advantages of multi-source data to obtain more comprehensive and accurate fused data regarding obstacle positions and outlines. This fusion strategy effectively overcomes the limitations of a single sensor in specific scenarios, thereby significantly improving the accuracy and reliability of obstacle detection. Transforming the fused obstacle data into a navigation map marked with areas of varying danger levels allows for clear identification of safe zones in the environment during path planning, providing a solid basis for safe obstacle avoidance. When the original path is found to conflict with or be too close to a high-risk area, a new path can be planned that safely bypasses the obstacle and gets as close as possible to the mission objective. This achieves a complete closed loop from environmental perception to flight control, enabling the UAV to adjust its flight trajectory according to the real-time environment, ultimately effectively avoiding collision risks and ensuring flight safety.

[0028] In the second embodiment of the drone obstacle avoidance method of the present invention, step S102 includes: 201. Divide the image data into multiple local sub-regions according to the preset size; 202. Obtain the mean gray level and gray level variance of each local sub-region, and calculate the adaptive threshold of each local sub-region based on the mean gray level and gray level variance of each local sub-region; In this embodiment, the image data is divided into smaller local sub-regions based on a preset size (e.g., a 16x16 or 32x32 pixel square). The lighting conditions in different areas of an image may vary significantly. In icy or snowy environments, reflections may be concentrated in only a few local areas of the image, rather than being uniformly distributed. For each sub-region, the mean grayscale value and variance of all pixels within it are calculated. The mean grayscale value reflects the overall brightness level of the region, while the variance describes the dispersion (i.e., contrast) of the pixel grayscale values ​​within the region. Based on these two statistics, an adaptive threshold algorithm is used to calculate a unique adaptive threshold for that sub-region. The formula is as follows: Adaptive threshold = mean gray level - k Gray variance Where k is an adjustable parameter (can be set to 0.02), which makes the threshold increase accordingly in areas with high brightness and strong contrast.

[0029] This gives the threshold setting a high degree of flexibility and environmental awareness. It can dynamically adapt to the brightness characteristics of each small area, thereby more accurately distinguishing overexposed pixels from normal pixels and avoiding misjudgments that may occur when using a single global threshold.

[0030] 203. Compare the grayscale values ​​of each pixel in the local sub-region with the corresponding adaptive threshold to obtain the overexposed pixel region and the effective pixel region. The overexposed pixel region is a pixel region with a grayscale value greater than the adaptive threshold; the effective pixel region is a pixel region with a grayscale value less than or equal to the adaptive threshold. 204. Perform grayscale suppression processing on the overexposed pixel areas to obtain the repaired pixel areas; 205. Adjust the contrast of the effective pixel area to obtain an enhanced pixel area; 206. Generate filtered image data based on repaired and enhanced pixel regions.

[0031] In this embodiment, after determining the adaptive threshold for each local sub-region, the system iterates through each pixel within the sub-region, comparing its grayscale value with the adaptive threshold of its respective region. Based on the comparison results, pixels are divided into two categories: overexposed pixel regions (a set of pixels with grayscale values ​​greater than the adaptive threshold; these pixels typically lose detail due to excessive light) and effective pixel regions (a set of pixels with grayscale values ​​less than or equal to the adaptive threshold; these regions contain identifiable details in the image). This achieves accurate diagnosis and region segmentation of image content. For identified overexposed pixel regions, grayscale suppression processing is required to recover some details. For example, interpolation algorithms can be used to estimate and repair the reasonable grayscale of the overexposed region based on the grayscale values ​​of surrounding effective pixels; or the image can be converted to a color space such as HSV or LAB, and nonlinear compression (e.g., using logarithmic or gamma transformation) can be applied to the luminance channel separately to reduce the brightness of the highlight areas without significantly affecting the color. These methods effectively restore image details in the highlight areas. By selectively adjusting the grayscale of overexposed areas, the loss of detail caused by overexposure can be mitigated, making the outlines of obstacles in reflective areas faintly discernible. For effective pixel areas, the goal is not to reduce brightness, but to further enhance their contrast, making their details stand out more. This can be achieved through methods such as histogram equalization (stretching the grayscale histogram distribution of the area to cover a wider grayscale range), contrast stretching, or adaptive gamma correction. These methods can increase the grayscale difference between adjacent pixels, thus making edges, textures, and other features clearer; this improves the clarity and discernibility of details in normal areas. In icy and snowy environments, this is crucial for identifying non-reflective obstacles (such as dark rocks and trees) from complex backgrounds, providing a higher-quality data foundation for subsequent obstacle detection. Finally, the system recombines the repaired pixel areas and the enhanced pixel areas. During the recombination process, it is necessary to ensure a natural transition at the seams between areas, avoiding obvious processing boundaries, and generating a complete, improved quality filtered image.

[0032] In the third embodiment of the drone obstacle avoidance method of the present invention, step S103 includes: The scanning parameters include scanned images of multiple curved substrates; 301. A direct-pass filtering algorithm is used to filter out false point clouds caused by snowflake scattering in lidar point cloud data in order to obtain obstacle point cloud data; In this embodiment, pass-through filtering is a fast point cloud filtering method based on spatial coordinate range. It retains only point cloud data falling within a preset range by setting threshold ranges for one or more coordinate axes in three-dimensional space. Specifically, in icy and snowy environments, the spatial distribution characteristics of point clouds generated by snowflake scattering are analyzed (for example, false point clouds caused by snowflakes are often sparsely distributed and randomly located), thereby setting a reasonable filtering range. Through pass-through filtering, these irregularly distributed false point clouds generated by snowflake scattering can be effectively distinguished and filtered out, retaining point cloud data representing real obstacles. It can effectively eliminate a large number of noise points generated by environmental factors such as snowflake scattering, significantly improving the signal-to-noise ratio of point cloud data and providing a cleaner and more reliable data foundation for subsequent obstacle recognition and tracking.

[0033] 302. Extract the distance and velocity data of obstacles from millimeter-wave radar data according to preset extraction fields; In this embodiment, the millimeter-wave radar transmits electromagnetic waves of a specific waveform (such as a linear frequency modulated continuous wave) and receives the echo signals reflected by obstacles. Using a Fast Fourier Transform (FFT), key characteristic frequency components containing information about the obstacle's distance and radial velocity are extracted from the echo signals. Preset extraction fields are frequency or data fields containing specific information. The system calculates the relative distance and relative velocity between the obstacle and the drone based on these fields.

[0034] 303. Histogram equalization algorithm is used to enhance the temperature difference features between obstacles and the snow and ice environment in environmental thermal imaging data in order to extract the outline information of obstacles.

[0035] In this embodiment, the grayscale distribution (histogram) of the original environmental thermal imaging image is remapped to ensure that the grayscale distribution of the output image is as uniform as possible across the entire grayscale range. In icy and snowy environments, the surface temperature of obstacles (such as rocks, trees, and buildings) often differs from the icy and snowy background, but this temperature difference may appear as a weak grayscale contrast in the original thermal image. Histogram equalization can significantly stretch and enhance this grayscale difference caused by temperature differences, making the outlines of warmer or colder obstacles exceptionally clear relative to the uniform icy and snowy background. This facilitates subsequent image segmentation or contour extraction algorithms, effectively enhancing the thermal contrast (temperature difference feature) between obstacles and the cold icy and snowy background. As a result, obstacles that may be difficult to identify in visible light images due to reflection, overexposure, or blurring appear with clearer outlines in thermal imaging.

[0036] In the fourth embodiment of the drone obstacle avoidance method of the present invention, step S104 includes: 401. Determine the center coordinates and contour range of the obstacle based on the filtered image data and obstacle contour information to obtain the center coordinates and contour range information of the first obstacle; 402. Determine the center coordinates and outline range of the obstacle based on the obstacle point cloud data, and obtain the center coordinates and outline range information of the second obstacle; 403. Determine the center coordinates of the obstacles based on their distance and speed data, and obtain the center coordinates of the third obstacle; In this embodiment, filtered image data provides rich texture and detail, while enhanced thermal imaging data clearly delineates the boundaries between obstacles and the background (especially in environments with significant temperature differences, such as ice and snow). The system uses image recognition algorithms (such as edge detection and contour extraction) and thermal imaging contour information to calculate the center coordinates (first obstacle center coordinates) and outer boundary (first contour range information) of the obstacle. Similarly, using three-dimensional spatial calculation techniques, the geometric center (second obstacle center coordinates) and three-dimensional dimensions (second contour range information) of the obstacle are determined from the obstacle point cloud data. Based on the obstacle's distance and velocity data, the distance and radial velocity of the UAV relative to the obstacle can be obtained, thereby calculating the obstacle's position coordinates in the current coordinate system (third obstacle center coordinates).

[0037] 404. Based on the preset first confidence weight coefficient, the center coordinates of the first obstacle, the center coordinates of the second obstacle, and the center coordinates of the third obstacle are weighted and fused to obtain the fused center coordinates; In this embodiment, a preset first confidence weight coefficient is assigned to each set of coordinates. This weight coefficient is dynamic or preset, reflecting the reliability of different sensors in the current specific environment. For example, under strong light, the weights of lidar and millimeter-wave radar may be higher; while at night, the weight of thermal imaging may be increased. Finally, the weighted coordinates are synthesized to obtain the optimal fusion center coordinates. Further, the confidence weights of each sensor are set according to the characteristics of the ice and snow environment: the weight of the second obstacle center coordinate is set to 60%-70%, the weight of the third obstacle center coordinate is set to 20%-25%, and the weight of the first obstacle center coordinate is set to 5%-10%; through weight allocation, data complementarity is achieved, and false detection data from a single sensor (such as false obstacles misjudged due to reflection of the first obstacle center coordinate) is eliminated, outputting accurate obstacle fusion data.

[0038] To aid understanding, consider the following example: The x-parameter of the second obstacle's center coordinates is 8 (weight 65%), the x-parameter of the third obstacle's center coordinates is 7.8 (weight 25%), and the x-parameter of the first obstacle's center coordinates is 8.2 (weight 10%). The final merged x-parameter = 8 × 0.65 + 7.8 × 0.25 + 8.2 × 0.1 = 7.97. Similarly, the remaining y-parameters and z-parameters are calculated in the same way to obtain the merged obstacle center coordinates.

[0039] 405. The first contour range information and the second contour range information are weighted and fused based on the preset second confidence weight coefficient to obtain the fused contour range information; In this embodiment, a preset second confidence weighting coefficient is used to perform weighted fusion of the first contour range information obtained from visual / thermal imaging and the second contour range information obtained from laser point cloud, resulting in more accurate fused contour range information. It should be noted that both the first and second contour range information refer to the coordinate points of the obstacle's outer contour. This fusion generates a more complete and accurate description of the obstacle's shape. Laser point cloud provides precise geometric dimensions, while visual information may contain details (such as mesh-like structures) that are difficult to distinguish from point cloud data. Combining the advantages of both helps the UAV more accurately determine the actual area occupied by the obstacle and the passable gaps, providing a more detailed basis for safe path planning. The weighted fusion method is the same as in step 404 and will not be repeated here.

[0040] 406. Determine obstacle fusion data; the obstacle fusion data includes fusion center coordinates and fusion contour range information.

[0041] In this embodiment, the fusion center coordinates and fusion contour range information are combined to form unified obstacle fusion data that includes spatial location and geometric dimensions.

[0042] In the fifth embodiment of the drone obstacle avoidance method of the present invention, step S105 includes: 501. Determine the coordinates of the fusion center and the fusion contour range of the obstacles based on the obstacle fusion data; 502. Mark the corresponding coordinates in the preset environmental map data according to the fusion center coordinates to obtain the marked coordinates; 503. Using the marked coordinates as the origin, construct an obstacle warning area on the preset environmental map data based on the fused contour range information to form a dynamic cost map.

[0043] In this embodiment, the location information of obstacles is associated with a global environment map. On a preset environment map data (e.g., a grid map or topology map with the initial position of the drone as the origin), map coordinates corresponding to the coordinates of the fusion center are found and marked, thus obtaining the marked coordinates. This marked point serves as the geometric center or reference origin for subsequently constructing the warning area. Using the marked coordinates as the origin, an obstacle warning area is constructed on the preset environment map based on the fusion contour range information. For example, for a rectangular obstacle, the warning area may extend beyond its contour by a safety boundary (e.g., 50 cm), forming a warning area slightly larger than the physical obstacle. Each grid (or point) within this area is assigned a different cost value based on its distance from the obstacle; the closer the distance, the higher the cost and the greater the passage risk. This ultimately forms an intuitive and machine-understandable spatial risk map. The dynamic cost map quantifies the existence and risk level of obstacles in the environment into grid cost information that can be directly processed by the path planning algorithm. This enables drones to clearly identify which areas in the environment are safe (low cost) and which areas are dangerous (high cost / unlimited cost), thus providing the most direct decision-making basis for generating safe and smooth flight paths.

[0044] In the sixth embodiment of the drone obstacle avoidance method of the present invention, step S106 includes: 601. Mark the coordinates and flight direction of each flight node of the UAV on the dynamic cost map according to the preset flight path to form a reference flight path; 602. Compare the reference flight path with the obstacle warning area. If the distance between the reference flight path and the obstacle warning area is less than or equal to the preset safe distance, then use the obstacle warning area as the search origin and search for the safe area on the dynamic cost map. 603. Correct the preset flight path according to the safe zone to obtain the corrected flight path.

[0045] In this embodiment, the system compares the spatial range of the reference flight path with the obstacle warning areas marked on the dynamic cost map. The system calculates the Euclidean distance between each point on the path and the boundary of the nearest obstacle warning area. Once a segment of the path is identified as being less than or equal to a system-preset safe distance (e.g., 1 meter) from a warning area, that area is determined to be a risk area. Subsequently, the system uses this risk warning area as the origin (or center) and searches outwards on the dynamic cost map for low-cost, passable areas, i.e., safe areas. Once a safe area is found, the original preset flight path is locally adjusted or replanned. If multiple safe areas are found, the algorithm calculates the total distance from the center of each area to the nodes before and after the risk path segment, selecting the safe area with the shortest total distance as the optimal detour point to ensure the overall efficiency of the path. Finally, the system generates a corrected flight path that bypasses the original risk area and passes through the safe area.

[0046] In the seventh embodiment of the drone obstacle avoidance method of the present invention, step S603 includes: 701. If the number of safe zones is greater than 1, obtain the coordinates of two adjacent flight nodes that are close to the obstacle warning zone; 702. Calculate the total distance between the coordinates of the two flight nodes and the center of each safe zone, and take the safe zone with the shortest total distance as the detour area; 703. Correct the preset flight path according to the detour area to obtain the corrected flight path.

[0047] In this embodiment, when the system detects that the distance between the preset flight path and the obstacle warning area is too close, and more than one safe area is found on the dynamic cost map, a multi-objective decision-making process is triggered. The system first locates the coordinates of the two flight nodes closest to the risk obstacle warning area on the original preset flight path. These two node coordinates define the decision interval for path correction (one node represents the point where the drone is about to approach the risk area, and the other node represents the point where it is expected to leave the risk area). Simultaneously, the system extracts the geometric center coordinates of all identified safe areas (i.e., passable and low-cost areas) as candidate detour target points. The system calculates the sum of the geometric distances of the broken-line paths from the first flight node to the center of the safe area, and then from that center to the second flight node. The system calculates such a total distance value for each safe area. By comparing the total distances corresponding to all safe areas, the system selects the safe area with the shortest total distance as the final detour area.

[0048] The above describes the drone obstacle avoidance method in the embodiments of the present invention. The following describes the drone obstacle avoidance system in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 One embodiment of the drone obstacle avoidance system in this invention includes: The acquisition module 801 is used to acquire lidar point cloud data, image data, millimeter-wave radar data, and environmental thermal imaging data. The suppression module 802 is used to perform reflection suppression processing on the image data to obtain filtered image data; The preprocessing module 803 is used to preprocess the lidar point cloud data, millimeter-wave radar data and environmental thermal imaging data respectively to obtain obstacle point cloud data, obstacle distance and velocity data and obstacle contour information. The fusion module 804 is used to fuse filtered image data, obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information to obtain obstacle fused data. Module 805 is used to construct a dynamic cost map based on obstacle fusion data and preset environmental map data; The correction module 806 is used to correct the preset flight path according to the dynamic cost map to obtain the corrected flight path.

[0049] above Figure 2 The UAV obstacle avoidance system in this embodiment of the invention is described in detail from the perspective of modular functional entities. The UAV obstacle avoidance device in this embodiment of the invention is described in detail from the perspective of hardware processing.

[0050] Figure 3 This is a schematic diagram of the structure of a drone obstacle avoidance device 900 provided in an embodiment of the present invention. The drone obstacle avoidance device 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 933 or data 932. The memory 920 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the drone obstacle avoidance device 900. Furthermore, the processor 910 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the drone obstacle avoidance device 900 to implement the steps of the drone obstacle avoidance method provided in the above-described method embodiments.

[0051] The drone obstacle avoidance device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input / output interfaces 960, and / or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The structure of the drone obstacle avoidance device shown does not constitute a limitation on the drone obstacle avoidance device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the drone obstacle avoidance method.

[0053] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system or system / unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0054] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for obstacle avoidance by unmanned aerial vehicles (UAVs), characterized in that, Including the following steps: Acquire lidar point cloud data, image data, millimeter-wave radar data, and environmental thermal imaging data; The image data is subjected to reflection suppression processing to obtain filtered image data; The lidar point cloud data, millimeter-wave radar data, and environmental thermal imaging data are preprocessed to obtain obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information. Data fusion is performed on filtered image data, obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information to obtain obstacle fused data; A dynamic cost map is constructed based on obstacle fusion data and pre-set environmental map data; The preset flight path is corrected based on the dynamic cost map to obtain the corrected flight path.

2. The obstacle avoidance method for unmanned aerial vehicles according to claim 1, characterized in that, The process of performing reflection suppression processing on image data to obtain filtered image data includes: The image data is divided into multiple local sub-regions according to a preset size; The mean and variance of gray levels of each local sub-region are obtained, and the adaptive threshold of each local sub-region is calculated based on the mean and variance of gray levels of each local sub-region. The grayscale values ​​of each pixel within a local sub-region are compared with the corresponding adaptive threshold to obtain overexposed pixel regions and effective pixel regions; the overexposed pixel regions are pixel regions with grayscale values ​​greater than the adaptive threshold; the effective pixel regions are pixel regions with grayscale values ​​less than or equal to the adaptive threshold. Grayscale suppression processing is applied to the overexposed pixel areas to obtain the repaired pixel areas; Contrast adjustment is performed on the effective pixel area to obtain an enhanced pixel area; Filtered image data is generated based on repaired and enhanced pixel regions.

3. The obstacle avoidance method for unmanned aerial vehicles according to claim 1, characterized in that, The preprocessing of lidar point cloud data, millimeter-wave radar data, and environmental thermal imaging data to obtain obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information includes: A pass-through filtering algorithm is used to filter out false point clouds caused by snowflake scattering in lidar point cloud data in order to obtain obstacle point cloud data; The distance and velocity data of obstacles are extracted from millimeter-wave radar data based on preset extraction fields. Histogram equalization algorithm is used to enhance the temperature difference features between obstacles and the icy and snowy environment in environmental thermal imaging data in order to extract the outline information of obstacles.

4. The obstacle avoidance method for unmanned aerial vehicles according to claim 1, characterized in that, The process of fusing filtered image data, obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information to obtain obstacle fused data includes: The center coordinates and outline range of the obstacle are determined based on the filtered image data and obstacle outline information, thus obtaining the center coordinates and outline range information of the first obstacle; Based on the obstacle point cloud data, determine the center coordinates and outline range of the obstacle, and obtain the center coordinates and outline range information of the second obstacle. The center coordinates of the obstacle are determined based on the distance and speed data of the obstacle, thus obtaining the center coordinates of the third obstacle. The center coordinates of the first obstacle, the center coordinates of the second obstacle, and the center coordinates of the third obstacle are weighted and fused based on a preset first confidence weight coefficient to obtain the fused center coordinates; The first contour range information and the second contour range information are weighted and fused based on the preset second confidence weight coefficient to obtain the fused contour range information. Determine obstacle fusion data; the obstacle fusion data includes fusion center coordinates and fusion contour range information.

5. The obstacle avoidance method for unmanned aerial vehicles according to claim 4, characterized in that, The construction of a dynamic cost map based on obstacle fusion data and preset environmental map data includes: Determine the coordinates of the fusion center and the range of the fusion outline of the obstacles based on the obstacle fusion data; The coordinates in the preset environmental map data are marked according to the coordinates of the fusion center to obtain the marked coordinates; Using the marked coordinates as the origin, an obstacle warning area is constructed on the preset environmental map data based on the fused contour range information to form a dynamic cost map.

6. The obstacle avoidance method for unmanned aerial vehicles according to claim 5, characterized in that, The step of correcting the preset flight path based on the dynamic cost map to obtain the corrected flight path includes: Based on the preset flight path, mark the coordinates and flight direction of each flight node of the UAV on the dynamic cost map to form a reference flight path; The reference flight path is compared with the obstacle warning area. If the distance between the reference flight path and the obstacle warning area is less than or equal to the preset safe distance, the safe area is searched on the dynamic cost map with the obstacle warning area as the search origin. The preset flight path is corrected based on the safe zone to obtain the corrected flight path.

7. The obstacle avoidance method for unmanned aerial vehicles according to claim 6, characterized in that, The step of correcting the preset flight path based on the safe zone to obtain the corrected flight path includes: If the number of safe zones is greater than 1, obtain the coordinates of two adjacent flight nodes that are close to the obstacle warning zone; Calculate the total distance between the coordinates of the two flight nodes and the center of each safe zone, and take the safe zone with the shortest total distance as the detour area; The preset flight path is corrected based on the detour area to obtain the corrected flight path.

8. An obstacle avoidance system for unmanned aerial vehicles (UAVs), characterized in that, include: The acquisition module is used to acquire lidar point cloud data, image data, millimeter-wave radar data, and environmental thermal imaging data. The suppression module is used to perform reflection suppression processing on image data to obtain filtered image data; The preprocessing module is used to preprocess the lidar point cloud data, millimeter-wave radar data and environmental thermal imaging data respectively to obtain obstacle point cloud data, obstacle distance and velocity data and obstacle contour information. The fusion module is used to fuse filtered image data, obstacle point cloud data, obstacle distance and velocity data, and obstacle contour information to obtain obstacle fused data. The building module is used to construct a dynamic cost map based on obstacle fusion data and preset environmental map data; The correction module is used to correct the preset flight path based on the dynamic cost map to obtain the corrected flight path.

9. An obstacle avoidance device for unmanned aerial vehicles (UAVs), characterized in that, The drone obstacle avoidance device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the drone obstacle avoidance device to perform the steps of the drone obstacle avoidance method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the UAV obstacle avoidance method as described in any one of claims 1-7.

Images