Unmanned aerial vehicle autonomous obstacle avoidance method and system based on multi-sensor fusion

Abstract

The invention provides an unmanned aerial vehicle autonomous obstacle avoidance method and system based on multi-sensor fusion. According to the method, three types of heterogeneous sensing data are acquired in the flight process of the unmanned aerial vehicle; establishing a data compensation rule according to the three types of data, and performing spatial interpolation compensation when the image data of the optical camera unit is missing; performing dynamic feature analysis on the linear obstacle based on the compensated sensing data to generate a three-dimensional obstacle topological graph; according to the real-time dynamic relation between the extension direction and the flight course in the topological graph, the variable quantity of the included angle between the extension direction and the flight course is converted into an avoiding direction adjustment weight, and a continuous deflection instruction is generated in combination with the space curvature and the distribution density parameter; and generating multiple sections of obstacle avoidance tracks based on the instruction, and performing closed-loop correction on the instruction by fusing millimeter wave distance data and infrared thermal characteristic data during execution. According to the invention, the dynamic avoidance precision and flight safety of the unmanned aerial vehicle to the linear obstacle are improved.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous obstacle avoidance for unmanned aerial vehicles (UAVs), and in particular to a method and system for autonomous obstacle avoidance for UAVs based on multi-sensor fusion. Background Art

[0002] In complex low-altitude scenarios like power line inspections and mountain logistics, drones must avoid long, thin linear obstacles like high-voltage power lines and cableways. These targets are characterized by their small diameter, low reflectivity, and strong spatial scalability. This requires the obstacle avoidance system to possess stable recognition capabilities in perception-restricted environments like rain, fog, and low light; the ability to dynamically model the geometric shape and spatial orientation of linear obstacles in real time; and a lightweight obstacle avoidance decision-making mechanism suitable for high-speed flight.

[0003] The current mainstream solution uses lidar point cloud clustering and visual semantic fusion technology: lidar scanning is used to generate an environmental point cloud, and Euclidean clustering is used to extract candidate areas for linear objects. Visible light cameras are simultaneously used for semantic segmentation to identify specific categories of targets such as wires and cables. The two types of results are aligned and fused through timestamps to construct a static obstacle map and plan a global obstacle avoidance path.

[0004] Existing solutions have the following flaws: LiDAR detects sparse point clouds for slender objects with small diameters, and vision fails in rain, fog, or low light, resulting in missed detections of linear targets. They rely on static maps with low update frequency and are unable to respond to the dynamic position changes of linear obstacles during high-speed flight. Global path planning ignores the extension characteristics of linear obstacles, often generating dangerous approaching trajectories parallel to the direction of power lines. Summary of the Invention

[0005] The present application provides a method and system for autonomous obstacle avoidance of UAVs based on multi-sensor fusion, which is used to solve the problem of insufficient accuracy of dynamic avoidance of linear obstacles by UAVs in the prior art.

[0006] In a first aspect, the present application provides a method for autonomous obstacle avoidance of a UAV based on multi-sensor fusion, comprising: During the flight of the drone, three types of heterogeneous sensing data are simultaneously acquired from the optical camera unit, millimeter wave detection unit, and infrared sensing unit;

[0007] A data compensation rule is established based on the differences in spatiotemporal coverage of the three types of heterogeneous sensing data. When the optical camera unit loses image data due to insufficient illumination, the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit are used to perform spatial interpolation compensation according to the data compensation rule.

[0008] Based on the compensated perception data, the system dynamically analyzes the linear obstacles in the flight path, extracts the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results, and generates an obstacle topology map in a three-dimensional spatial coordinate system with the current position of the drone as the origin.

[0009] According to the real-time dynamic relationship between the extension direction in the obstacle topology map and the current flight heading of the UAV, the change in the angle between the extension direction and the heading is converted into an avoidance direction adjustment weight, and combined with the spatial curvature and continuous distribution density parameters, a continuous deflection instruction of the obstacle avoidance heading is generated;

[0010] Based on the continuous deflection command, multiple obstacle avoidance trajectories are generated. When the drone executes the multiple obstacle avoidance trajectories, the distance data and the thermal feature distribution data are fused. The continuous deflection command is closed-loop corrected using the fused data. The correction process is repeated until the relative distance between the drone and the linear obstacle reaches a safety threshold.

[0011] Optionally, based on the compensated perception data, dynamic feature analysis is performed on linear obstacles in the flight path, extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results are extracted, and an obstacle topology map is generated in a three-dimensional spatial coordinate system with the current position of the drone as the origin, including:

[0012] Segmenting the compensated perception data into multiple independent spatial units, and screening out target units corresponding to the linear obstacles based on the geometric features of the spatial units in three-dimensional space;

[0013] Performing a main direction analysis on the target unit to determine the extension direction of the target unit in three-dimensional space, and performing continuity fitting on the extension direction according to a spatial adjacent relationship to generate the overall extension direction of the linear obstacle;

[0014] Calculating a curvature parameter of the linear obstacle in three-dimensional space as a spatial curvature according to an extension direction angle deviation value between adjacent target units, wherein the extension direction angle deviation value is determined by the main direction angle between adjacent target units;

[0015] The number distribution of the target units per unit volume in a three-dimensional spatial coordinate system with the current position of the drone as the origin is counted, a continuous distribution density parameter of the linear obstacle is generated based on the number distribution, and the overall extension direction, the spatial curvature, and the continuous distribution density parameter are mapped to the three-dimensional spatial coordinate system to form an obstacle topology map.

[0016] Optionally, the step of converting a change in the angle between the extension direction and the heading into an avoidance direction adjustment weight based on the real-time dynamic relationship between the extension direction in the obstacle topology map and the current flight heading of the UAV, and generating a continuous deflection instruction for the obstacle avoidance heading in combination with the spatial curvature and the continuous distribution density parameter, includes:

[0017] Obtaining in real time a real-time angle value between an extension direction in the obstacle topology map and the current flight direction of the UAV, and calculating a difference between the real-time angle value and the angle at the previous moment according to the extension direction as an angle change;

[0018] Generating a curvature influence coefficient and a density influence coefficient according to the spatial curvature and the continuous distribution density parameter, respectively, wherein the curvature influence coefficient is positively correlated with the spatial curvature, and the density influence coefficient is negatively correlated with the continuous distribution density parameter;

[0019] Inputting the angle variation, the curvature influence coefficient, and the density influence coefficient into a preset dynamic relationship evaluation parameter table, and determining the avoidance direction adjustment weight by looking up the table;

[0020] According to the avoidance direction adjustment weight, an avoidance direction component orthogonal to the extension direction is generated in a vertical plane of the current flight heading of the UAV, and the avoidance direction component is superimposed on the current heading to generate a continuous deflection instruction.

[0021] Optionally, generating multiple obstacle avoidance trajectories based on the continuous deflection instructions, fusing the distance data with the thermal signature distribution data when the UAV executes the multiple obstacle avoidance trajectories, performing closed-loop correction on the continuous deflection instructions using the fused data, and repeating the correction process until the relative distance between the UAV and the linear obstacle reaches a safety threshold, includes:

[0022] Decomposing the continuous deflection instruction into a trajectory segment parameter set, wherein the trajectory segment parameter set includes a heading angle change and a corresponding flight speed value;

[0023] When the UAV flies according to the trajectory segment parameter set, the distance data and the thermal signature distribution data are synchronously acquired, and the fusion credibility of the trajectory segment parameter set is determined according to the overlap ratio between the coverage range of the distance data and the intensity area of ​​the thermal signature distribution data;

[0024] Based on the fusion credibility, a deviation evaluation is performed on the heading angle change and the corresponding flight speed value, wherein when the coverage range is lower than a preset ratio during the evaluation process, the center point of the intensity area of ​​the thermal feature distribution data is preferentially used as a deviation evaluation benchmark;

[0025] Based on the deviation evaluation result, a reverse compensation adjustment is performed on the heading angle change and the flight speed value in the next trajectory segment parameter set. After the adjustment, the generation process of the trajectory segment parameter set is repeated until the minimum relative distance between the UAV and the linear obstacle in the distance data reaches the safety threshold.

[0026] Optionally, the data compensation rule is established based on the temporal and spatial coverage differences of the three types of heterogeneous sensing data. When the optical camera unit loses image data due to insufficient illumination, spatial interpolation compensation is performed using the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit according to the data compensation rule, including:

[0027] detecting a pixel grayscale value of a target area in the image data output by the optical camera unit, and marking a boundary of the target area as a missing area boundary when the pixel grayscale value is lower than a preset illumination threshold and the area of ​​a continuous pixel missing region of the target area exceeds a set ratio;

[0028] Calculating a joint coverage capability parameter based on a density distribution of distance measurement points of the millimeter wave detection unit within a boundary of the missing area and a distribution of thermal signature intensity of the infrared sensing unit in the missing area;

[0029] Based on the joint coverage capability parameter, spatial weight allocation is performed on the distance measurement points of the millimeter wave detection unit and the thermal characteristic intensity distribution points of the infrared sensing unit;

[0030] According to the allocation result, an interpolation surface is constructed within the boundary of the missing area with the distance measurement points as geometric constraints and the thermal feature intensity distribution points as attribute constraints, and the interpolation surface is mapped to the image data within the boundary of the missing area to complete spatial interpolation compensation.

[0031] Optionally, the real-time acquisition of a real-time angle value between an extension direction in the obstacle topology map and the current flight direction of the UAV, and the calculation of an angle difference between the real-time angle value and a previous moment according to the extension direction as an angle variation, includes:

[0032] Extracting the extension direction vector of the linear obstacle from the obstacle topology map, and synchronously obtaining the current flight heading vector output by the UAV flight control unit;

[0033] Calculating the angle between the plane projection of the extension direction vector and the flight heading vector in three-dimensional space as a real-time angle value;

[0034] Performing a difference operation on the real-time angle value and the historical angle value stored at the previous moment to obtain the angle change at adjacent moments;

[0035] When the real-time angle value is first acquired, the angle variation is initialized to zero, and the historical angle value is continuously updated as the current real-time angle value at subsequent moments.

[0036] Optionally, performing a main direction analysis on the target unit to determine an extension direction of the target unit in three-dimensional space, and performing continuity fitting on the extension direction according to a spatial adjacent relationship to generate an overall extension direction of the linear obstacle includes:

[0037] Performing a three-dimensional spatial distribution analysis on the geometric features of the target unit, and determining the longest extension axis direction of the target unit in the three-dimensional space as the main direction according to the analysis results;

[0038] establishing a direction association relationship between adjacent target units according to the main direction, and marking the adjacent target units with the direction association relationship as direction association units;

[0039] Performing consistency adjustment on the main directions of the direction association units, so that the main direction angles of the direction association units are gradually reduced in spatial adjacent order to within a preset allowable deviation range through the adjustment process;

[0040] The main directions of the adjusted direction association units are connected in spatial adjacent order to form a continuous extension direction of the linear obstacle as the overall extension direction.

[0041] In a second aspect, the present application provides a drone autonomous obstacle avoidance system based on multi-sensor fusion, comprising:

[0042] The acquisition module is used to synchronously acquire three types of heterogeneous sensing data from the optical camera unit, millimeter wave detection unit, and infrared sensing unit during the flight of the drone;

[0043] a compensation module, configured to establish data compensation rules based on the differences in spatiotemporal coverage of the three types of heterogeneous sensing data, and to perform spatial interpolation compensation using the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit according to the data compensation rules when image data of the optical camera unit is missing due to insufficient illumination;

[0044] The parsing module is used to perform dynamic feature analysis on linear obstacles in the flight path based on the compensated perception data, extract the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results, and generate an obstacle topology map in a three-dimensional spatial coordinate system with the current position of the drone as the origin;

[0045] a conversion module for converting a change in the angle between the extension direction and the current flight heading of the UAV in the obstacle topology map into an avoidance direction adjustment weight based on the real-time dynamic relationship between the extension direction and the current flight heading of the UAV, and generating a continuous deflection instruction for the obstacle avoidance heading by combining the spatial curvature and the continuous distribution density parameter;

[0046] A correction module is configured to generate multiple obstacle avoidance trajectories based on the continuous deflection instructions, fuse the distance data with the thermal signature distribution data when the UAV executes the multiple obstacle avoidance trajectories, perform closed-loop correction on the continuous deflection instructions using the fused data, and repeat the correction process until the relative distance between the UAV and the linear obstacle reaches a safety threshold.

[0047] In a third aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a drone autonomous obstacle avoidance method based on multi-sensor fusion as described in the first aspect above.

[0048] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements an autonomous obstacle avoidance method for a drone based on multi-sensor fusion as described in the first aspect.

[0049] In an embodiment of the present application, during the flight of a drone, three types of heterogeneous perception data are synchronously acquired from an optical camera unit, a millimeter wave detection unit, and an infrared sensing unit; a data compensation rule is established based on the temporal and spatial coverage differences of the three types of heterogeneous perception data. When the optical camera unit loses image data due to insufficient illumination, the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit are used for spatial interpolation compensation according to the data compensation rule; based on the compensated perception data, a dynamic feature analysis is performed on the linear obstacles in the flight path, and the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results are extracted, and a dynamic feature analysis is generated. An obstacle topology map is provided in a three-dimensional spatial coordinate system with the current position of the UAV as the origin; based on the real-time dynamic relationship between the extension direction in the obstacle topology map and the current flight heading of the UAV, the change in the angle between the extension direction and the heading is converted into an avoidance direction adjustment weight, and continuous deflection instructions for the obstacle avoidance heading are generated in combination with the spatial curvature and the continuous distribution density parameter; multiple obstacle avoidance trajectories are generated based on the continuous deflection instructions, and the distance data and the thermal feature distribution data are fused when the UAV executes the multiple obstacle avoidance trajectories. The continuous deflection instructions are closed-loop corrected using the fused data, and the correction process is repeated until the relative distance between the UAV and the linear obstacle reaches a safety threshold.

[0050] The technical solution of the present application has the following beneficial effects: solving the problem of blind spots in the perception of a single sensor in a complex environment, and providing a data basis for multimodal compensation; using the spatial complementarity of millimeter-wave and infrared data to generate complete environmental perception when optical data is missing; extracting the geometric characteristics of linear obstacles and constructing a dynamic spatial model that adapts to the posture of the UAV; generating obstacle avoidance headings in real time according to the extension characteristics of linear obstacles and the flight status to avoid the risk of parallel approach; and correcting obstacle avoidance trajectories in real time through multi-sensor data fusion to ensure the safety of dynamic avoidance.

[0051] Furthermore, the compensated perception data is segmented into spatial units, and linear obstacle target units are selected based on their geometric features. The principal directions of the target units are analyzed and continuously fitted to generate the overall extension direction. Spatial curvature is calculated based on the angles between the principal directions of adjacent units. The number of target units per unit volume is counted to generate a distribution density parameter. Finally, the obstacle topology map is mapped to a three-dimensional coordinate system. By implementing structured analysis of the geometric characteristics of linear obstacles, high-precision spatial topological feature input is provided for dynamic obstacle avoidance.

[0052] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A flowchart of a method for autonomous obstacle avoidance of a UAV based on multi-sensor fusion provided by the present application is shown;

[0055] Figure 2 A scenario diagram showing a method for autonomous obstacle avoidance of a UAV based on multi-sensor fusion provided by the present application is shown;

[0056] Figure 3 The present invention provides a schematic diagram of a multi-sensor fusion-based autonomous obstacle avoidance system for unmanned aerial vehicles.

[0057] Figure 4 A schematic structural diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0059] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0060] Existing LiDAR and vision fusion solutions face fundamental limitations when dealing with linear obstacle avoidance: their perception layer relies on a single physical property. When rain and fog weaken laser reflection or dim light obscures visual texture, the system lacks a cross-modal compensation mechanism, resulting in perception gaps and persistent missed detection of slender targets. The decision layer, which constructs a static map through timestamp alignment, cannot represent the dynamic spatial relationship between the drone and linear obstacles at high speeds in real time, causing delayed obstacle avoidance commands. The planning layer simplifies linear obstacles into point-like structures, ignoring the strong constraints imposed by their spatial extension on the avoidance direction, resulting in a dangerous approach trajectory parallel to the power lines. These three flaws collectively limit the reliability of obstacle avoidance in complex scenarios.

[0061] In response to the above-mentioned defects, this application proposes a drone obstacle avoidance method based on multimodal collaborative perception and linear feature directional decision-making: first, through the spatiotemporal complementarity of optical, millimeter wave, and infrared data, the millimeter wave distance points and infrared thermal radiation characteristics are automatically integrated for spatial compensation when optical failure occurs, eliminating perception blind spots; based on the compensation data, the geometric distribution characteristics of linear obstacles are analyzed in real time, and a dynamic topological model based on the drone's posture is constructed to achieve millisecond-level spatial relationship response; finally, avoidance instructions are dynamically generated based on the real-time angle change between the obstacle extension direction and the heading, and the curvature and distribution density parameters are combined to output obstacle avoidance trajectories that are perpendicular or cross the obstacle extension direction at a large angle. This method completely solves the problem of missed detection caused by perception faults, the response lag caused by static modeling, and the risk of parallel approach brought by homogeneous strategies, significantly improving the robustness and safety of linear obstacle avoidance in complex environments.

[0062] The technical solution of the present application can be applied to scenarios with linear obstacles such as cables or ropeways.

[0063] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0064] Figure 1 A flowchart of a method for autonomous obstacle avoidance of a drone based on multi-sensor fusion is provided for the embodiment of the present application. Figure 1 As shown, the method includes:

[0065] 101. During the flight of the drone, three types of heterogeneous sensing data are simultaneously acquired from the optical camera unit, millimeter wave detection unit and infrared sensing unit;

[0066] In the above scheme, the optical camera unit refers to a device that uses visible spectrum imaging to capture the geometric outline of linear obstacles. The millimeter wave detection unit refers to a sensor that emits 30-300GHz electromagnetic waves, and uses its ability to penetrate low-visibility media such as rain, fog, and dust to obtain the three-dimensional spatial position coordinates of the obstacle. The infrared sensing unit refers to a sensor that receives the thermal radiation of an object, and extracts its thermal radiation distribution data by identifying the temperature difference characteristics between transparent or low-reflectivity linear obstacles and the environmental background. The three types of heterogeneous perception data refer to the original information sets synchronously output by the above three sensors with different physical principles and different data dimensions, including optical image sequences, millimeter wave point cloud distance data, and infrared thermal imaging matrices.

[0067] In this embodiment, the hardware clock signal from the drone's flight control system is used to send synchronization acquisition instructions to the optical camera unit, millimeter wave detection unit, and infrared sensing unit, forcing the three sensors to start data capture at the same physical moment. For example, the optical unit captures an RGB image sequence at a 30fps frame rate, the millimeter wave unit transmits FMCW frequency-modulated waves at a 100Hz frequency and receives echoes, and the infrared unit generates raw thermal radiation data at 25Hz based on a microbolometer array, ensuring that the timing starting points of the multi-source data are consistent.

[0068] Next, physical properties are specifically extracted from the raw data from each sensor. The optical image sequence undergoes edge gradient analysis and morphological closing operations to extract the continuous pixel boundaries of linear obstacles, generating a binary mask outline. The millimeter-wave echo signal uses a fast Fourier transform to calculate the distance and azimuth of the scattering point. This is then combined with the drone's IMU pose data to convert it into a three-dimensional point cloud in a global coordinate system. The infrared raw data undergoes non-uniformity correction and temperature calibration, mapping grayscale values ​​to absolute temperature values ​​and outputting a temperature-spatial distribution matrix. For example, each row of the millimeter-wave point cloud matrix contains three-dimensional data for distance, azimuth, and pitch angle, while the infrared matrix elements correspond to actual temperatures ranging from -40°C to 150°C.

[0069] Next, the three types of data are aligned in space and time. Spatial alignment involves matching the millimeter-wave point cloud with the optical contour corners using an iterative closest point algorithm, and registering the infrared temperature matrix to the optical image coordinate system using an affine transformation. Temporal alignment is based on hardware timestamps, using cubic spline interpolation to compensate for sampling rate differences and unify them into a single time series. For example, a 100Hz millimeter-wave point cloud is interpolated onto a 30Hz optical data time axis, ensuring that all data at time t reflects the same instantaneous state. Finally, the spatiotemporally aligned heterogeneous data is packaged to generate a structured data packet for subsequent processing. For example, a data packet at time t contains a 512×512 optical binary mask (contour point set), an N×3 millimeter-wave point cloud matrix, and a 64×64 infrared temperature matrix. The spatial coordinate error between the three is less than 5cm, and the temporal deviation is less than 1ms.

[0070] 102. Establishing a data compensation rule based on the differences in spatiotemporal coverage of the three types of heterogeneous sensing data, such that when image data of the optical camera unit is missing due to insufficient illumination, spatial interpolation compensation is performed using the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit according to the data compensation rule;

[0071] Optionally, step 102 may specifically include the following steps:

[0072] 1021. Detecting a pixel grayscale value of a target area in the image data output by the optical camera unit, and when the pixel grayscale value is lower than a preset illumination threshold and an area of ​​a continuous pixel missing region in the target area exceeds a set ratio, marking a boundary of the target area as a missing region boundary;

[0073] 1022. Calculate a joint coverage capability parameter based on the density distribution of distance measurement points of the millimeter wave detection unit within the boundary of the missing area and the distribution of thermal feature intensity of the infrared sensing unit in the missing area.

[0074] 1023. Based on the joint coverage capability parameter, perform spatial weight allocation on the distance measurement points of the millimeter wave detection unit and the thermal characteristic intensity distribution points of the infrared sensing unit;

[0075] 1024. Construct an interpolation surface within the boundary of the missing area based on the allocation result, with the distance measurement points as geometric constraints and the thermal feature intensity distribution points as attribute constraints, and map the interpolation surface to the image data within the boundary of the missing area to complete spatial interpolation compensation.

[0076] In the above scheme, the missing area boundary refers to the outer contour coordinate set of the continuous blank area in the optical image where linear obstacles cannot be identified due to insufficient lighting. The distance measurement point density distribution represents the distribution of the number of effective detection points of the millimeter wave detection unit per unit area. The thermal feature intensity distribution refers to the spatial discrete point set of the surface temperature values ​​of linear obstacles captured by the infrared sensing unit. The joint coverage capability parameter is an indicator that quantifies the spatial complementarity of millimeter wave and infrared data in the missing area, and is determined by the distance point density and the uniformity of the thermal intensity distribution. The geometric constraint requires that the interpolated surface must pass through the spatial position of the millimeter wave distance measurement point. The attribute constraint requires that the temperature attribute value of the interpolated surface is consistent with the infrared thermal feature distribution point.

[0077] In this embodiment, first, step 1021 is used to determine if optical data is missing. The grayscale values ​​of all pixels within the preset wire detection frame are summed and divided by the total number of pixels to calculate the average grayscale of the region. Connected domains are scanned to mark continuous blank pixel blocks, and their area-to-detection frame ratio is calculated. When the average grayscale is less than 50 and the area-to-detection frame ratio is greater than 0.3, a boundary tracing algorithm is used to extract a closed polygon coordinate sequence along the outer edge of the blank pixels. For example, in a high-voltage wire scene, the average grayscale within the detection frame is 45, and the blank area of ​​20×30 pixels accounts for 33%. The boundary vertex coordinates are output as [(10,20), (10,50), (40,50), (40,20)].

[0078] Then, in step 1022, the complementarity of the multimodal data is calculated. A 1m×1m grid is created within the missing boundary, and the number of valid distance points in each grid is counted. The standard deviation of the temperature values ​​of all thermal feature points is calculated. Dynamic quantification is performed using the formula: joint coverage parameter = average number of grid points × (1 - temperature standard deviation / 15). For example, if the average number of grid points in a certain area is 5 points / ㎡ and the temperature standard deviation is 8°C, the parameter = 5 × (1 - 8 / 15) = 2.33.

[0079] At the same time, data weights are dynamically assigned in step 1023, dividing the missing area into 0.2m×0.2m subgrids. If there are ≥3 distance points within a subgrid, a geometric weight of 0.7 is assigned. If the subgrid temperature standard deviation is ≤5°C, an attribute weight of 0.3 is assigned. If all conditions are met, the total weight is 1.0. For example, if a subgrid contains 4 mmWave points (geometric weight 0.7) and a temperature standard deviation of 4°C (attribute weight 0.3), the total weight is 1.0.

[0080] Finally, in step 1024, the optical data is reconstructed. The 3D coordinates of the millimeter-wave distance points are used as nodes, and the interpolated surface is forced to pass through these spatial locations. The infrared temperature values ​​are used as control points to constrain the surface temperature distribution. The node data is combined according to the sub-grid weights, and a continuous surface is generated through radial basis function interpolation. The surface temperature values ​​are linearly converted to grayscale values, with each 1°C increase in temperature corresponding to a grayscale increase of 6. For example, if the 25°C position is mapped to grayscale 120, then the 28°C position is mapped to grayscale 138, and the wire outline is reconstructed.

[0081] In a real-world application, a drone inspecting a 500kV high-voltage power line at dusk detected an anomaly in the preset wire detection frame (coordinate range [100:300,150:350]) in the 640×480 image captured by the optical camera unit in step 1021. The system calculated the mean grayscale value of the pixels in this area by summing the grayscale values ​​of the 96,000 pixels in the region (∑ grayscale = 3,840,000) and dividing it by the total number of pixels to obtain a mean of 40, with a threshold of 50. The scan revealed a continuous 50×40 pixel blank block in the upper left corner, occupying 0.35% of the detection frame area, which is greater than the 0.3 threshold. Using the Moore-Neighbor boundary tracing algorithm, the vertex coordinates of the closed polygon [(100,200), (100,240), (150,240), (150,200)] were extracted along the outer edge of the blank pixels.

[0082] In step 1022, multimodal data analysis is performed within the marked boundary. The 200 m2 missing area is divided into 1 m × 1 m grids. The number of valid points in each grid is counted. For example, the grid (110-111, 210-211) contains 3 points, and the average density is calculated to be 4.2 points / m2. The temperature values ​​of 32 thermal feature points {26.3°C, 27.1°C, ..., 28.5°C} are extracted, and the standard deviation σ is calculated to be 7°C. The joint coverage parameter is 4.2 × (1-7 / 15) = 2.24, which quantifies the data complementarity.

[0083] Step 1023 divides the missing area into 5,000 subgrids of 0.2 m × 0.2 m. A geometric weight of 0.7 is assigned to the grids containing 6 distance points (110.0-110.2, 210.0-210.2). An attribute weight of 0.3 is assigned to the grids with a temperature standard deviation of 3°C (149.6-149.8, 239.6-239.8). A total weight of 1.0 is assigned to grids that meet all the conditions, such as (120.0-120.2, 220.0-220.2) containing 4 points and a standard deviation of 4°C.

[0084] Through step 1024, the millimeter wave point (110.1, 210.3, 15.2m) is used as the spatial anchor point to force the surface to pass through this position; the infrared point (149.7, 239.8) temperature of 28.2°C is used as the benchmark; radial basis function interpolation is used to generate a continuous surface according to the sub-grid weight, and the basis function φ(r) = r²log(r) is used; a linear relationship between temperature and grayscale is established with a slope k = 6 grayscale / °C and an intercept b = -30, 28.2°C is mapped to a grayscale of 141.6, and the original blank area is filled to reconstruct the wire shape.

[0085] The overall solution of step 102 above triggers multimodal compensation through a precise optical failure determination mechanism, constructs a dual-constraint surface based on the spatial distribution characteristics of the millimeter-wave point cloud and the physical properties of infrared thermal features, and restores the precise geometric shape and surface characteristics of linear obstacles such as high-voltage lines under strong light interference, ensuring the perception continuity of the obstacle avoidance system in complex environments.

[0086] 103. Based on the compensated perception data, perform dynamic feature analysis on linear obstacles in the flight path, extract the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results, and generate an obstacle topology map in a three-dimensional spatial coordinate system with the current position of the drone as the origin;

[0087] Optionally, step 103 may specifically include the following steps:

[0088] 1031. Segment the compensated perception data into multiple independent spatial units, and select target units corresponding to the linear obstacles based on geometric features of the spatial units in three-dimensional space.

[0089] 1032. Perform a main direction analysis on the target unit to determine the extension direction of the target unit in three-dimensional space, and perform continuity fitting on the extension direction according to a spatial adjacent relationship to generate the overall extension direction of the linear obstacle;

[0090] Among them, step 1032 may specifically include the following processes: performing a three-dimensional spatial distribution analysis on the geometric features of the target unit, and determining the longest extension axis direction of the target unit in the three-dimensional space as the main direction based on the analysis results; establishing a direction association relationship between adjacent target units based on the main direction, and marking the adjacent target units with the direction association relationship as direction association units; adjusting the main direction of the direction association unit for consistency, and through the adjustment process, making the main direction angle of the direction association unit gradually decrease in the spatial adjacent order to within the preset allowable deviation range; connecting the main directions of the adjusted direction association units in the spatial adjacent order to form the continuous extension direction of the linear obstacle as the overall extension direction.

[0091] 1033. Calculate a curvature parameter of the linear obstacle in three-dimensional space as a spatial curvature based on an extension direction angle deviation value between adjacent target units, wherein the extension direction angle deviation value is determined by an included angle between main directions of adjacent target units.

[0092] 1034. Count the number distribution of the target units per unit volume in a three-dimensional spatial coordinate system with the current position of the drone as the origin, generate a continuous distribution density parameter of the linear obstacle based on the number distribution, and map the overall extension direction, the spatial curvature, and the continuous distribution density parameter to the three-dimensional spatial coordinate system to form an obstacle topology map.

[0093] In the above scheme, a spatial unit refers to a cubic data block that divides the perception data into fixed volumes. A target unit refers to a spatial unit whose geometric form conforms to linear features. The extension direction represents the orientation vector of the main axis of the target unit in three-dimensional space. Continuity fitting refers to generating a global direction by constraining the smooth transition of the direction vectors of adjacent units. The extension direction angular deviation value is the absolute value of the angle between the main direction vectors of adjacent target units. The curvature parameter is quantified by the mean angular deviation. The continuous distribution density parameter is a spatial distribution function of the number of target units per unit volume.

[0094] In this embodiment, spatial segmentation and feature screening are first performed in step 1031: An octree structure is used to recursively subdivide the three-dimensional space, initially using 1m³ as the base voxel, and spatially partitioning the compensated millimeter-wave point cloud and infrared temperature data. When the number of point clouds within a voxel exceeds 50, segmentation is continued to a minimum of 0.125m³, ensuring that each spatial unit contains an independent geometric structure. The eigenvalues ​​of the point cloud covariance matrix within the unit are calculated (λ1 ≥ λ2 ≥ λ3). If the linearity index IL = (λ1-λ2) / λ1 > 0.8 and the aspect ratio λ1 / λ2 > 5:1, the unit is determined to be a linear target unit. For example, in a high-voltage line point cloud unit, λ1 = 15.2, λ2 = 1.8, λ3 = 0.5 → IL = (15.2-1.8) / 15.2 = 0.88 > 0.8, and the aspect ratio is 8.4:1, and the unit is marked as a target unit.

[0095] Subsequently, step 1032 completes directional analysis and global fitting: singular value decomposition (SVD) is performed on the target cell point cloud, and the right singular vector corresponding to the maximum singular value is taken as the extended direction vector v = (vx, vy, vz). The modulus is normalized to the unit vector. The target cells are connected in spatial order (e.g., A→B→C), and the dot product of the adjacent cell direction vectors, cosθ = vA·vB, is calculated. If |θ| < 15°, spherical linear interpolation (slerp) is used to calculate the intermediate direction: vmid = [sin((1-t)Ω)vA + sin(tΩ)vB] / sinΩ (where cosΩ = vA·vB and t = 0.5). For example, if the direction of high-voltage line cell A is (0.87, 0.50, 0) and cell B is (0.90, 0.44, 0), the intermediate direction (0.885, 0.47, 0) is interpolated to construct the overall direction.

[0096] At the same time, the spatial curvature characteristics are quantified by step 1033: the direction vector sequence of 5 consecutive adjacent target units {v1, v2, v3, v4, v5} is taken, and the angle between the adjacent vectors θᵢ=arccos(vᵢ·v i+1 ); calculate the moving average κ = (∑θᵢ) / 4 of the angle sequence to eliminate local fluctuations. For example, at a high-voltage line bend, θ1 = 5°, θ2 = 7°, θ3 = 9°, and θ4 = 11°, resulting in a curvature κ = (5+7+9+11) / 4 = 8°.

[0097] Finally, a dynamic topology map is constructed in step 1034: A 1m×1m×1m voxel grid is created with the drone's current position as the origin. The number of target cells Nvoxel within each voxel is counted. The overall extension direction vector, curvature value κ, and density value Nvoxel are associated with the voxel center coordinates P(x, y, z) and stored as a topology map node Node={P,v,κ,N}. For example, if the voxel (15,25,50) contains 6 target cells, the stored value is Node={(15,25,50),(0.71,0.71,0),8°,6}.

[0098] In actual applications, in the 500kV high-voltage line inspection scenario, the drone flies at an altitude of 50 meters to obtain compensated millimeter-wave point cloud and infrared temperature data. Through step 1031, octree recursive segmentation is used: the initial division is 1 meter³ voxel, and the number of point clouds in the detection unit (32.1, 45.3, 52.7) is 68>50 threshold; it is divided into 8 0.5 meter³ sub-units, of which the sub-units (32.1-32.6, 45.3-45.8, 52.7-53.2) contain high-voltage line point clouds; the eigenvalues ​​of the covariance matrix of the sub-unit are calculated: λ1=12.8,λ2=1.5,λ3=0.3; the linearity index IL=(12.8-1.5) / 12.8=0.88>0.8, and the aspect ratio λ1 / λ2=8.5:1>5:1; it is marked as the target unit, and the geometric center coordinates (32.35, 45.55, 52.95) are output.

[0099] Through step 1032, based on the target unit sequence output in step 1031: perform singular value decomposition (SVD) on the unit (32.35, 45.55, 52.95) point cloud, the maximum singular value corresponds to the right singular vector vA = (0.91, 0.41, 0.02); the direction vB of the adjacent unit (32.85, 46.05, 52.95) = (0.89, 0.45, 0.02); calculate the directional dot product: vA vB = 0.91 × 0.89 + 0.41 × 0.45 + 0.02 × 0.02 =0.998; angle θ=arccos(0.998)≈3.2°<15°, enable spherical linear interpolation: interpolation parameters t=0.5, Ω=3.2°vmid=[sin((1-0.5)×3.2°)×vA+sin(0.5×3.2°)×vB] / sin(3.2°), and obtain the intermediate direction (0.90, 0.43, 0.02); concatenate 10 sets of adjacent unit interpolation directions to generate the overall direction of the high-voltage line N25°E (direction vector (0.82, 0.57, 0)).

[0100] Through step 1033, five consecutive target cells are selected along the extension direction of the high-voltage line: the direction vector v1 of the cell coordinates (32.35, 45.55, 52.95) is (0.91, 0.41, 0.02), and the adjacent angle θ1 is arccos (v1·v2) = 3.2°; the direction vector v2 of the cell coordinates (32.85, 46.05, 52.95) is (0.89, 0.45, 0.02), and the adjacent angle θ2 is arccos (v2·v3) = 4.1°; the direction vector v3 of the cell coordinates (33.35, The direction vector v3 at the element coordinate (46.55,52.95) is (0.87,0.49,0.02), and the adjacent angle θ3 is arccos(v3·v4)=5.3°; the direction vector v4 at the element coordinate (33.85,47.05,52.95) is (0.85,0.53,0.02), and the adjacent angle θ4 is arccos(v4·v5)=6.8°; the direction vector v5 at the element coordinate (34.35,47.55,52.95) is (0.82,0.57,0.02). The spatial curvature parameter κ is (θ1+θ2+θ3+θ4) / 4=(3.2+4.1+5.3+6.8) / 4=4.85°.

[0101] Through step 1034, with the current position of the UAV (0, 0, 50) as the origin: establish a 1 meter³ voxel grid, with voxels (30-31, 45-46, 50-51) covering 5 target cells; statistical density parameter Nvoxel = 5; associated feature parameters: extension direction is the overall strike vector (0.82, 0.57, 0), spatial curvature κ = 4.85°, distribution density is 5 cells / meter³; generate a topology node: Node = {coordinates (30.5, 45.5, 50.5), direction (0.82, 0.57, 0), curvature 4.85°, density 5}.

[0102] The overall solution of step 103 above accurately extracts linear targets through recursive spatial segmentation and geometric feature constraints, generates continuous and smooth extension directions based on singular value decomposition and spherical interpolation, quantifies spatial curvature characteristics using moving average angles, and constructs a dynamic topology map in combination with voxelized density statistics, providing the UAV with a refined spatial feature description that is adaptable to complex linear obstacles such as high-voltage lines and cables.

[0103] 104. Based on the real-time dynamic relationship between the extension direction in the obstacle topology map and the current flight heading of the UAV, the change in the angle between the extension direction and the heading is converted into an avoidance direction adjustment weight, and combined with the spatial curvature and continuous distribution density parameters, a continuous deflection instruction for the obstacle avoidance heading is generated;

[0104] Optionally, step 104 may specifically include the following steps:

[0105] 1041. Obtain in real time a real-time angle between an extension direction in the obstacle topology map and the current flight direction of the UAV, and calculate a difference between the real-time angle and the angle at the previous moment based on the extension direction as an angle change;

[0106] Among them, step 1041 may specifically include the following processes: extracting the extension direction vector of the linear obstacle from the obstacle topology map, and synchronously obtaining the current flight heading vector output by the UAV flight control unit; calculating the plane projection angle between the extension direction vector and the flight heading vector in three-dimensional space as the real-time angle value; performing a difference operation on the real-time angle value and the historical angle value stored at the previous moment to obtain the angle change at adjacent moments; when the real-time angle value is obtained for the first time, initializing the angle change to zero, and continuously updating the historical angle value as the current real-time angle value at subsequent moments.

[0107] 1042. Generate a curvature influence coefficient and a density influence coefficient according to the spatial curvature and the continuous distribution density parameter, respectively, wherein the curvature influence coefficient is positively correlated with the spatial curvature, and the density influence coefficient is negatively correlated with the continuous distribution density parameter;

[0108] 1043. Input the angle variation, the curvature influence coefficient, and the density influence coefficient into a preset dynamic relationship evaluation parameter table, and determine the avoidance direction adjustment weight by looking up the table;

[0109] 1044. Adjust the weight according to the avoidance direction, generate an avoidance direction component orthogonal to the extension direction in a vertical plane of the current flight heading of the UAV, and superimpose the avoidance direction component with the current heading to generate a continuous deflection instruction.

[0110] In the above scheme, the real-time angle value refers to the angle between the plane projection of the extension direction vector and the flight heading vector at the current moment. The angle change is the algebraic difference between the current angle and the historical angle one second ago. The curvature influence coefficient is a linear function of the spatial curvature. The density influence coefficient is an inverse function of the distribution density. The dynamic relationship evaluation parameter table is a three-dimensional mapping table with angle change, curvature coefficient, and density coefficient as input dimensions and output weight values. The orthogonal avoidance component refers to the direction vector perpendicular to the extension direction and located in the heading perpendicular plane.

[0111] In the embodiment of the present application, first, step 1041 is used to calculate the dynamic angle change in real time, extract the three-dimensional extension direction vector vobj of the linear obstacle from the obstacle topology map, and synchronously read the current heading vector vuav of the drone output by the flight control system; project the two vectors onto the horizontal plane, set the Z component to 0, and calculate the plane projection angle according to the formula θt=arccos((vobjx·vuavx+vobjy·vuavy) / (√(vobjx²+vobjy²)×√(vuavx²+vuavy²))), where √ represents the square root, and the result is in degrees, 0°~180°. The historical angle value θ{t-1} stored in the previous control cycle is retrieved, and the algebraic difference Δθ=θt-θ{t-1} between the current angle and the previous value is calculated. A positive value indicates that the drone is approaching the extension direction of the obstacle, and a negative value indicates that it is moving away. For example, the extension direction vector of the high-voltage line is (0.82, 0.57, 0), the heading of the UAV is (0.87, 0.50, 0), and the projection calculation is θt=5.2°; the historical record is θ{t-1}=3.1°→Δθ=2.1°.

[0112] Next, in step 1042, the risk impact coefficient is quantified. The spatial curvature value κ of the topology node is read and the curvature impact coefficient is calculated using the linear relationship kc = 0.1 × κ. The greater the curvature, the greater the coefficient, indicating a greater avoidance margin due to sharp turns in the high-voltage line. The node distribution density D is read in units / m³ and the density impact coefficient is calculated using the inverse relationship kd = 5 / D. The higher the density, the smaller the coefficient, indicating a more cautious and smaller avoidance margin is required for densely packed cable clusters. For example, a high-voltage line node with κ = 4.85° will have kc = 0.485, and a density D = 5 units / m³ will have kd = 1.0.

[0113] Next, determine the avoidance weight in step 1043. Preset the three-dimensional array weight[Δθ][kc][kd], where Δθ ranges from -30° to +30°, with a step size of 1°, covering the drone's maximum angular velocity of 300° / s; kc ranges from 0 to 1.5, corresponding to a curvature of 0 to 15°; and kd ranges from 0.2 to 1.0, corresponding to a density of 1 to 25 cells / m³. Round Δθ to the nearest integer, retaining kc and kd to one decimal place. Then index the array to obtain the weight value w, which ranges from 0 to 1.0. For example, if Δθ = 2.1° ≈ 2°, kc = 0.485 ≈ 0.5, and kd = 1.0, then look up the table to obtain w = 0.6.

[0114] Finally, step 1044 generates a deflection command. The cross product of the extension direction vobj and the heading vuav is calculated as n = vobj × vuav to obtain the vertical plane normal. A unit vector vortho = normalize(n × vobj) is generated orthogonal to vobj in the plane defined by n. This is then scaled by the weight to obtain the avoidance component vavoid = w × vortho. This avoidance component is then added to the current heading vcmd = vuav + vavoid, resulting in a continuous deflection command. For example, when w = 0.6, vortho = (0.08, -0.06, 0) → vavoid = (0.048, -0.036, 0). Adding the heading (0.87, 0.50, 0) yields the new command (0.918, 0.464, 0).

[0115] In a practical application, in a 500kV high-voltage line avoidance scenario, the drone flies at a speed of 12m / s. Step 1041 extracts the high-voltage line extension direction vector vobj = (0.82, 0.57, 0) from the topology map, corresponding to a direction of N25°E. The flight control system outputs the current heading vector vuav = (0.87, 0.50, 0), corresponding to a direction of N30°E. The projection angle is calculated as follows: horizontal projection vobjproj = (0.82, 0.57, 0). vuavproj = (0.87, 0.50, 0); dot product operation dot = 0.82 × 0.87 + 0.57 × 0.50 = 0.7134 + 0.285 = 0.9984; modulus calculation |vobj| = sqrt(0.82² + 0.57²) = 1.0, |vuav| = sqrt(0.87² + 0.50²) = 1.0; real-time angle θt = arccos(0.9984) ≈ 5.2°. Variation generation: read the historical value θt-1 = 3.1° from 100ms ago, and the algebraic difference Δθ = 5.2° - 3.1° = 2.1°. A positive value indicates that the drone is approaching the high-voltage power line.

[0116] Through step 1042, the curvature influence coefficient: read the spatial curvature κ = 4.85°, linear mapping kc = 0.1 × κ = 0.1 × 4.85 = 0.485; obtain the distribution density D = 5 units / m³, inverse calculation: kd = 5 / D = 5 / 5 = 1.0.

[0117] Through step 1043, the Δθ dimension is set from -30° to +30°, with a step size of 1°, covering an angular velocity of 300° / s; the kc dimension is set from 0 to 1.5, corresponding to a curvature of 0 to 15°; and the kd dimension is set from 0.2 to 1.0, corresponding to a density of 1 to 25 cells / m³. Input quantization: Δθ = 2.1° → 2°, kc = 0.485 → 0.5, kd = 1.0; output weight: w = 0.6.

[0118] In step 1044, the cross product is used to calculate the normal vector: n = vobj × vuav = i (0 × 0 - 0 × 0.50) - j (0 × 0 - 0.82 × 0) + k (0.82 × 0.50 - 0.57 × 0.87) = (0, 0, -0.1109);

[0119] Generate orthogonal directions: n×vobj=i(0.57×(-0.1109)-0×0)-j(0.82×(-0.1109)-0×0)+k(0.82×0-0.57×0)=(-0.063,0.090,0);

[0120] Normalization: vortho=(-0.063 / 0.11,0.090 / 0.11,0)=(-0.573,0.819,0);

[0121] Weight scaling: vavoid=0.6×(-0.573,0.819,0)=(-0.344,0.491,0);

[0122] New heading: vcmd=(0.87,0.50,0)+(-0.344,0.491,0)=(0.526,0.991,0).

[0123] Heading angle change: Original N30°E (tan⁻¹(0.50 / 0.87)=30°) → New heading N62°E.

[0124] The overall solution of step 104 described above dynamically captures the approach trend between the drone and the high-voltage line through real-time projection of the angle change, quantifies the avoidance requirements based on the physical properties of curvature and density, and uses a preset parameter table to efficiently determine the weights. Ultimately, an orthogonal avoidance component is generated in the heading vertical plane to ensure that the trajectory crosses the extension direction of the linear obstacle at a large angle, completely eliminating the risk of parallel approach.

[0125] 105. Generate multiple obstacle avoidance trajectories based on the continuous deflection command, fuse the distance data with the thermal signature distribution data when the UAV executes the multiple obstacle avoidance trajectories, perform closed-loop correction on the continuous deflection command using the fused data, and repeat the correction process until the relative distance between the UAV and the linear obstacle reaches a safety threshold.

[0126] Optionally, step 105 may specifically include the following steps:

[0127] 1051. Decompose the continuous deflection instruction into a trajectory segment parameter set, where the trajectory segment parameter set includes a heading angle change and a corresponding flight speed value;

[0128] 1052. When the UAV flies according to the trajectory segment parameter set, synchronously acquire the distance data and the thermal signature distribution data, and determine the fusion credibility of the trajectory segment parameter set based on the coverage range of the distance data and the overlap ratio of the intensity area of ​​the thermal signature distribution data;

[0129] 1053. Perform a deviation assessment on the heading angle change and the corresponding flight speed value based on the fusion credibility. When the coverage range is lower than a preset ratio during the assessment, the center point of the intensity area of ​​the thermal signature distribution data is preferentially used as a deviation assessment benchmark.

[0130] 1054. Based on the deviation evaluation result, reverse compensation adjustment is performed on the heading angle change and the flight speed value in the next trajectory segment parameter set. After the adjustment, the generation process of the trajectory segment parameter set is repeated until the minimum relative distance between the UAV and the linear obstacle in the distance data reaches the safety threshold.

[0131] In the above scheme, the trajectory segment parameter set refers to the segmented control parameter group formed by discretizing continuous heading instructions, each group containing the heading angle adjustment and flight speed of a single segment trajectory. Coverage refers to the distribution area ratio of the effective distance measurement points of the millimeter wave detection unit in the trajectory segment space. The intensity area refers to the continuous area where the surface temperature of the linear obstacle detected by the infrared sensing unit is significantly higher than the environmental background. The overlap ratio is the ratio of the overlapping area of ​​the distance data coverage and the thermal feature intensity area to the area of ​​the union of the two. The fusion credibility is a sensor data reliability score calculated by the overlap ratio. The deviation assessment benchmark is a reference position used to quantify the trajectory execution error, and the center point of the multi-sensor consistent area is preferred. Reverse compensation adjustment refers to the reverse correction of the heading angle and speed values ​​of the next segment based on the deviation direction and magnitude of the current trajectory segment.

[0132] In the embodiment of the present application, first, instruction decomposition is performed through step 1051, and the continuous deflection instructions are divided into multiple segments at fixed time intervals, each segment corresponding to a fixed actual flight distance of the drone; the heading angle change in each time window is averaged, such as the heading changes from 30°→32.1° within 0-0.5 seconds, then Δψ=+2.1°, and the speed value is the instantaneous speed of the window starting point; finally, a set of trajectory segment parameters is generated, for example, the parameters of the first segment are [heading angle change +2.1°, speed 12m / s].

[0133] Subsequently, step 1052 calculates the fusion credibility: During the trajectory segment, the millimeter wave detection unit's range point cloud and infrared thermal imaging data are collected in real time and aligned to the same moment using timestamps. Within the spatial region corresponding to the trajectory segment, the distribution ratio of effective millimeter wave measurement points with a signal-to-noise ratio greater than 20 dB to the total monitored area is calculated. The infrared thermal image is segmented using a fixed threshold, with temperatures greater than 50°C identified as high-voltage lines, and binary mask outlines of significant temperature regions are extracted. The millimeter wave point cloud is projected onto the infrared image coordinate system, and the ratio of the overlapping area between the two to the total infrared high-temperature area is calculated. A value (ranging from 0 to 1.0) is output using the weighted formula: credibility = 0.7 × coverage ratio + 0.3 × overlap ratio. For example, if the millimeter wave coverage ratio is 80% and the overlap ratio is 70%, the credibility = 0.7 × 0.8 + 0.3 × 0.7 = 0.77.

[0134] At the same time, through step 1053 dynamic deviation evaluation, when the millimeter wave coverage ratio is less than 60%, the center of mass coordinates of the infrared high-temperature area is used as the reference point; otherwise, the center point of the millimeter wave and infrared overlap area is taken; the current position of the drone is connected to the reference point to generate a theoretical direction vector, and the actual heading vector is compared to calculate the angle deviation, such as theoretical direction 35° vs actual heading 32° → deviation +3°; the difference between the actual displacement of the drone and the planned displacement is measured, and the speed correction is calculated based on the time window, such as a 0.8 meter lag within 0.5 seconds → a speed increase of 1.6 m / s is required.

[0135] Finally, through the closed-loop adjustment in step 1054, the heading angle compensation is the planned heading angle change for the next segment minus the current deviation value. For example, if the current deviation is +3°, the original plan for the next segment is +4.3°, which is corrected to +1.3°. The speed compensation is the speed value for the next segment plus the current speed deviation. For example, if the original speed is 12 m / s, it is corrected to 13.6 m / s. A new trajectory segment is generated using the corrected parameters, and steps 1052-1054 are repeated. The correction is terminated when the millimeter wave detects that the minimum distance between the drone and the high-voltage line is ≤3 meters.

[0136] In practical applications, when a drone evades a 500kV high-voltage power line, command segmentation and parameterization are performed in step 1051. The continuous deflection command is divided into 0.5-second time windows, each corresponding to a 6-meter flight distance (at a speed of 12 m / s). In the first segment (0-0.5 seconds), the heading angle increases from 30.0° to 32.1°, calculating the mean change of +2.1°. The instantaneous speed at the start of the window is 12 m / s, encapsulated as the parameter set for trajectory segment 1 [+2.1°, 12 m / s]. Similarly, segment 2 [+4.3°, 12 m / s] and segment 3 [+6.4°, 12 m / s] are generated, completing the command execution conversion.

[0137] The reliability of multi-source data fusion is calculated through step 1052. When executing segment 1, within the spatial area (x=30-36m, y=45-48m, z=50-52m): the millimeter wave effective point covers 28.8m³, the monitoring area is 36m³, and the coverage ratio is 80%; the infrared thermal image is segmented by a 50°C threshold to extract a 25.2m² high-temperature area; through rigid body transformation to align the point cloud and the thermal map, an overlapping area of ​​18.9m² is measured, with an overlap ratio of 75%; according to the weighted formula "0.7×coverage ratio + 0.3×overlap ratio", the synthesized fusion reliability is 78.5%, providing a data reliability basis for deviation assessment.

[0138] Dynamic deviation quantification is completed in step 1053. Based on the fusion confidence of 78.5% and the coverage ratio of 80%>60% threshold, the center point of the overlapping area (32.1, 45.3, 52.7) is selected as the benchmark. The actual endpoint of the drone (31.9, 45.1, 52.5) ​​is compared with the benchmark point to generate a theoretical direction vector (2.1, 0.3, 0.2), with a horizontal projection heading angle of 8.1°. A heading deviation of +24.0° is generated compared to the actual heading of 32.1°. At the same time, the difference of 4.10 meters between the planned displacement of 6 meters and the actual displacement of 1.90 meters is calculated. Combined with the 0.5-second window, the speed compensation required to increase the speed by 8.2 m / s is obtained.

[0139] Closed-loop correction and termination are achieved through step 1054, and reverse compensation is applied to the parameters of segment 2: the heading angle change is corrected to +4.3°-24.0°=-19.7°, and the speed is corrected to 12+8.2=20.2m / s. When executing the corrected parameters, real-time monitoring of the millimeter wave distance is performed, and it is detected that the minimum distance to the high-voltage line has dropped from 3.2 meters to 2.8 meters, which is lower than the safety threshold of 3.0 meters. The correction process is immediately terminated and the safe hovering mode is triggered, completing the closed-loop optimization of the obstacle avoidance trajectory.

[0140] The overall solution of step 105 above makes the instructions executable through time window discretization, dynamically evaluates the credibility based on the spatial consistency of millimeter wave and infrared data, intelligently selects deviation reference points in combination with the coverage ratio threshold, and finally realizes closed-loop trajectory correction through the reverse compensation mechanism of heading and speed, ensuring that the drone always maintains a safe distance when avoiding linear obstacles such as high-voltage lines.

[0141] The following is a complete embodiment of steps 101 to 105:

[0142] like Figure 2As shown in the figure, a drone is flying in a 500kV high-voltage power line inspection scenario at an altitude of 50m and a speed of 12m / s. Multimodal data is synchronously acquired through step 101: the optical camera unit captures visible light images at 30 frames per second, identifying the continuous outline of a 3cm diameter high-voltage power line, such as a thin, elongated chain of pixels with a grayscale value of 120, extending along the N25°E axis. The millimeter-wave detection unit transmits a 77GHz frequency-modulated continuous wave through the mist, generating a spatial point cloud of the high-voltage power line with an accuracy of 0.1m and a point density of 5 points / square meter. The cloud contains key locations such as the coordinates (110.2, 210.5, and 15.3). The infrared sensing unit uses 14µm thermal imaging to detect the cable surface temperature, outputting a 64×64 thermal signature matrix. The temperature of the high-voltage power line region, 45°C, differs significantly from the ambient temperature of 30°C. The three types of data are synchronously triggered by the hardware clock. The millimeter-wave point cloud and the infrared thermal image are registered to the optical image coordinate system using an iterative closest point algorithm. The time deviation is controlled within 1ms, and the spatial registration error is less than 5cm.

[0143] Optical failure compensation is performed through step 102. When the UAV enters a strong backlight area, that is, the light intensity drops to 40 lux, the high-voltage line segment in the optical image has a coordinate range of x=100-150, y=200-240, an average grayscale value of 40, and 35% contour loss: based on the preset rules, 32 effective distance points of the millimeter wave are counted in the missing area, with a distribution density of 4.2 points / square meter, a standard deviation of the infrared thermal feature point temperature of 7°C, and a maximum allowable deviation of 15°C; the joint coverage capability parameter is calculated as 4.2×(1-7 / 15)=2.24; the missing area is divided into For a 5×4 grid, a geometric weight of 0.7 is assigned to the upper left subgrid with a millimeter wave point density of 6 points / square meter, and an attribute weight of 0.3 is assigned to the lower right subgrid with an infrared temperature standard deviation of 3°C. The millimeter wave points (110.1, 210.3, 15.2) are used as geometric constraint nodes, and the infrared point (149.7, 239.8) with a temperature of 28.2°C is used as an attribute constraint node. A continuous surface is generated through radial basis function interpolation, and the temperature values ​​are mapped with a slope of 6 grayscale / °C to fill in the missing areas, thereby reconstructing the complete morphological contour of the high-voltage line. The error between the reconstructed contour point coordinate set and the original morphological contour is less than 0.15 meters.

[0144] The dynamic analysis of linear features is completed through step 103. Based on the compensated data, the octree is used to recursively segment the three-dimensional space. The initial voxel is 1 cubic meter and the minimum unit is 0.125 cubic meter. The target unit with linearity index greater than 0.8 and aspect ratio greater than 5:1 is selected. The high-voltage line unit eigenvalues ​​λ1=12.8, λ2=1.5, linearity 0.88, and aspect ratio 8.5:1 are used. The singular value decomposition is performed on the point cloud of the unit (32.35, 45.55, 52.95). The maximum singular value corresponds to the right singular vector (0.91, 0.41, 0.02) as the local extension direction. The dot product of the adjacent unit direction vectors is 0.998. When the angle is 3.2°<15°, spherical linear interpolation is used to generate the intermediate direction (0.90, 0 .43,0.02), and the high-voltage line is connected in series to form an overall direction of N25°E, with a global direction vector of (0.82,0.57,0); 5 consecutive cells are selected along the extension direction to calculate the adjacent direction angles (3.2°+4.1°+5.3°+6.8°=19.4°), and the moving average is used to obtain a spatial curvature of 4.85° / meter; a 1 cubic meter voxel grid is established with the real-time position of the drone (0,0,50) as the origin, and the 5 target cells in the voxels (30-31,45-46,50-51) are counted to generate a distribution density of 5 cells / cubic meter. Finally, a topological map node {coordinates (30.5,45.5,50.5), direction (0.82,0.57,0), curvature 4.85°, density 5} is constructed.

[0145] Generate a real-time avoidance command through step 104, extract the high-voltage line extension direction vector (0.82, 0.57, 0) from the topology map, and project it onto the horizontal plane with the current heading of the UAV (0.87, 0.50, 0) to calculate the real-time angle of 5.2°; retrieve the historical angle of 3.1° 100 milliseconds ago to obtain the change Δθ=+2.1°, approximating the trend; read the node curvature of 4.85° to generate the influence coefficient k_c=0.1×4.85=0.485, and the density of 5 units / cubic meter to generate the coefficient k_d=5 / 5=1.0; and set the parameter (Δθ =2°,k_c=0.5,k_d=1.0) inputs the preset 3D mapping table, and the index outputs the avoidance weight of 0.6. Using the vector cross product algorithm, the vector (0,0,-0.1109) is constructed in the vertical plane to form an orthogonal unit vector (-0.573,0.819,0), which is then scaled by the weight to obtain the avoidance component (-0.344,0.491,0). This is then superimposed on the original heading to generate a new command vector (0.526,0.991,0), which adjusts the heading angle from 30° to 62° and increases the angle with the high-voltage line to 37°.

[0146] The closed-loop trajectory correction is achieved through step 105: the continuous instruction is decomposed into three segments, each with a flight distance of 0.5 seconds / 6 meters: segment 1 parameters [heading angle change +2.1°, speed 12m / s], segment 2 [+4.3°, 12m / s], segment 3 [+6.4°, 12m / s]; when executing segment 1, within the corresponding spatial area (30-36m, 45-48m, 50-52m), the millimeter wave effective point covers 28.8 cubic meters, the monitoring area is 36 cubic meters, and the coverage ratio is 80%; the infrared thermal feature extracts a 25.2 square meter high temperature area, and the overlapping area after rigid body registration is 18.9 square meters, with an overlap ratio of 75%; according to the weights The fusion confidence level was calculated to be 78.5% using 0.7 × coverage ratio + 0.3 × overlap ratio. Based on this, the center points of the overlap area (32.1, 45.3, 52.7) were selected as the reference. Compared with the actual position of the drone (31.9, 45.1, 52.5), the heading deviation was +24.0° and the displacement lag was 4.10 meters, requiring an increase in speed of 8.2 m / s. The parameters in segment 2 were reversely compensated: the heading angle was adjusted to +4.3° - 24.0° = -19.7°, and the speed was increased to 20.2 m / s. When executing the corrected trajectory, the minimum distance for real-time millimeter-wave detection dropped to 2.8 meters, below the safety threshold of 3.0 meters, immediately terminating the correction and triggering hovering.

[0147] In the high-voltage transmission line avoidance scenario, this application effectively eliminates the perception blind spots caused by light attenuation and meteorological interference through the spatiotemporal complementarity mechanism of multimodal sensors; realizes refined dynamic modeling of linear obstacles based on octree segmentation and geometric feature constraints; generates directional avoidance instructions using the real-time dynamic relationship between extension direction and heading to ensure that the trajectory crosses the obstacle perpendicularly; and finally, through the credibility evaluation and reverse compensation mechanism of multiple trajectories, realizes closed-loop correction until the safety distance threshold is met, significantly improving the robustness and safety of autonomous obstacle avoidance of drones in complex environments.

[0148] Figure 3 The present invention provides a schematic diagram of a multi-sensor fusion-based autonomous obstacle avoidance system for UAVs. Figure 3 As shown, the system includes:

[0149] Acquisition module 31, for synchronously acquiring three types of heterogeneous sensing data from the optical camera unit, the millimeter wave detection unit and the infrared sensing unit during the flight of the drone;

[0150] a compensation module 32 for establishing a data compensation rule based on the spatiotemporal coverage differences of the three types of heterogeneous sensing data, and for performing spatial interpolation compensation using the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit according to the data compensation rule when image data of the optical camera unit is missing due to insufficient illumination;

[0151] The analysis module 33 is used to perform dynamic feature analysis on linear obstacles in the flight path based on the compensated perception data, extract the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results, and generate an obstacle topology map in a three-dimensional spatial coordinate system with the current position of the UAV as the origin;

[0152] a conversion module 34 for converting a change in the angle between the extension direction and the current flight heading of the UAV in the obstacle topology map into an avoidance direction adjustment weight based on the real-time dynamic relationship between the extension direction and the current flight heading of the UAV, and generating a continuous deflection instruction for the obstacle avoidance direction by combining the spatial curvature and the continuous distribution density parameter;

[0153] The correction module 35 is used to generate multiple obstacle avoidance trajectories based on the continuous deflection instructions, fuse the distance data with the thermal feature distribution data when the UAV executes the multiple obstacle avoidance trajectories, perform closed-loop correction on the continuous deflection instructions based on the fused data, and repeat the correction process until the relative distance between the UAV and the linear obstacle reaches a safety threshold.

[0154] Figure 3 The autonomous obstacle avoidance system for UAV based on multi-sensor fusion can be executed Figure 1 The implementation principles and technical effects of the multi-sensor fusion-based autonomous obstacle avoidance method for drones described in the illustrated embodiment will not be elaborated on here. The specific manner in which each module and unit performs operations in the multi-sensor fusion-based autonomous obstacle avoidance system for drones in the aforementioned embodiment has been described in detail in the relevant embodiments of the method and will not be elaborated on here.

[0155] In one possible design, Figure 3 The autonomous obstacle avoidance system for a UAV based on multi-sensor fusion of the embodiment shown can be implemented as a computing device, such as Figure 4 As shown, the computing device may include a storage component 41 and a processing component 42;

[0156] The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 42 .

[0157] The processing component 42 is used for the above Figure 1 The embodiment provides an autonomous obstacle avoidance method for a UAV based on multi-sensor fusion.

[0158] The processing component 42 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0159] The storage component 41 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0160] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0161] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0162] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0163] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0164] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The embodiment shown is an autonomous obstacle avoidance method for a UAV based on multi-sensor fusion.

[0165] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0167] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for autonomous obstacle avoidance of UAV based on multi-sensor fusion, characterized in that: include: During the flight of the drone, three types of heterogeneous sensing data are simultaneously acquired from the optical camera unit, millimeter wave detection unit, and infrared sensing unit; A data compensation rule is established based on the differences in spatiotemporal coverage of the three types of heterogeneous sensing data. When the optical camera unit loses image data due to insufficient illumination, the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit are used to perform spatial interpolation compensation according to the data compensation rule. Based on the compensated perception data, the system dynamically analyzes the linear obstacles in the flight path, extracts the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results, and generates an obstacle topology map in a three-dimensional spatial coordinate system with the current position of the drone as the origin. According to the real-time dynamic relationship between the extension direction in the obstacle topology map and the current flight heading of the UAV, the change in the angle between the extension direction and the heading is converted into an avoidance direction adjustment weight, and combined with the spatial curvature and continuous distribution density parameters, a continuous deflection instruction of the obstacle avoidance heading is generated; Based on the continuous deflection command, multiple obstacle avoidance trajectories are generated. When the drone executes the multiple obstacle avoidance trajectories, the distance data and the thermal feature distribution data are fused. The continuous deflection command is closed-loop corrected using the fused data. The correction process is repeated until the relative distance between the drone and the linear obstacle reaches a safety threshold.

2. The method according to claim 1, characterized in that Based on the compensated perception data, the linear obstacles in the flight path are dynamically analyzed, the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results are extracted, and an obstacle topology map is generated in a three-dimensional spatial coordinate system with the current position of the drone as the origin, including: Segmenting the compensated perception data into multiple independent spatial units, and screening out target units corresponding to the linear obstacles based on the geometric features of the spatial units in three-dimensional space; Performing a main direction analysis on the target unit to determine the extension direction of the target unit in three-dimensional space, and performing continuity fitting on the extension direction according to a spatial adjacent relationship to generate the overall extension direction of the linear obstacle; Calculating a curvature parameter of the linear obstacle in three-dimensional space as a spatial curvature according to an extension direction angle deviation value between adjacent target units, wherein the extension direction angle deviation value is determined by the main direction angle between adjacent target units; The number distribution of the target units per unit volume in a three-dimensional spatial coordinate system with the current position of the drone as the origin is counted, a continuous distribution density parameter of the linear obstacle is generated based on the number distribution, and the overall extension direction, the spatial curvature, and the continuous distribution density parameter are mapped to the three-dimensional spatial coordinate system to form an obstacle topology map.

3. The method according to claim 1, characterized in that The method converts the change in the angle between the extension direction and the current flight direction of the UAV in the obstacle topology map into an avoidance direction adjustment weight based on the real-time dynamic relationship between the extension direction and the current flight direction of the UAV, and generates a continuous deflection instruction for the obstacle avoidance direction in combination with the spatial curvature and continuous distribution density parameters, including: Obtaining in real time a real-time angle value between an extension direction in the obstacle topology map and the current flight direction of the UAV, and calculating a difference between the real-time angle value and the angle at the previous moment according to the extension direction as an angle change; Generating a curvature influence coefficient and a density influence coefficient according to the spatial curvature and the continuous distribution density parameter, respectively, wherein the curvature influence coefficient is positively correlated with the spatial curvature, and the density influence coefficient is negatively correlated with the continuous distribution density parameter; Inputting the angle variation, the curvature influence coefficient, and the density influence coefficient into a preset dynamic relationship evaluation parameter table, and determining the avoidance direction adjustment weight by looking up the table; According to the avoidance direction adjustment weight, an avoidance direction component orthogonal to the extension direction is generated in a vertical plane of the current flight heading of the UAV, and the avoidance direction component is superimposed on the current heading to generate a continuous deflection instruction.

4. The method according to claim 1, wherein The method comprises: generating a plurality of obstacle avoidance trajectories based on the continuous deflection instructions, fusing the distance data and the thermal signature distribution data when the UAV executes the plurality of obstacle avoidance trajectories, performing closed-loop correction on the continuous deflection instructions using the fused data, and repeating the correction process until the relative distance between the UAV and the linear obstacle reaches a safety threshold, including: Decomposing the continuous deflection instruction into a trajectory segment parameter set, wherein the trajectory segment parameter set includes a heading angle change and a corresponding flight speed value; When the UAV flies according to the trajectory segment parameter set, the distance data and the thermal signature distribution data are synchronously acquired, and the fusion credibility of the trajectory segment parameter set is determined according to the overlap ratio between the coverage range of the distance data and the intensity area of ​​the thermal signature distribution data; Based on the fusion credibility, a deviation evaluation is performed on the heading angle change and the corresponding flight speed value, wherein when the coverage range is lower than a preset ratio during the evaluation process, the center point of the intensity area of ​​the thermal feature distribution data is preferentially used as a deviation evaluation benchmark; Based on the deviation evaluation result, a reverse compensation adjustment is performed on the heading angle change and the flight speed value in the next trajectory segment parameter set. After the adjustment, the generation process of the trajectory segment parameter set is repeated until the minimum relative distance between the UAV and the linear obstacle in the distance data reaches the safety threshold.

5. The method according to claim 1, characterized in that The data compensation rules are established based on the differences in spatiotemporal coverage of the three types of heterogeneous sensing data. When the optical camera unit loses image data due to insufficient illumination, spatial interpolation compensation is performed using the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit according to the data compensation rules, including: detecting a pixel grayscale value of a target area in the image data output by the optical camera unit, and marking a boundary of the target area as a missing area boundary when the pixel grayscale value is lower than a preset illumination threshold and the area of ​​a continuous pixel missing region of the target area exceeds a set ratio; Calculating a joint coverage capability parameter based on a density distribution of distance measurement points of the millimeter wave detection unit within a boundary of the missing area and a distribution of thermal signature intensity of the infrared sensing unit in the missing area; Based on the joint coverage capability parameter, spatial weight allocation is performed on the distance measurement points of the millimeter wave detection unit and the thermal characteristic intensity distribution points of the infrared sensing unit; According to the allocation result, an interpolation surface is constructed within the boundary of the missing area with the distance measurement points as geometric constraints and the thermal feature intensity distribution points as attribute constraints, and the interpolation surface is mapped to the image data within the boundary of the missing area to complete spatial interpolation compensation.

6. The method according to claim 3, characterized in that The step of obtaining a real-time angle value between an extension direction in the obstacle topology map and the current flight direction of the UAV, and calculating a difference between the real-time angle value and the angle at the previous moment according to the extension direction as an angle variation includes: Extracting the extension direction vector of the linear obstacle from the obstacle topology map, and synchronously obtaining the current flight heading vector output by the UAV flight control unit; Calculating the angle between the plane projection of the extension direction vector and the flight heading vector in three-dimensional space as a real-time angle value; Performing a difference operation on the real-time angle value and the historical angle value stored at the previous moment to obtain the angle change at adjacent moments; When the real-time angle value is first acquired, the angle variation is initialized to zero, and the historical angle value is continuously updated as the current real-time angle value at subsequent moments.

7. The method according to claim 2, characterized in that The performing of main direction analysis on the target unit to determine the extension direction of the target unit in three-dimensional space, and performing continuity fitting on the extension direction according to a spatial adjacent relationship to generate the overall extension direction of the linear obstacle includes: Performing a three-dimensional spatial distribution analysis on the geometric features of the target unit, and determining the longest extension axis direction of the target unit in the three-dimensional space as the main direction according to the analysis results; establishing a direction association relationship between adjacent target units according to the main direction, and marking the adjacent target units with the direction association relationship as direction association units; Performing consistency adjustment on the main directions of the direction association units, so that the main direction angles of the direction association units are gradually reduced in spatial adjacent order to within a preset allowable deviation range through the adjustment process; The main directions of the adjusted direction association units are connected in spatial adjacent order to form a continuous extension direction of the linear obstacle as the overall extension direction.

8. An autonomous obstacle avoidance system for UAV based on multi-sensor fusion, characterized in that: include: The acquisition module is used to synchronously acquire three types of heterogeneous sensing data from the optical camera unit, millimeter wave detection unit, and infrared sensing unit during the flight of the drone; a compensation module, configured to establish data compensation rules based on the differences in spatiotemporal coverage of the three types of heterogeneous sensing data, and to perform spatial interpolation compensation using the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit according to the data compensation rules when image data of the optical camera unit is missing due to insufficient illumination; The parsing module is used to perform dynamic feature analysis on linear obstacles in the flight path based on the compensated perception data, extract the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results, and generate an obstacle topology map in a three-dimensional spatial coordinate system with the current position of the drone as the origin; a conversion module for converting a change in the angle between the extension direction and the current flight heading of the UAV in the obstacle topology map into an avoidance direction adjustment weight based on the real-time dynamic relationship between the extension direction and the current flight heading of the UAV, and generating a continuous deflection instruction for the obstacle avoidance heading by combining the spatial curvature and the continuous distribution density parameter; A correction module is configured to generate multiple obstacle avoidance trajectories based on the continuous deflection instructions, fuse the distance data with the thermal signature distribution data when the UAV executes the multiple obstacle avoidance trajectories, perform closed-loop correction on the continuous deflection instructions using the fused data, and repeat the correction process until the relative distance between the UAV and the linear obstacle reaches a safety threshold.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement an autonomous obstacle avoidance method for a drone based on multi-sensor fusion as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, the method for autonomous obstacle avoidance of a drone based on multi-sensor fusion as described in any one of claims 1 to 7 is implemented.

Images