An unmanned aerial vehicle autonomous obstacle avoidance method and system based on multi-sensor fusion

Abstract

The application provides a kind of unmanned aerial vehicle autonomous obstacle avoidance method and system based on multi-sensor fusion. Among them, the method obtains three types of heterogeneous sensing data during the flight of the unmanned aerial vehicle;According to the three types of data, a data compensation rule is established, and spatial interpolation compensation is performed when the image data of the optical camera unit is missing;Based on the compensated sensing data, the dynamic characteristics of linear obstacles are analyzed, and a three-dimensional obstacle topology map is generated;According to the real-time dynamic relationship between the extension direction and the flight direction in the topology map, the angle change between the extension direction and the flight direction is converted into an avoidance direction adjustment weight, and a continuous deflection command is generated by combining the spatial curvature and distribution density parameters;Based on the instruction, a multi-section obstacle avoidance trajectory is generated, and when executed, the instruction is closed loop corrected by combining millimeter wave distance data and infrared thermal feature data. The application improves the dynamic avoidance accuracy and flight safety of the unmanned aerial vehicle for linear obstacles.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle autonomous obstacle avoidance, and particularly relates to an unmanned aerial vehicle autonomous obstacle avoidance method and system based on multi-sensor fusion. BACKGROUND

[0002] In complex low-altitude scenes such as power inspection and mountain logistics, the unmanned aerial vehicle needs to avoid slender linear obstacles such as high-voltage lines and cableways. Such targets have the characteristics of small diameter, low reflectivity and strong spatial extension, and the obstacle avoidance system is required to have stable identification capability in limited perception environments such as rain, fog and dim light; real-time dynamic modeling capability for the geometric shape and spatial trend of linear obstacles; and lightweight obstacle avoidance decision mechanism suitable for high-speed flight.

[0003] The current mainstream scheme adopts laser radar point cloud clustering and visual semantic fusion technology: generate environment point cloud through laser radar scanning, use Euclidean clustering segmentation to extract linear object candidate area; simultaneously use a visible light camera for semantic segmentation to identify specific category targets such as power lines and cables; align and fuse the two types of results through time stamp to construct a static obstacle map and plan a global obstacle avoidance path.

[0004] The existing scheme has the following defects: the laser radar is sparse in point cloud for slender objects with too small diameter, and the vision is invalid in rain, fog and dim light, resulting in missed detection rate of linear targets; it relies on low-frequency static map updates and cannot respond to dynamic pose changes of linear obstacles when the unmanned aerial vehicle is flying at high speed; the global path planning ignores the extension characteristics of linear obstacles, and often generates dangerous approaching trajectories parallel to the trend of power lines. SUMMARY

[0005] The present application provides an unmanned aerial vehicle autonomous obstacle avoidance method and system based on multi-sensor fusion to solve the problem of insufficient dynamic avoidance precision of linear obstacles by the unmanned aerial vehicle in the prior art.

[0006] In the first aspect, the present application provides an unmanned aerial vehicle autonomous obstacle avoidance method based on multi-sensor fusion, comprising:

[0007] During the flight of the unmanned aerial vehicle, three types of heterogeneous perception data from an optical camera unit, a millimeter wave detection unit and an infrared perception unit are synchronously acquired;

[0008] A data compensation rule is established according to the spatiotemporal coverage differences of the three types of heterogeneous perception data, and when the optical camera unit causes image data to be missing due to insufficient light, the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared perception unit are used for spatial interpolation compensation through the data compensation rule;

[0009] Based on the compensated perception data, the linear obstacle existing in the flight path is analyzed for dynamic characteristics, the extension direction, the spatial curvature and the continuous distribution density parameter of the linear obstacle in the analysis result are extracted, and the obstacle topology graph in the three-dimensional space coordinate system with the current position of the unmanned aerial vehicle as the origin is generated;

[0010] According to the real-time dynamic relationship between the extension direction in the obstacle topology graph and the current flight heading of the unmanned aerial vehicle, the extension direction and the heading angle change are converted into an avoidance direction adjustment weight, the continuous deflection instruction of the obstacle avoidance heading is generated in combination with the spatial curvature and the continuous distribution density parameter;

[0011] Based on the continuous deflection instruction, a multi-section obstacle avoidance trajectory is generated, the distance data and the thermal feature distribution data are fused when the unmanned aerial vehicle executes the multi-section obstacle avoidance trajectory, the continuous deflection instruction is corrected in a closed loop through the fused data, and the correction process is repeated until the relative distance between the unmanned aerial vehicle and the linear obstacle reaches a safety threshold.

[0012] Optionally, based on the compensated perception data, the linear obstacle existing in the flight path is analyzed for dynamic characteristics, the extension direction, the spatial curvature and the continuous distribution density parameter of the linear obstacle in the analysis result are extracted, and the obstacle topology graph in the three-dimensional space coordinate system with the current position of the unmanned aerial vehicle as the origin is generated, comprising:

[0013] The compensated perception data is divided into a plurality of independent space units, and according to the geometric shape characteristics of the space units in the three-dimensional space, the target unit corresponding to the linear obstacle is screened out;

[0014] The main direction of the target unit is analyzed to determine the extension direction of the target unit in the three-dimensional space, and the extension direction is continuously fitted according to the spatial adjacent relationship to generate the overall extension direction of the linear obstacle;

[0015] According to the extension direction angle deviation value between adjacent target units, the bending degree parameter of the linear obstacle in the three-dimensional space is calculated as the spatial curvature, wherein the extension direction angle deviation value is determined by the main direction angle of the adjacent target units;

[0016] The number distribution of the target unit in a unit volume in the three-dimensional space coordinate system with the current position of the unmanned aerial vehicle as the origin is counted, the continuous distribution density parameter of the linear obstacle is generated according to the number distribution, and the overall extension direction, the spatial curvature and the continuous distribution density parameter are mapped into the three-dimensional space coordinate system to form the obstacle topology graph.

[0017] Optionally, the extension direction in the obstacle topology map and the real-time dynamic relationship between the current flight heading of the unmanned aerial vehicle are used to convert the extension direction and heading angle change amount into an avoidance direction adjustment weight, and the spatial curvature and continuous distribution density parameters are combined to generate a continuous deflection instruction of the obstacle avoidance heading, including:

[0018] The real-time angle value between the extension direction in the obstacle topology map and the current flight heading of the unmanned aerial vehicle is obtained in real time, and the difference between the real-time angle value and the angle at the last time is calculated as the angle change amount according to the extension direction;

[0019] According to the spatial curvature and continuous distribution density parameters, a curvature influence coefficient and a density influence coefficient are respectively generated, wherein the curvature influence coefficient is positively correlated with the spatial curvature, and the density influence coefficient is negatively correlated with the continuous distribution density parameter;

[0020] The angle change amount, the curvature influence coefficient and the density influence coefficient are input into a preset dynamic relationship evaluation parameter table, and the avoidance direction adjustment weight is determined by table lookup;

[0021] According to the avoidance direction adjustment weight, an avoidance direction component orthogonal to the extension direction is generated in the vertical plane of the current flight heading of the unmanned aerial vehicle, and the avoidance direction component is superimposed with the current heading to generate a continuous deflection instruction.

[0022] Optionally, based on the continuous deflection instruction, a multi-segment obstacle avoidance trajectory is generated, and the distance data and the thermal feature distribution data are fused when the unmanned aerial vehicle executes the multi-segment obstacle avoidance trajectory, and the continuous deflection instruction is closed-loop corrected through the fused data, and the correction process is repeated until the relative distance between the unmanned aerial vehicle and the linear obstacle reaches a safety threshold, including:

[0023] The continuous deflection instruction is decomposed into a trajectory segment parameter set, and the trajectory segment parameter set includes a heading angle change amount and a corresponding flight speed value;

[0024] When the unmanned aerial vehicle flies according to the trajectory segment parameter set, the distance data and the thermal feature distribution data are synchronously obtained, and the fusion confidence of the trajectory segment parameter set is determined according to the overlap ratio of the coverage range of the distance data and the intensity region of the thermal feature distribution data;

[0025] According to the fusion confidence, the heading angle change amount and the corresponding flight speed value are evaluated, wherein when the coverage range is lower than a preset proportion during the evaluation process, the center point of the intensity region of the thermal feature distribution data is preferentially used as the deviation evaluation reference;

[0026] Based on the deviation evaluation result, the heading angle change and the flight speed value in the next trajectory segment parameter set are adjusted reversely, and the generation process of the trajectory segment parameter set is repeated after adjustment until the minimum relative distance between the UAV and the linear obstacle in the distance data reaches the safety threshold.

[0027] Optionally, the data compensation rule is established according to the spatio-temporal coverage difference of the three types of heterogeneous perception data, when the optical camera unit causes image data to be missing due to insufficient illumination, the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared perception unit are used for spatial interpolation compensation through the data compensation rule, including:

[0028] Detect the pixel gray value of the target region in the image data output by the optical camera unit, when the pixel gray value is lower than the preset illumination threshold and the area of the continuous pixel missing region of the target region exceeds the set proportion, mark the boundary of the target region as a missing region boundary;

[0029] According to the distance measurement point density distribution of the millimeter wave detection unit in the missing region boundary and the thermal feature intensity distribution of the infrared perception unit in the missing region, calculate the joint coverage capability parameter;

[0030] Based on the joint coverage capability parameter, the distance measurement points of the millimeter wave detection unit and the thermal feature intensity distribution points of the infrared perception unit are spatially weighted and allocated;

[0031] According to the allocation result, an interpolation surface is constructed in the missing region boundary, which is geometrically constrained by the distance measurement points and attribute-constrained by the thermal feature intensity distribution points, and the interpolation surface is mapped into the image data in the missing region boundary to complete the spatial interpolation compensation.

[0032] Optionally, the real-time angle value between the extension direction in the obstacle topology map and the current flight direction of the UAV is obtained in real time, and the difference value between the real-time angle value and the angle value at the last time is calculated as the angle change value according to the extension direction, including:

[0033] The extension direction vector of the linear obstacle is extracted from the obstacle topology map, and the current flight direction vector output by the UAV flight control unit is obtained synchronously;

[0034] The plane projection angle between the extension direction vector and the flight direction vector in the three-dimensional space is calculated as the real-time angle value;

[0035] The real-time angle value and the historical angle value stored at the last time are subjected to difference operation to obtain the angle change value at the adjacent time;

[0036] When the real-time included angle value is acquired for the first time, the included angle change amount is initialized to zero, and the historical included angle value is continuously updated as the current real-time included angle value at subsequent time points.

[0037] Optionally, the main direction analysis on the target unit is performed to determine the extension direction of the target unit in the three-dimensional space, and the extension directions are continuously fitted according to the spatial adjacent relationship to generate the overall extension direction of the linear obstacle, including:

[0038] The three-dimensional spatial distribution analysis is performed on the geometric features of the target unit, and the longest extension axis direction of the target unit in the three-dimensional space is determined as the main direction according to the analysis result;

[0039] The direction correlation relationship between adjacent target units is established according to the main direction, and the adjacent target units having the direction correlation relationship are marked as direction correlation units;

[0040] The main directions of the direction correlation units are adjusted for consistency, and the main direction angles of the direction correlation units are gradually decreased to within a preset allowable deviation range in the spatial adjacent order through the adjustment process;

[0041] The main directions of the adjusted direction correlation units are connected in the spatial adjacent order to form the continuous extension direction of the linear obstacle as the overall extension direction.

[0042] In a second aspect, the application provides an unmanned aerial vehicle autonomous obstacle avoidance system based on multi-sensor fusion, comprising:

[0043] An acquisition module is configured to synchronously acquire three types of heterogeneous sensing data from an optical camera unit, a millimeter wave detection unit and an infrared sensing unit during the flight of an unmanned aerial vehicle.

[0044] A compensation module is configured to establish a data compensation rule according to the spatiotemporal coverage differences of the three types of heterogeneous sensing data, and when the optical camera unit causes image data to be missing due to insufficient light, the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit are used for spatial interpolation compensation through the data compensation rule.

[0045] An analysis module is configured to analyze the dynamic characteristics of a linear obstacle existing in a flight path based on the compensated sensing data, extract the extension direction, spatial curvature and continuous distribution density parameters of the linear obstacle in the analysis result, and generate an obstacle topology graph in a three-dimensional coordinate system with the current position of the unmanned aerial vehicle as the origin.

[0046] a conversion module configured to convert the extension direction and heading angle change amount into an avoidance direction adjustment weight according to a real-time dynamic relationship between the extension direction in the obstacle topology map and a current flight heading of the UAV, and generate a continuous deflection instruction of an obstacle avoidance heading in combination with the spatial curvature and continuous distribution density parameters;

[0047] a correction module configured to generate a multi-segment obstacle avoidance trajectory based on the continuous deflection instruction, fuse the distance data and the thermal feature distribution data when the UAV executes the multi-segment obstacle avoidance trajectory, and perform a closed-loop correction on the continuous deflection instruction through the fused data, and repeat the correction process until a relative distance between the UAV and the linear obstacle reaches a safety threshold.

[0048] In a third aspect, an embodiment of the present application provides a computing device including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the method for autonomous obstacle avoidance of a UAV based on multi-sensor fusion according to the first aspect.

[0049] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, implements the method for autonomous obstacle avoidance of a UAV based on multi-sensor fusion according to the first aspect.

[0050] In an embodiment of the present application, three types of heterogeneous sensing data from an optical camera unit, a millimeter wave detection unit and an infrared sensing unit are synchronously acquired during flight of a UAV; a data compensation rule is established according to a spatiotemporal coverage difference of the three types of heterogeneous sensing data; when image data is missing due to insufficient illumination of the optical camera unit, the data compensation rule is used to perform spatial interpolation compensation on distance data of the millimeter wave detection unit and thermal feature distribution data of the infrared sensing unit; based on the compensated sensing data, a linear obstacle existing in a flight path is analyzed for dynamic features, extension direction, spatial curvature and continuous distribution density parameters of the linear obstacle are extracted from an analysis result, and an obstacle topology map in a three-dimensional coordinate system with a current position of the UAV as an origin is generated; an avoidance direction adjustment weight is converted from an extension direction and heading angle change amount according to a real-time dynamic relationship between the extension direction in the obstacle topology map and a current flight heading of the UAV, and a continuous deflection instruction of an obstacle avoidance heading is generated in combination with the spatial curvature and continuous distribution density parameters; a multi-segment obstacle avoidance trajectory is generated based on the continuous deflection instruction, the distance data and the thermal feature distribution data are fused when the UAV executes the multi-segment obstacle avoidance trajectory, a closed-loop correction is performed on the continuous deflection instruction through the fused data, and the correction process is repeated until a relative distance between the UAV and the linear obstacle reaches a safety threshold.

[0051] The technical scheme has the following beneficial effects: the single sensor can solve the problem of sensing blind area in a complex environment, and provide data basis for multi-modal compensation; when optical data is missing, the spatial complementarity of millimeter wave and infrared data is used to generate complete environment sensing; the geometric characteristics of linear obstacles are extracted, and a dynamic space model suitable for the pose of the unmanned aerial vehicle is constructed; the obstacle avoidance direction is generated in real time according to the extension characteristics of the linear obstacle and the flight state, so as to avoid the risk of parallel approach; the obstacle avoidance trajectory is corrected in real time through multi-sensor data fusion, and the safety of dynamic avoidance is ensured.

[0052] Further, the compensated sensing data is segmented into spatial units, and linear obstacle target units are screened based on geometric morphological characteristics; the main direction of the target units is analyzed and continuously fitted to generate the overall extension direction; the spatial curvature is calculated according to the included angle of the main directions of adjacent units; the number of target units in a unit volume is counted to generate a distribution density parameter; and finally, the three-dimensional coordinate system is mapped to form an obstacle topology graph. Through the structural analysis of the geometric characteristics of the linear obstacle, high-precision spatial topology feature input is provided for dynamic obstacle avoidance.

[0053] These aspects or other aspects of the present application will be more apparent in the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0055] Figure 1 A flowchart of an unmanned aerial vehicle autonomous obstacle avoidance method based on multi-sensor fusion provided by the present application is shown;

[0056] Figure 2 A scene diagram of an unmanned aerial vehicle autonomous obstacle avoidance method based on multi-sensor fusion provided by the present application is shown;

[0057] Figure 3 A structural schematic diagram of an unmanned aerial vehicle autonomous obstacle avoidance system based on multi-sensor fusion provided by the present application is shown;

[0058] Figure 4 A structural schematic diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0060] In some of the flowcharts described in the specification and claims of the present application and in the above-described figures, a plurality of operations are included which occur in a particular order, but it should be clearly understood that the operations can be performed in an order other than that in which they appear or in parallel, and the serial numbers of the operations, such as 101, 102, etc., are merely used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, the flowcharts can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this paper are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types.

[0061] The existing laser radar and vision fusion scheme has a fundamental limitation in dealing with linear obstacle avoidance: the perception layer relies on a single physical characteristic. When rain and fog weaken laser reflection or dark light shields visual texture, the system lacks a cross-modal compensation mechanism, resulting in a perception fault, which leads to continuous missing of slender targets. The decision layer constructs a static map through timestamp alignment, which cannot express the dynamic spatial relationship between the unmanned aerial vehicle and the linear obstacle in real time when the unmanned aerial vehicle moves at high speed, causing the avoidance command to lag. The planning layer simplifies the linear obstacle to a point obstacle, ignoring the strong constraint of the spatial extension characteristic of the linear obstacle on the avoidance direction, generating a dangerous approaching trajectory parallel to the direction of the wire. The three defects jointly restrict the reliability of obstacle avoidance in complex scenes.

[0062] To overcome the above-mentioned defects, the present application proposes an unmanned aerial vehicle obstacle avoidance method based on multi-modal collaborative perception and linear feature directional decision: first, through the spatiotemporal complementarity of optical, millimeter wave, and infrared data, the system automatically fuses millimeter wave distance points and infrared thermal radiation features to compensate for the space when optical data fails, eliminating the perception blind area; based on the compensation data, the geometric distribution characteristics of the linear obstacle are analyzed in real time, a dynamic topology model based on the unmanned aerial vehicle pose is constructed, and a millisecond-level spatial relationship response is achieved; finally, according to the real-time angle change between the obstacle extension direction and the heading direction, the avoidance command is dynamically generated, and the curvature and distribution density parameters are combined to output the avoidance trajectory perpendicular or at a large angle to the obstacle extension direction. This method completely solves the missing problem caused by the perception fault, the response lag caused by static modeling, and the parallel approaching risk caused by homogeneous strategies, significantly improving the robustness and safety of linear obstacle avoidance in complex environments.

[0063] The technical solution of the present application can be applied to cable / road cable type linear obstacle scenes.

[0064] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0065] Figure 1 A flow chart of a method for autonomous obstacle avoidance of a UAV based on multi-sensor fusion is provided in the embodiments of the present application, as shown in FIG. 1, the method comprises the following steps. Figure 1

[0066] 101, in the process of UAV flight, synchronously acquiring three types of heterogeneous sensing data from an optical camera unit, a millimeter wave detection unit and an infrared perception unit;

[0067] In the above scheme, the optical camera unit refers to a device for imaging through the visible spectrum, which is used to capture the geometric profile of linear obstacles. The millimeter wave detection unit refers to a sensor that emits electromagnetic waves of 30-300 GHz, which uses the characteristics of penetrating low-visibility media such as rain, fog and dust to obtain the three-dimensional spatial position coordinates of obstacles. The infrared perception unit refers to a sensor that receives the thermal radiation of an object, which extracts the thermal radiation distribution data by identifying the temperature difference characteristics of transparent or low reflectivity linear obstacles and the environment background. The three types of heterogeneous sensing data refer to the original information set output by the three types of sensors with different physical principles and different data dimensions, including optical image sequences, millimeter wave point cloud distance data and infrared thermal imaging matrices.

[0068] In the embodiments of the present application, first, the hardware clock signal of the UAV flight control system sends a synchronous acquisition instruction to the optical camera unit, the millimeter wave detection unit and the infrared perception unit, forcing the three types of sensors to start data capture at the same physical time. For example, the optical unit captures RGB image sequences at a frame rate of 30 fps, the millimeter wave unit transmits FMCW frequency modulation waves at a frequency of 100 Hz and receives echoes, and the infrared unit generates thermal radiation raw data based on a micro bolometer array at a frequency of 25 Hz, ensuring that the time sequence of the multi-source data is consistent.

[0069] ​Secondly, the physical characteristics of the raw data of each sensor are extracted: the optical image sequence is analyzed by edge gradient analysis and morphological closing operation to extract the continuous pixel boundary of linear obstacles and generate a binary mask profile; the millimeter wave echo signal is converted into a three-dimensional point cloud in the global coordinate system by calculating the scattering point distance and azimuth angle through fast Fourier transform combined with the UAV IMU pose data; the infrared raw data is corrected for non-uniformity and temperature calibration to map the gray value to the absolute temperature value, and output the temperature-spatial distribution matrix. For example, each row of the millimeter wave point cloud matrix contains three-dimensional data of distance, azimuth angle and pitch angle, and the elements of the infrared matrix correspond to the actual temperature from -40°C to 150°C.

[0070] Thirdly, the three types of data are aligned in space and time. Spatial alignment is to match the millimeter wave point cloud with the optical profile corner point through the iterative closest point algorithm, and the infrared temperature matrix is registered to the optical image coordinate system through affine transformation; time alignment is to compensate for the sampling rate difference based on the hardware timestamp using cubic spline interpolation to unify the same time sequence. For example, the 100Hz millimeter wave point cloud is interpolated to the 30Hz optical data time axis, so that all data at time t reflect the same instantaneous state. Finally, the heterogeneous data aligned in space and time are packaged to generate structured data packets for subsequent processing. For example, the data packet at time t contains an optical 512x512 binary mask (profile point set), a millimeter wave Nx3 point cloud matrix, and an infrared 64x64 temperature matrix, with a spatial coordinate error of <5cm and a time deviation of <1ms.

[0071] 102. Establish a data compensation rule based on the spatio-temporal coverage difference of the three types of heterogeneous sensing data. When the optical camera unit lacks sufficient light to cause image data loss, the data compensation rule is used to perform spatial interpolation compensation using the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit.

[0072] Optionally, step 102 can specifically include the following steps:

[0073] 1021. Detect the pixel gray value of the target region in the image data output by the optical camera unit. When the pixel gray value is lower than the preset illumination threshold and the missing area of the continuous pixels of the target region exceeds the set proportion, mark the boundary of the target region as a missing region boundary;

[0074] 1022. Calculate the joint coverage capability parameter based on the distance measurement point density distribution of the millimeter wave detection unit within the missing region boundary and the thermal feature intensity distribution of the infrared sensing unit in the missing region.

[0075] 1023. Based on the joint coverage capability parameter, spatial weight distribution is performed on the distance measurement points of the millimeter wave detection unit and the thermal feature intensity distribution points of the infrared sensing unit.

[0076] 1024、according to the allocation result, constructing an interpolation surface with the distance measurement points as geometric constraints and the thermal feature intensity distribution points as attribute constraints within the missing area boundary, and mapping the interpolation surface into the image data within the missing area boundary to complete the spatial interpolation compensation.

[0077] In the above scheme, the missing area boundary refers to the outer contour coordinate set of a continuous blank area in which a linear obstacle cannot be identified in an optical image due to insufficient illumination. The distance measurement point density distribution represents the number distribution of effective detection points of the millimeter wave detection unit in a unit area. The thermal feature intensity distribution refers to a set of spatial discrete points of the surface temperature values of the linear obstacle captured by the infrared perception unit. The joint coverage capability parameter is an index quantifying the spatial complementarity of millimeter wave and infrared data in the missing area, which is jointly determined by the distance point density and the uniformity of the thermal intensity distribution. The geometric constraint requires that the interpolation surface must pass through the spatial positions of the millimeter wave distance measurement points. The attribute constraint requires that the temperature attribute value of the interpolation surface is consistent with the infrared thermal feature distribution points.

[0078] In the embodiments of the present application, first, the optical data missing determination is performed through step 1021. The sum of the gray values of all pixels in a preset power line detection frame is divided by the total number of pixels to calculate the area average gray value. The area proportion of the continuous blank pixel block is calculated by scanning the connected domain and marking the continuous blank pixels. When the average gray value is less than 50 and the area proportion is greater than 0.3, the boundary tracking algorithm is used to extract the closed polygon coordinate sequence along the outer edge of the blank pixels. For example, in a high-voltage line scene, the average gray value in the detection frame is 45, and the 20x30 pixel blank area accounts for 33% of the detection frame. The boundary vertex coordinates [(10, 20), (10, 50), (40, 50), (40, 20)] are output.

[0079] Subsequently, the multi-modal data complementarity is calculated through step 1022. The 1m x 1m grid is divided within the missing boundary, and the number of effective distance points in each grid is counted. The standard deviation of the temperature values of all thermal feature points is calculated. The joint coverage capability parameter is dynamically quantified according to the formula: joint coverage capability parameter = grid average point number x (1- temperature standard deviation / 15). For example, the grid average point number in a certain area is 5 points / m2, and the temperature standard deviation is 8°C. Therefore, the parameter = 5 x (1-8 / 15) = 2.33.

[0080] At the same time, the data weight is dynamically allocated through step 1023. The missing area is divided into 0.2m x 0.2m subgrids. If there are more than 3 distance points in a subgrid, a geometric weight of 0.7 is allocated. If the temperature standard deviation of the subgrid is less than or equal to 5°C, an attribute weight of 0.3 is allocated. When both conditions are met, the total weight is 1.0. For example, a subgrid contains 4 millimeter wave points (geometric weight 0.7) and a temperature standard deviation of 4°C (attribute weight 0.3), and the total weight = 1.0.

[0081] Finally, the optical data is reconstructed by step 1024, with the three-dimensional coordinates of the millimeter wave distance points as nodes, forcing the interpolation surface to pass through these spatial positions; with the infrared temperature values as control points, constraining the surface temperature distribution; combining the node data according to the sub-grid weights, generating a continuous surface through radial basis function interpolation; linearly converting the surface temperature values to gray scale values, with an increase of 6 gray scale values for every 1°C increase in temperature. For example, a 25°C position maps to a gray scale value of 120, and a 28°C position maps to a gray scale value of 138, reconstructing the power line profile.

[0082] In actual application, when the unmanned aerial vehicle patrols the 500kV high-voltage line at dusk, in step 1021, the preset power line detection frame (coordinate range [100:300, 150:350]) in the 640×480 image captured by the optical camera unit appears abnormal. The system calculates the average gray scale value of the pixels in this area: the sum of the gray scale values of the 96,000 pixels in the area (∑gray scale = 3,840,000) is divided by the total number of pixels to obtain the average value 40, which is greater than the threshold value 50; scanning finds that there is a 50×40 pixel continuous blank block in the upper left corner, which accounts for 0.35>0.3 of the detection frame area; the Moore-Neighbor boundary tracking algorithm is used to extract the closed polygon vertex coordinates [(100, 200), (100, 240), (150, 240), (150, 200)] along the outer edge of the blank pixels.

[0083] Through step 1022, multi-modal data analysis is performed within the marked boundary, and the 200㎡ missing area is divided into 1m×1m grids, and the number of valid points in each grid is counted, such as the grid (110-111, 210-211) containing 3 points, the average density is calculated as 4.2 points / ㎡; the temperature values of 32 thermal feature points {26.3°C, 27.1°C,..., 28.5°C} are extracted, and the standard deviation σ = 7°C is calculated; the joint coverage capability parameter = 4.2×(1-7 / 15) = 2.24, quantifying the complementarity of the data.

[0084] Through step 1023, the missing area is divided into 5,000 sub-grids of 0.2m×0.2m; the grid (110.0-110.2, 210.0-210.2) containing 6 distance points is assigned a geometric weight of 0.7; the grid (149.6-149.8, 239.6-239.8) with a temperature standard deviation of 3°C is assigned an attribute weight of 0.3; the grid that meets both conditions is assigned a total weight of 1.0, such as (120.0-120.2, 220.0-220.2) containing 4 points + standard deviation 4°C.

[0085] By step 1024, take the millimeter wave point (110.1, 210.3, 15.2m) as the spatial anchor point, force the curved surface to pass through this position; take the infrared point (149.7, 239.8) with a temperature of 28.2°C as the reference; use radial basis function interpolation, generate a continuous curved surface according to the sub-grid weight, the basis function φ(r)=r²log(r); establish a linear relationship between temperature and gray scale, the slope k=6 gray / °C, the intercept b=-30, map 28.2°C to gray scale 141.6, fill in the original blank area to reconstruct the wire shape.

[0086] The overall scheme of the above step 102 triggers multi-modal compensation through an accurate optical failure judgment mechanism, constructs a double-constrained curved surface based on the spatial distribution characteristics of the millimeter wave point cloud and the physical properties of the infrared thermal features, restores the accurate geometric shape and surface characteristics of high-voltage lines and other linear obstacles under strong light interference, and guarantees the perception continuity of the obstacle avoidance system in complex environments.

[0087] 103. Based on the compensated perception data, analyze the dynamic characteristics of the linear obstacles existing in the flight path, extract the extension direction, spatial curvature, and continuous distribution density parameters of the linear obstacles in the analysis results, and generate an obstacle topology graph in a three-dimensional spatial coordinate system with the current position of the unmanned aerial vehicle as the origin;

[0088] Optionally, step 103 can specifically include the following steps:

[0089] 1031. Divide the compensated perception data into multiple independent spatial units, and according to the geometric shape characteristics of the spatial units in the three-dimensional space, screen out the target units corresponding to the linear obstacles;

[0090] 1032. Perform main direction analysis on the target units to determine the extension direction of the target units in the three-dimensional space, and continuously fit the extension direction according to the spatial adjacent relationship to generate the overall extension direction of the linear obstacles;

[0091] In step 1032, the process can specifically include the following: perform three-dimensional spatial distribution analysis on the geometric characteristics of the target units, and determine the longest extension axis direction of the target units in the three-dimensional space as the main direction according to the analysis results; establish a direction association relationship between adjacent target units according to the main direction, and mark the adjacent target units with the direction association relationship as direction association units; adjust the main direction of the direction association units for consistency, and through the adjustment process, the main direction of the direction association units is gradually decreased to within a preset allowable deviation range according to the spatial adjacent order; connect the main direction of the adjusted direction association units according to the spatial adjacent order to form the continuous extension direction of the linear obstacles as the overall extension direction.

[0092] 1033、According to the angle deviation value of the extension direction between adjacent target units, a bending degree parameter of the linear obstacle in three-dimensional space is calculated as a spatial curvature, wherein the angle deviation value of the extension direction is determined by the included angle of the main direction of adjacent target units;

[0093] 1034、The number distribution of the target units in a unit volume in a three-dimensional space coordinate system with the current position of the unmanned aerial vehicle as the origin is counted, a continuous distribution density parameter of the linear obstacle is generated according to the number distribution, and the overall extension direction, the spatial curvature and the continuous distribution density parameter are mapped into the three-dimensional space coordinate system to form an obstacle topology map.

[0094] In the above scheme, the space unit refers to a cubic data block divided by fixed volume according to the perception data. The target unit refers to a space unit with a geometric shape conforming to a linear feature. The extension direction represents the main axis line orientation vector of the target unit in three-dimensional space. The continuity fitting refers to generating a global direction by smooth transition constraint of the direction vectors of adjacent units. The extension direction angle deviation value is the absolute value of the included angle of the main direction vectors of adjacent target units. The bending degree parameter is quantified by the average value of the angle deviation. The continuous distribution density parameter is a spatial distribution function of the number of target units in a unit volume.

[0095] In the embodiments of the present application, first, spatial segmentation and feature screening are performed by step 1031: an octree structure is used to recursively subdivide the three-dimensional space, and 1m³ is used as the initial voxel to divide the compensated millimeter wave point cloud and infrared temperature data in space; when the number of point clouds in the voxel is > 50, continue to subdivide to the smallest 0.125m³ unit to ensure that each space unit contains an independent geometric structure; the eigenvalues (λ1≥λ2≥λ3) of the point cloud covariance matrix in the unit are calculated, and if the linearity index IL=(λ1-λ2) / λ1>0.8 and the aspect ratio λ1 / λ2>5:1, it is determined as a linear target unit. For example, in the high-voltage line point cloud unit, λ1=15.2, λ2=1.8, λ3=0.5→IL=(15.2-1.8) / 15.2=0.88>0.8, and the aspect ratio is 8.4:1, which is marked as a target unit.

[0096] Subsequently, the direction resolution and global fitting are completed by step 1032: the target unit point cloud is singular value decomposed (SVD), the right singular vector corresponding to the maximum singular value is taken as the extension direction vector v=(vx, vy, vz), and the modulus is normalized to a unit vector; the target units are connected in spatial adjacent order (such as A→B→C), the adjacent unit direction vector dot product cosθ=vA·vB is calculated, if |θ|<15°, the spherical linear interpolation (slerp) is used to calculate the intermediate direction: vmid=[sin((1-t)Ω)vA+sin(tΩ)vB] / sinΩ(where cosΩ=vA·vB, t=0.5). For example, the high-voltage line unit A direction (0.87, 0.50, 0), unit B (0.90, 0.44, 0), and the interpolated intermediate direction (0.885, 0.47, 0) are constructed to build the overall trend.

[0097] At the same time, the spatial bending characteristics are quantified by step 1033: the direction vector sequence {v1, v2, v3, v4, v5} of the continuous 5 adjacent target units is taken, the adjacent vector included angle θi=arccos(vi·vi+1) is calculated; the moving average κ=(∑θi) / 4 is calculated for the included angle sequence to eliminate local fluctuations. For example, the high-voltage line turns at θ1=5°, θ2=7°, θ3=9°, θ4=11°→curvature κ=(5+7+9+11) / 4=8°.

[0098] Finally, the dynamic topology graph is constructed by step 1034: taking the current position of the unmanned aerial vehicle as the origin, a 1m×1m×1m voxel grid is established, and the number of target units in each voxel Nvoxel is counted; the overall extension direction vector, curvature value κ, and density value Nvoxel are associated to the voxel center coordinates P(x, y, z), and stored as a topology graph node Node={P, v, κ, N}. For example, the voxel (15, 25, 50) contains 6 target units→stores Node={(15, 25, 50), (0.71, 0.71, 0), 8°, 6}.

[0099] In practical applications, in the 500 kV high-voltage line inspection scene, the unmanned aerial vehicle flies at a height of 50 meters, and obtains the compensated millimeter wave point cloud and infrared temperature data. Through step 1031, octree recursive segmentation is adopted: the initial division is 1 m³ voxel, and the number of point clouds in the detection unit (32.1, 45.3, 52.7) is 68>50 threshold; 8 0.5 m³ sub-units are divided, and the sub-unit (32.1-32.6, 45.3-45.8, 52.7-53.2) contains high-voltage line point clouds; the eigenvalues of the covariance matrix of the sub-unit are calculated: λ1=12.8, λ2=1.5, λ3=0.3; the linearity index IL=(12.8-1.5) / 12.8=0.88>0.8, and the aspect ratio λ1 / λ2=8.5:1>5:1; it is marked as a target unit, and the geometric center coordinates (32.35, 45.55, 52.95) are output.

[0100] Through step 1032, based on the target unit sequence output by step 1031: singular value decomposition (SVD) is performed on the unit (32.35, 45.55, 52.95) point cloud, and the maximum singular value corresponds to the right singular vector vA=(0.91, 0.41, 0.02); the direction of the adjacent unit (32.85, 46.05, 52.95) is vB=(0.89, 0.45, 0.02); the dot product of the directions is calculated: vA·vB=0.91×0.89+0.41×0.45+0.02×0.02=0.998; the included angle θ=arccos(0.998)≈3.2°<15°, the spherical linear interpolation is enabled: the interpolation parameter t=0.5, Ω=3.2° vmid=[sin((1-0.5)×3.2°)×vA+sin(0.5×3.2°)×vB] / sin(3.2°), the intermediate direction (0.90, 0.43, 0.02) is obtained; 10 groups of adjacent unit interpolation directions are connected in series to generate the overall trend of the high-voltage line N25°E (direction vector (0.82, 0.57, 0)).

[0101] Through step 1033, 5 target units are selected along the extension direction of the high-voltage line: the direction vector v1 of the unit coordinate (32.35, 45.55, 52.95) is (0.91, 0.41, 0.02), the adjacent angle θ1 = arccos (v1·v2) = 3.2°; the direction vector v2 of the unit coordinate (32.85, 46.05, 52.95) is (0.89, 0.45, 0.02), the adjacent angle θ2 = arccos (v2·v3) = 4.1°; the direction vector v3 of the unit coordinate (33.35, 46.55, 52.95) is (0.87, 0.49, 0.02), the adjacent angle θ3 = arccos (v3·v4) = 5.3°; the direction vector v4 of the unit coordinate (33.85, 47.05, 52.95) is (0.85, 0.53, 0.02), the adjacent angle θ4 = arccos (v4·v5) = 6.8°; the direction vector v5 of the unit coordinate (34.35, 47.55, 52.95) is (0.82, 0.57, 0.02). The spatial curvature parameter κ = (θ1+θ2+θ3+θ4) / 4 = (3.2+4.1+5.3+6.8) / 4 = 4.85°.

[0102] Through step 1034, taking the current position (0, 0, 50) of the unmanned aerial vehicle as the origin: a 1m³ voxel grid is established, and the voxel (30-31, 45-46, 50-51) covers 5 target units; the statistical density parameter Nvoxel = 5; the associated feature parameters: the extension direction is the overall strike vector (0.82, 0.57, 0), the spatial curvature κ = 4.85°, and the distribution density is 5 units / m³; the topological graph node is generated: Node = {coordinate (30.5, 45.5, 50.5), direction (0.82, 0.57, 0), curvature 4.85°, density 5}.

[0103] The overall scheme of the above step 103 accurately extracts linear targets through recursive spatial segmentation and geometric feature constraints, generates a continuous and smooth extension direction based on singular value decomposition and spherical interpolation, quantifies the spatial bending characteristics by using the moving average angle, and constructs a dynamic topological graph by combining voxel density statistics, thereby providing the unmanned aerial vehicle with fine spatial feature description for complex linear obstacles such as high-voltage lines and cables.

[0104] 104. According to the real-time dynamic relationship between the extension direction in the obstacle topological graph and the current flight direction of the unmanned aerial vehicle, the extension direction and the heading angle change are converted into an avoidance direction adjustment weight, and the spatial curvature and continuous distribution density parameters are combined to generate a continuous deflection instruction for the obstacle avoidance heading;

[0105] Optionally, step 104 can specifically include the following steps:

[0106] 1041、real-time acquisition of a real-time included angle value between the extension direction in the obstacle topology map and a current flight heading of the unmanned aerial vehicle, and calculation of a difference value between the real-time included angle value and an included angle at a previous time as an included angle change amount according to the extension direction;

[0107] The step 1041 can specifically include the following process: an extension direction vector of a linear obstacle is extracted from the obstacle topology map, and a current flight heading vector output by a flight control unit of the unmanned aerial vehicle is synchronously acquired; a plane projection included angle between the extension direction vector and the flight heading vector in a three-dimensional space is calculated as a real-time included angle value; a difference value between the real-time included angle value and a historical included angle value stored at a previous time is calculated to obtain an included angle change amount at a next time; when the real-time included angle value is acquired for the first time, the included angle change amount is initialized to zero, and the historical included angle value is continuously updated to a current real-time included angle value at a subsequent time.

[0108] 1042、generating a curvature influence coefficient and a density influence coefficient according to the spatial curvature and the continuous distribution density parameter, wherein the curvature influence coefficient is in a positive correlation with the spatial curvature, and the density influence coefficient is in a negative correlation with the continuous distribution density parameter;

[0109] 1043、inputting the included angle change amount, the curvature influence coefficient and the density influence coefficient into a preset dynamic relationship evaluation parameter table, and determining an avoidance direction adjustment weight through a table lookup method;

[0110] 1044、generating an avoidance direction component orthogonal to the extension direction in a vertical plane of a current flight heading of the unmanned aerial vehicle according to the avoidance direction adjustment weight, and superimposing the avoidance direction component and the current heading to generate a continuous deflection instruction.

[0111] In the above scheme, the real-time included angle value refers to a plane projection included angle between an extension direction vector and a flight heading vector at a current time. The included angle change amount is an algebraic difference value between a current included angle and a historical included angle at a previous 1 second. The curvature influence coefficient is a linear function of the spatial curvature. The density influence coefficient is an inverse function of the distribution density. The dynamic relationship evaluation parameter table is a three-dimensional mapping table with the included angle change amount, the curvature coefficient and the density coefficient as input dimensions and a weight value as an output. The orthogonal avoidance component refers to a direction vector perpendicular to the extension direction and located in the heading vertical plane.

[0112] In the embodiment of the present application, first, the dynamic angle change amount is calculated in real time through step 1041, the three-dimensional extension direction vector vobj of the linear obstacle is extracted from the obstacle topology graph, and the current heading vector vuav of the unmanned aerial vehicle output by the flight control system is synchronously read; the two vectors are projected onto the horizontal plane, the Z component is set to 0, and the plane projection angle is calculated according to the formula θt=arccos((vobjx·vuavx+vobjy·vuavy) / (√(vobjx²+vobjy²)×√(vuavx²+vuavy²))), wherein √ represents the square root, the result is in degrees, and the range is 0°~180°. The historical angle value θ{t-1} stored in the last control cycle is called, and the algebraic difference Δθ=θt-θ{t-1} between the current angle and the previous value is calculated, a positive value indicating that the unmanned aerial vehicle is approaching the extension direction of the obstacle, and a negative value indicating that the unmanned aerial vehicle is moving away. For example, the extension direction vector of the high-voltage line is (0.82, 0.57, 0), the heading of the unmanned aerial vehicle is (0.87, 0.50, 0), the projection calculation gives θt=5.2°, and the historical record θ{t-1}=3.1°→Δθ=2.1°.

[0113] Secondly, the risk influence coefficient is quantified through step 1042, the spatial curvature value κ of the topology graph node is read, and the curvature influence coefficient kc=0.1×κ is generated according to the linear relationship, the larger the curvature, the larger the coefficient, and the high-voltage line needs a larger avoidance range due to the sharp turn. The node distribution density value D is read, the unit is unit / m³, and the density influence coefficient kd=5 / D is generated according to the inverse relationship, the higher the density, the smaller the coefficient, and the dense cable group needs to be more cautious and small avoidance. For example, the high-voltage line node κ=4.85°→kc=0.485, and the density D=5 units / m³→kd=1.0.

[0114] Thirdly, the avoidance weight is determined through step 1043, a three-dimensional array weight[Δθ][kc][kd] is preset, wherein Δθ dimension: -30° to +30°, step 1°, covering the maximum angular velocity 300° / s of the unmanned aerial vehicle; kc dimension: 0~1.5, corresponding to curvature 0~15°; kd dimension: 0.2~1.0, corresponding to density 1~25 units / m³; Δθ is rounded to an integer degree, kc and kd are kept to one decimal place, the index array obtains the weight value w, and the range is 0~1.0. For example, Δθ=2.1°≈2°, kc=0.485≈0.5, kd=1.0→table lookup w=0.6.

[0115] Finally, the deflection instruction is generated through step 1044. The cross product n = vobj x vuav of the extension direction vobj and the heading vuav is calculated to obtain the normal vector of the vertical plane, and a unit vector vortho = normalize(n x vobj) orthogonal to vobj is generated in the plane defined by n. The avoidance component vavoid = w x vortho is scaled by the weight. The avoidance component is superimposed on the current heading vcmd = vuav + vavoid, and the output is the continuous deflection instruction. For example, when w = 0.6, vortho = (0.08, -0.06, 0) → vavoid = (0.048, -0.036, 0), and the superimposed heading (0.87, 0.50, 0) gives the new instruction (0.918, 0.464, 0).

[0116] In practical application, in the 500kV high-voltage line avoidance scene, the UAV flies at a speed of 12m / s, and the high-voltage line extension direction vector vobj = (0.82, 0.57, 0) is extracted from the topological graph through step 1041, corresponding to the N25°E strike; the flight control system outputs the current heading vector vuav = (0.87, 0.50, 0), N30°E direction; projection angle calculation: horizontal projection vobjproj = (0.82, 0.57, 0), vuavproj = (0.87, 0.50, 0); dot product operation dot = 0.82 x 0.87 + 0.57 x 0.50 = 0.7134 + 0.285 = 0.9984; length calculation |vobj| = sqrt(0.82² + 0.57²) = 1.0, |vuav| = sqrt(0.87² + 0.50²) = 1.0; real-time angle θt = arccos(0.9984) ≈ 5.2°. Change generation: read the historical value θt-1 = 3.1° at 100ms ago, algebraic difference Δθ = 5.2° - 3.1° = 2.1°, positive value indicating that the UAV is approaching the high-voltage line strike.

[0117] Through step 1042, the curvature influence coefficient: read the spatial curvature κ = 4.85°, linear mapping kc = 0.1 x κ = 0.1 x 4.85 = 0.485; obtain the distribution density D = 5 units / m³, inverse calculation: kd = 5 / D = 5 / 5 = 1.0.

[0118] Through step 1043, Δθ dimension: -30° to +30°, step 1°, covering 300° / s angular velocity; kc dimension: 0~1.5, corresponding to curvature 0~15°; kd dimension: 0.2~1.0, corresponding to density 1~25 units / m³. Input quantization: Δθ = 2.1° → 2°, kc = 0.485 → 0.5, kd = 1.0; output weight: w = 0.6.

[0119] By step 1044, the cross product method vector: n = vobj x vuav = i (0 x 0 - 0 x 0.50) - j (0 x 0 - 0.82 x 0) + k (0.82 x 0.50 - 0.57 x 0.87) = (0, 0, -0.1109);

[0120] The orthogonal direction is generated: n x vobj = i (0.57 x (-0.1109) - 0 x 0) - j (0.82 x (-0.1109) - 0 x 0) + k (0.82 x 0 - 0.57 x 0) = (-0.063, 0.090, 0);

[0121] Normalization: vortho = (-0.063 / 0.11, 0.090 / 0.11, 0) = (-0.573, 0.819, 0);

[0122] Weight scaling: vavoid = 0.6 x (-0.573, 0.819, 0) = (-0.344, 0.491, 0);

[0123] New heading: vcmd = (0.87, 0.50, 0) + (-0.344, 0.491, 0) = (0.526, 0.991, 0).

[0124] Heading angle change: original N30°E (tan-1(0.50 / 0.87) = 30°) → new heading N62°E.

[0125] The overall scheme of step 104 above dynamically captures the approaching trend of the UAV and the high-voltage line by projecting the angle change in real time, quantifies the avoidance demand based on the curvature and density physical characteristics, efficiently decides the weight using the preset parameter table, and finally generates an orthogonal avoidance component in the vertical plane of the heading, ensuring that the trajectory extends in the direction of the linear obstacle with a large angle, completely eliminating the risk of parallel approach.

[0126] 105、Based on the continuous deflection instruction, a multi-segment obstacle avoidance trajectory is generated, and the distance data and the thermal feature distribution data are fused when the UAV executes the multi-segment obstacle avoidance trajectory, the continuous deflection instruction is corrected in a closed loop through the fused data, and the correction process is repeated until the relative distance between the UAV and the linear obstacle reaches a safety threshold.

[0127] Optionally, step 105 can specifically include the following steps:

[0128] 1051、The continuous deflection instruction is decomposed into a trajectory segment parameter set, which contains a heading angle change and a corresponding flight speed value;

[0129] 1052、in the unmanned aerial vehicle according to the trajectory segment parameter set, the distance data and the thermal feature distribution data are acquired synchronously, and the fusion reliability of the trajectory segment parameter set is determined according to the overlapping proportion of the coverage range of the distance data and the intensity region of the thermal feature distribution data;

[0130] 1053、according to the fusion reliability, deviation evaluation is performed on the heading angle change and the corresponding flight speed value, and when the coverage range is lower than a preset proportion in the evaluation process, the center point of the intensity region of the thermal feature distribution data is used as a deviation evaluation reference;

[0131] 1054、based on the deviation evaluation result, the heading angle change and the flight speed value in the next trajectory segment parameter set are reversely compensated and adjusted, and after the adjustment, the generation process of the trajectory segment parameter set is repeatedly performed until the minimum relative distance between the unmanned aerial vehicle and the linear obstacle in the distance data reaches the safety threshold.

[0132] In the above scheme, the trajectory segment parameter set refers to a segmented control parameter group formed by discretizing continuous heading instructions, each group containing the heading angle adjustment and the flight speed of a single trajectory segment. The coverage range refers to the proportion of the distribution area of the effective distance measurement points of the millimeter wave detection unit in the trajectory segment space. The intensity region refers to the continuous region in which the surface temperature of the linear obstacle detected by the infrared perception unit is significantly higher than the environmental background. The overlapping proportion is the proportion of the overlapping area of the distance data coverage range and the thermal feature intensity region to the union area of the two. The fusion reliability is the sensor data reliability score calculated by the overlapping proportion. The deviation evaluation reference is a reference position for quantifying trajectory execution error, and the center point of the consistent region of multiple sensors is preferentially selected. The reverse compensation adjustment refers to the reverse correction of the heading angle and the speed value of the next segment according to the deviation direction and amplitude of the current trajectory segment.

[0133] In the embodiment of the application, first, the instruction decomposition is performed by step 1051 to cut continuous deflection instructions into multiple segments at a fixed time interval, each segment corresponding to a fixed actual flight distance of the unmanned aerial vehicle; the mean value of the heading angle change in each time window is taken, such as 0-0.5 seconds, the heading angle is from 30° to 32.1°, and Δψ=+2.1°, and the speed value is taken at the starting point of the window; finally, the trajectory segment parameter set is generated, for example, the first segment parameter is [heading angle change +2.1°, speed 12 m / s].

[0134] Subsequently, the fusion confidence is calculated through step 1052: during the trajectory segment execution process, the distance point cloud of the millimeter wave detection unit and the infrared thermal imaging data are collected in real time, and are aligned to the same time through time stamp; in the space region corresponding to the trajectory segment, the proportion of the distribution area of the millimeter wave effective measurement points with a signal-to-noise ratio greater than 20 dB to the total monitoring region is calculated; the infrared thermal image is subjected to fixed threshold segmentation, and a temperature greater than 50°C is determined as a high-voltage line, and a binary mask profile of the temperature significant region is extracted; the millimeter wave point cloud is projected to the infrared image coordinate system, and the proportion of the overlapping area of the two to the total area of the infrared high-temperature region is calculated; and a numerical value (range 0~1.0) is output according to the weighted formula confidence=0.7×coverage proportion+0.3×overlap proportion. For example, the millimeter wave coverage proportion is 80%, and the overlap proportion is 70%→confidence=0.7×0.8+0.3×0.7=0.77.

[0135] At the same time, through step 1053 dynamic deviation evaluation, when the millimeter wave coverage proportion is less than 60%, the centroid coordinates of the infrared high-temperature region are taken as the reference point; otherwise, the center point of the millimeter wave and infrared overlapping region is taken; the theoretical direction vector is generated by connecting the current position of the unmanned aerial vehicle and the reference point, and the angle deviation is calculated by comparing the actual heading vector, such as theoretical direction 35° vs. actual heading 32°→deviation +3°; the difference between the actual displacement and the planned displacement of the unmanned aerial vehicle is measured, and the speed correction amount is calculated in combination with the time window, such as lagging behind 0.8 meters in 0.5 seconds→speed needs to be increased by 1.6 m / s.

[0136] Finally, through step 1054 closed-loop adjustment, the heading angle compensation is the change amount of the next segment planned heading angle minus the current deviation value, such as current deviation +3°, next segment original planned +4.3°→modified to +1.3°; the speed compensation is the next segment speed value plus the current speed deviation amount, such as original speed 12 m / s→modified to 13.6 m / s; the new trajectory segment is generated with the modified parameters, and steps 1052-1054 are repeatedly executed; when the millimeter wave detects that the minimum distance between the unmanned aerial vehicle and the high-voltage line is less than or equal to 3 meters, the modification is terminated.

[0137] In actual application, in the process of avoiding 500 kV high-voltage lines by the unmanned aerial vehicle, through step 1051, the instruction segmentation and parameterization are executed, the continuous deflection instructions are cut into 0.5 second time windows, and each window corresponds to a flight distance of 6 meters (12 m / s speed). The heading angle of the first segment 0-0.5 seconds increases from 30.0° to 32.1°, and the average change amount is +2.1°; the instantaneous speed of the window starting point is 12 m / s, and is packaged as the trajectory segment 1 parameter set [+2.1°, 12 m / s]. Similarly, segment 2 [+4.3°, 12 m / s] and segment 3 [+6.4°, 12 m / s] are generated, and the instruction executable conversion is completed.

[0138] The multi-source data fusion credibility is calculated by step 1052. When performing section 1, in the spatial region (x=30-36m, y=45-48m, z=50-52m): the millimeter wave effective point covers 28.8m³, the monitoring area is 36m³, the coverage ratio is 80%; the infrared thermal image is segmented by a 50°C threshold to extract a 25.2m² high-temperature area; the coincident area is measured to be 18.9m² by registering the point cloud and the thermal image through rigid transformation, and the overlap ratio is 75%; the fusion credibility is synthesized to be 78.5% according to the weighted formula “0.7×coverage ratio+0.3×overlap ratio”, which provides data reliability basis for bias evaluation.

[0139] The dynamic bias quantification is completed by step 1053. Based on the fusion credibility of 78.5% and the coverage ratio of 80%>60% threshold, the center point (32.1, 45.3, 52.7) of the coincident area is selected as the reference; the theoretical direction vector (2.1, 0.3, 0.2) is generated by comparing the actual end point (31.9, 45.1, 52.5) of the unmanned aerial vehicle with the reference point, the horizontal projection heading angle is 8.1°; a +24.0° heading bias is generated compared with the actual heading of 32.1°; the difference between the planned displacement of 6 meters and the actual displacement of 1.90 meters is calculated to be 4.10 meters, and combined with the 0.5 second window, the speed compensation amount of 8.2m / s is obtained.

[0140] The closed-loop correction and termination are realized by step 1054. The parameters of section 2 are implemented in reverse compensation: the heading angle change is corrected to +4.3°-24.0°=-19.7°, and the speed is corrected to 12+8.2=20.2m / s; after executing the corrected parameters, the millimeter wave distance is monitored in real time, and it is detected that the minimum distance to the high-voltage line decreases from 3.2 meters to 2.8 meters, which is lower than the safety threshold of 3.0 meters, so the correction process is terminated immediately and the safety hovering mode is triggered, and the obstacle avoidance trajectory closed-loop optimization is completed.

[0141] The overall scheme of the above step 105 realizes the executable of the instruction through time window discretization, dynamically evaluates the credibility based on the spatial consistency of millimeter wave and infrared data, intelligently selects the bias reference point combined with the coverage ratio threshold, and finally realizes the closed-loop trajectory correction through the reverse compensation mechanism of heading and speed, ensuring that the unmanned aerial vehicle always maintains a safe distance when avoiding linear obstacles such as high-voltage lines.

[0142] The following is a complete embodiment for steps 101-105:

[0143] As Figure 2As shown, assuming the UAV flies in a 500kV high-voltage line inspection scene, the height is 50m, and the speed is 12m / s. Through step 101, multi-modal data synchronous acquisition is realized: the optical camera unit captures visible light images at 30 frames per second, and identifies the continuous contour of the 3cm diameter high-voltage line, such as the elongated pixel chain with a gray value of 120, with the length direction extending along N25°E; the millimeter wave detection unit transmits a 77GHz frequency-modulated continuous wave through the thin fog, generates a high-voltage line spatial point cloud with an accuracy of 0.1 meters and a point density of 5 points per square meter, including key positions such as coordinates (110.2, 210.5, 15.3); the infrared perception unit detects the cable surface temperature through 14-micron waveband thermal imaging, outputs a 64x64 thermal feature matrix, and the high-voltage line area temperature of 45°C forms a significant temperature difference with the environment background of 30°C; the three types of data are triggered by hardware clock synchronization, and the millimeter wave point cloud and infrared thermal map are registered to the optical image coordinate system using the iterative closest point algorithm, with a time deviation controlled within 1 millisecond and a spatial registration error less than 5 centimeters.

[0144] Through step 102, optical failure compensation is performed. When the UAV enters a strong backlight area, i.e. the light intensity decreases to 40 lux, the high-voltage line segment in the optical image has a coordinate range of x=100-150, y=200-240, an average gray value of 40 and a 35% contour loss: based on the preset rules, 32 effective distance points of the millimeter wave are counted in the missing area, with a distribution density of 4.2 points per square meter, an infrared thermal feature point temperature standard deviation of 7°C, and a maximum allowed deviation of 15°C; the joint coverage capability parameter = 4.2 x (1-7 / 15) = 2.24 is calculated; the missing area is divided into a 5x4 grid, the left upper sub-grid with a millimeter wave point density of 6 points per square meter is assigned a geometric weight of 0.7, and the right lower sub-grid with an infrared temperature standard deviation of 3°C is assigned an attribute weight of 0.3; taking the millimeter wave point (110.1, 210.3, 15.2) as the geometric constraint node and the infrared point (149.7, 239.8) with a temperature of 28.2°C as the attribute constraint node, a continuous surface is generated by radial basis function interpolation, the temperature value is mapped and filled into the missing area at a slope of 6 gray / °C, the complete shape contour of the high-voltage line is reconstructed, and the error between the reconstructed contour point coordinate set and the original shape is less than 0.15 meters.

[0145] The linear feature dynamic analysis is completed by step 103. Based on the compensated data, the three-dimensional space is recursively divided by octree, the initial voxel is 1 cubic meter, the minimum unit is 0.125 cubic meter, the target unit with linear index > 0.8 and aspect ratio > 5:1 is screened, the eigenvalue of high-voltage line unit is λ1=12.8, λ2=1.5, the linear degree is 0.88, and the aspect ratio is 8.5:1; singular value decomposition is performed on the point cloud of the unit (32.35, 45.55, 52.95), the maximum singular value corresponds to the right singular vector (0.91, 0.41, 0.02) as the local extension direction, the direction vector dot product of adjacent units is 0.998, and the included angle is 3.2°<15°. When the included angle is 3.2°<15°, the spherical linear interpolation is used to generate the intermediate direction (0.90, 0.43, 0.02), and the overall trend of the high-voltage line is formed in series N25°E, and the global direction vector is (0.82, 0.57, 0); the continuous 5 units are selected along the extension direction to calculate the adjacent direction included angle (3.2°+4.1°+5.3°+6.8°=19.4°), and the spatial curvature is 4.85° / m by moving average; the voxel grid with a size of 1 cubic meter is established with the real-time position of the unmanned aerial vehicle (0, 0, 50) as the origin, the distribution density of 5 target units in the voxel (30-31, 45-46, 50-51) is calculated to be 5 units / cubic meter, and finally the topological graph node {coordinate (30.5, 45.5, 50.5), direction (0.82, 0.57, 0), curvature 4.85°, density 5} is constructed.

[0146] The real-time avoidance instruction is generated by step 104. The extension direction vector (0.82, 0.57, 0) of the high-voltage line is extracted from the topological graph, and the real-time included angle is calculated by projecting the current heading (0.87, 0.50, 0) of the unmanned aerial vehicle onto the horizontal plane, which is 5.2°; the change amount Δθ=+2.1° is obtained by calling the historical included angle 3.1° 100 milliseconds ago, and the trend is approximated; the influence coefficient k_c=0.1×4.85=0.485 is generated by reading the node curvature 4.85°, and the density 5 units / cubic meter generates the coefficient k_d=5 / 5=1.0; the parameters (Δθ=2°, k_c=0.5, k_d=1.0) are input into the preset three-dimensional mapping table, and the index output avoidance weight is 0.6; the vector (0, 0, -0.1109) is constructed in the vertical plane by the vector cross product algorithm, the orthogonal unit vector (-0.573, 0.819, 0) is constructed, and the avoidance component (-0.344, 0.491, 0) is obtained by scaling according to the weight; the new instruction vector (0.526, 0.991, 0) is generated by superimposing the original heading, the heading angle is adjusted from 30° to 62°, and the included angle with the high-voltage line trend is expanded to 37°.

[0147] The trajectory closed-loop correction is realized by step 105: the continuous instruction is decomposed into 3 segment trajectories, each segment is 0.5 seconds / 6 meters of flight distance: segment 1 parameter [heading angle change + 2.1°, speed 12 m / s], segment 2 [+4.3°, 12 m / s], segment 3 [+6.4°, 12 m / s]; when executing segment 1, in the corresponding space region (30-36m, 45-48m, 50-52m), the millimeter wave effective point covers 28.8 cubic meters, the monitoring area is 36 cubic meters, the coverage ratio is 80%; the infrared thermal feature extraction is 25.2 square meters of high temperature area, the coincident area after rigid body registration is 18.9 square meters, the overlap ratio is 75%; the fusion confidence is 78.5% according to the weight 0.7x coverage ratio + 0.3x overlap ratio; accordingly, the center point (32.1, 45.3, 52.7) of the coincident area is selected as the reference, the heading deviation +24.0° and displacement lag 4.10 meters are generated by comparing the actual position of the unmanned aerial vehicle (31.9, 45.1, 52.5), the speed needs to be increased by 8.2 m / s; the segment 2 parameter is compensated in reverse: the heading angle is adjusted to +4.3°-24.0°=-19.7°, and the speed is increased to 20.2 m / s; when executing the corrected trajectory, the minimum distance detected by the millimeter wave in real time is reduced to 2.8 meters, which is lower than the safety threshold of 3.0 meters, so the correction is immediately terminated and the hovering is triggered.

[0148] In the high-voltage transmission line avoidance scene, the application effectively eliminates the sensing blind area caused by light attenuation and weather interference through the spatiotemporal complementary mechanism of multi-modal sensors; the fine dynamic modeling of linear obstacles is realized based on octree segmentation and geometric feature constraints; the directional avoidance instruction is generated by using the real-time dynamic relationship between the extension direction and the heading, ensuring that the trajectory vertically crosses the obstacle direction; finally, through the confidence evaluation and reverse compensation mechanism of multiple segment trajectories, the closed-loop correction is realized until the safety distance threshold is met, significantly improving the robustness and safety of the unmanned aerial vehicle autonomous obstacle avoidance in complex environments.

[0149] Figure 3 A structure diagram of an unmanned aerial vehicle autonomous obstacle avoidance system based on multi-sensor fusion is provided for an embodiment of the application, as shown in Figure 3 The system comprises:

[0150] The acquisition module 31 is used for synchronously acquiring three types of heterogeneous sensing data from the optical camera unit, the millimeter wave detection unit and the infrared perception unit during the flight of the unmanned aerial vehicle;

[0151] The compensation module 32 is used for establishing a data compensation rule according to the spatiotemporal coverage difference of the three types of heterogeneous sensing data; when the optical camera unit causes image data loss due to insufficient light, the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared perception unit are used for spatial interpolation compensation through the data compensation rule;

[0152] The analysis module 33 is configured to analyze the dynamic characteristics of the linear obstacle existing in the flight path based on the compensated perception data, extract the extension direction, spatial curvature and continuous distribution density parameters of the linear obstacle in the analysis result, and generate an obstacle topology graph in a three-dimensional coordinate system with the current position of the UAV as the origin;

[0153] The conversion module 34 is configured to convert the extension direction and the change amount of the included angle between the extension direction and the flight direction of the UAV into an avoidance direction adjustment weight according to the real-time dynamic relationship between the extension direction in the obstacle topology graph and the flight direction of the UAV, combine the spatial curvature and the continuous distribution density parameters, and generate a continuous deflection instruction of the obstacle avoidance direction;

[0154] The correction module 35 is configured to generate a multi-segment obstacle avoidance trajectory based on the continuous deflection instruction, fuse the distance data and the thermal feature distribution data when the UAV executes the multi-segment obstacle avoidance trajectory, and correct the continuous deflection instruction through the fused data in a closed loop, and repeat the correction process until the relative distance between the UAV and the linear obstacle reaches a safety threshold.

[0155] Figure 3 The UAV autonomous obstacle avoidance system based on multi-sensor fusion can perform Figure 1 The UAV autonomous obstacle avoidance method based on multi-sensor fusion of the embodiment has the same implementation principles and technical effects as the UAV autonomous obstacle avoidance system based on multi-sensor fusion. The specific operation modes of each module and unit of the UAV autonomous obstacle avoidance system based on multi-sensor fusion in the above embodiment have been described in detail in the embodiment of the method, and will not be described in detail here.

[0156] In one possible design, Figure 3 The UAV autonomous obstacle avoidance system based on multi-sensor fusion of the embodiment can be implemented as a computing device, such as a computer. Figure 4 As shown, the computing device can include a storage component 41 and a processing component 42.

[0157] The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 42.

[0158] The processing component 42 is configured to perform the above Figure 1 The UAV autonomous obstacle avoidance method based on multi-sensor fusion of the embodiment.

[0159] The processing component 42 can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components, configured to perform the methods described above.

[0160] The storage component 41 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0161] Of course, the computing device can also include other components, such as an input / output interface, a display component, a communication component, etc.

[0162] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0163] The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0164] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform, and the computing device can be a cloud server, and the processing component, the storage component, etc. can be a basic server resource rented or purchased from the cloud computing platform.

[0165] The embodiment of the application further provides a computer storage medium, which stores a computer program, and the computer program can implement the above Figure 1 An unmanned aerial vehicle autonomous obstacle avoidance method based on multi-sensor fusion is provided.

[0166] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0167] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0168] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0169] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-sensor fusion based method for autonomous obstacle avoidance of a UAV, characterized in that, The application relates to a method for realizing dynamic obstacle avoidance of a UAV (unmanned aerial vehicle) in flight. During the flight of the UAV, three types of heterogeneous sensing data from an optical camera unit, a millimeter wave detection unit and an infrared sensing unit are synchronously acquired; According to the time-space coverage difference of the three types of heterogeneous sensing data, a data compensation rule is established, when image data is missing due to insufficient light of the optical camera unit, distance data of the millimeter wave detection unit and thermal feature distribution data of the infrared sensing unit are used for spatial interpolation compensation through the data compensation rule; Based on the compensated sensing data, dynamic characteristic analysis is performed on a linear obstacle existing in a flight path, extension direction, spatial curvature and continuous distribution density parameters of the linear obstacle in the analysis result are extracted, and an obstacle topology graph in a three-dimensional space coordinate system with the current position of the UAV as the origin is generated; According to the real-time dynamic relationship between the extension direction in the obstacle topology graph and the current flight heading of the UAV, the extension direction and the heading angle change amount are converted into an avoidance direction adjustment weight, the spatial curvature and the continuous distribution density parameters are combined, and a continuous deflection instruction of an obstacle avoidance heading is generated; Based on the continuous deflection instruction, a multi-section obstacle avoidance trajectory is generated, the distance data and the thermal feature distribution data are fused when the UAV executes the multi-section obstacle avoidance trajectory, the continuous deflection instruction is closed-loop corrected through the fused data, and the correction process is repeated until the relative distance between the UAV and the linear obstacle reaches a safety threshold.

2. The method of claim 1, wherein, The method for realizing dynamic obstacle avoidance of the UAV in flight comprises the following steps: The compensated sensing data is divided into a plurality of independent space units, target units corresponding to the linear obstacle are selected according to the geometric morphological characteristics of the space units in the three-dimensional space; The target units are subjected to main direction analysis, the extension direction of the target units in the three-dimensional space is determined, the extension direction is continuously fitted according to the spatial adjacent relationship, and the overall extension direction of the linear obstacle is generated; According to the extension direction angle deviation value between adjacent target units, a bending degree parameter of the linear obstacle in the three-dimensional space is calculated as the spatial curvature, wherein the extension direction angle deviation value is determined by the main direction included angle of the adjacent target units; The number distribution of the target units in a unit volume in the three-dimensional space coordinate system with the current position of the UAV as the origin is counted, the continuous distribution density parameter of the linear obstacle is generated according to the number distribution, and the overall extension direction, the spatial curvature and the continuous distribution density parameter are mapped into the three-dimensional space coordinate system to form the obstacle topology graph.

3. The method of claim 1, wherein, The method for realizing dynamic obstacle avoidance of the UAV in flight comprises the following steps: Real-time acquisition of the real-time angle value between the extending direction in the obstacle topology map and the current flight direction of the unmanned aerial vehicle, and calculation of the difference value between the real-time angle value and the angle value at the previous time as the angle change value according to the extending direction; According to the spatial curvature and the continuous distribution density parameter, a curvature influence coefficient and a density influence coefficient are respectively generated, wherein the curvature influence coefficient is positively correlated with the spatial curvature, and the density influence coefficient is negatively correlated with the continuous distribution density parameter; The angle change value, the curvature influence coefficient and the density influence coefficient are input into a preset dynamic relationship evaluation parameter table, and the avoidance direction adjustment weight is determined by table lookup; According to the avoidance direction adjustment weight, an avoidance direction component orthogonal to the extending direction is generated in the vertical plane of the current flight direction of the unmanned aerial vehicle, and the avoidance direction component and the current direction are superimposed to generate a continuous deflection instruction.

4. The method of claim 1, wherein, The continuous deflection instruction is generated based on the continuous deflection instruction, and the distance data and the thermal feature distribution data are fused when the unmanned aerial vehicle executes the multi-segment obstacle avoidance trajectory. The continuous deflection instruction is corrected in a closed loop through the fused data, and the correction process is repeated until the relative distance between the unmanned aerial vehicle and the linear obstacle reaches a safety threshold, comprising: The continuous deflection instruction is decomposed into a trajectory segment parameter set, which includes a heading angle change value and a corresponding flight speed value; When the unmanned aerial vehicle flies according to the trajectory segment parameter set, the distance data and the thermal feature distribution data are acquired synchronously, and the fusion credibility of the trajectory segment parameter set is determined according to the overlap ratio of the coverage range of the distance data and the intensity region of the thermal feature distribution data; According to the fusion credibility, the heading angle change value and the corresponding flight speed value are evaluated, wherein when the coverage range is lower than the preset proportion during the evaluation process, the center point of the intensity region of the thermal feature distribution data is preferentially used as the deviation evaluation reference; Based on the deviation evaluation result, the heading angle change value and the flight speed value in the next trajectory segment parameter set are adjusted in the opposite direction. After adjustment, the generation process of the trajectory segment parameter set is repeated until the minimum relative distance between the unmanned aerial vehicle and the linear obstacle in the distance data reaches the safety threshold.

5. The method of claim 1, wherein, The data compensation rule is established according to the spatio-temporal coverage difference of the three types of heterogeneous sensing data. When the image data is missing due to insufficient illumination of the optical camera unit, the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit are used for spatial interpolation compensation through the data compensation rule, comprising: Detecting the pixel gray value of the target region in the image data output by the optical camera unit. When the pixel gray value is lower than the preset illumination threshold and the area of the continuous pixel missing region of the target region exceeds the set proportion, the boundary of the target region is marked as a missing region boundary; According to the distance measurement point density distribution of the millimeter wave detection unit in the missing region boundary and the thermal feature intensity distribution of the infrared sensing unit in the missing region, a joint coverage capability parameter is calculated; based on the joint coverage capability parameter, the distance measurement points of the millimeter wave detection unit are spatially weighted with the thermal feature intensity distribution points of the infrared sensing unit; according to the allocation result, an interpolation surface is constructed within the missing area boundary, with the distance measurement points as geometric constraints and the thermal feature intensity distribution points as attribute constraints, and the interpolation surface is mapped into the image data within the missing area boundary to complete the spatial interpolation compensation.

6. The method of claim 3, wherein, The real-time angle value between the extension direction in the obstacle topology map and the current flight direction of the unmanned aerial vehicle is obtained, and the difference value between the real-time angle value and the angle value at the previous time is calculated as the angle change value based on the extension direction, comprising: extracting the extension direction vector of the linear obstacle from the obstacle topology map, and synchronously obtaining the current flight direction vector output by the unmanned aerial vehicle flight control unit; calculating the plane projection angle between the extension direction vector and the flight direction vector in three-dimensional space as the real-time angle value; performing difference operation on the real-time angle value and the historical angle value stored at the previous time to obtain the angle change value at the adjacent time; when the real-time angle value is obtained for the first time, the angle change value is initialized to zero, and the historical angle value is continuously updated to the current real-time angle value at subsequent times.

7. The method of claim 2, wherein, The main direction of the target unit is analyzed to determine the extension direction of the target unit in three-dimensional space, and the extension direction is continuously fitted according to the spatial adjacent relationship to generate the overall extension direction of the linear obstacle, comprising: performing three-dimensional spatial distribution analysis on the geometric features of the target unit, and determining the longest extension axis direction of the target unit in three-dimensional space as the main direction according to the analysis result; establishing a directional association relationship between adjacent target units according to the main direction, and marking the adjacent target units with the directional association relationship as directional association units; adjusting the main direction of the directional association units for consistency, and gradually decreasing the main direction angle of the directional association units to within a preset allowable deviation range in spatial adjacent order through the adjustment process; connecting the adjusted main direction of the directional association units in spatial adjacent order to form the continuous extension direction of the linear obstacle as the overall extension direction.

8. An unmanned aerial vehicle autonomous obstacle avoidance system based on multi-sensor fusion, characterized in that, comprising: an acquisition module, configured to synchronously acquire three types of heterogeneous sensing data from an optical camera unit, a millimeter wave detection unit and an infrared sensing unit during unmanned aerial vehicle flight; a compensation module, configured to establish a data compensation rule according to the spatio-temporal coverage difference of the three types of heterogeneous sensing data, and when the optical camera unit causes image data to be missing due to insufficient light, the data compensation rule is used to perform spatial interpolation compensation on the distance data of the millimeter wave detection unit and the thermal feature distribution data of the infrared sensing unit; an analysis module, configured to analyze the dynamic features of the linear obstacle existing in the flight path based on the compensated sensing data, extract the extension direction, spatial curvature and continuous distribution density parameters of the linear obstacle in the analysis result, and generate an obstacle topology map in a three-dimensional coordinate system with the current position of the unmanned aerial vehicle as the origin; The conversion module is configured to convert the extension direction and heading angle change amount into an avoidance direction adjustment weight according to a real-time dynamic relationship between the extension direction and a current flight heading of the unmanned aerial vehicle in the obstacle topology map, and to generate a continuous deflection instruction of an obstacle avoidance heading in combination with the spatial curvature and the continuous distribution density parameter; The correction module is configured to generate a multi-segment obstacle avoidance trajectory based on the continuous deflection instruction, to fuse the distance data and the thermal feature distribution data when the unmanned aerial vehicle executes the multi-segment obstacle avoidance trajectory, and to perform a closed-loop correction on the continuous deflection instruction through the fused data, and to repeat the correction process until a relative distance between the unmanned aerial vehicle and the linear obstacle reaches a safety threshold.

9. A computing device, comprising: The storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the method for autonomous obstacle avoidance of the unmanned aerial vehicle based on multi-sensor fusion according to any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer program is stored in the computer and is executed by the computer to implement the method for autonomous obstacle avoidance of the unmanned aerial vehicle based on multi-sensor fusion according to any one of claims 1-7.

Images