Multi-sensor fusion obstacle avoidance path planning method for urban inspection low-altitude aircraft

Abstract

The application discloses a kind of multi-sensor fusion obstacle avoidance route planning methods for city inspection low-altitude aircraft, comprising: obtaining the multidirectional perception data, flight pose data and preset flight plan collected by multiple perception units;According to the pose after time alignment, multidirectional perception data is converted to navigation coordinate system;Range screening and voxel down-sampling are carried out to point cloud, obstacle point set is formed by clustering and noise filtering, and contour fitting is carried out to regular obstacle, and irregular obstacle is retained grid expression;Based on initial route, local reference track is generated and local planning coordinate system is established, candidate local track is generated by sampling in horizontal plane and executability detection and safety envelope collision detection are carried out;When there is no feasible track in horizontal plane, vertical obstacle avoidance is triggered, and finally flight control executable track point and heading angle command are output.The application can improve the autonomous obstacle avoidance capability and route planning stability of low-altitude aircraft in complex urban environment.

Description

Technical Field

[0001] This invention belongs to the field of autonomous perception and route planning technology for low-altitude aircraft, specifically involving a multi-sensor fusion obstacle avoidance route planning method for low-altitude aircraft used for urban inspection. Background Technology

[0002] In inspection scenarios involving urban roads, bridges, building facades, power lines, park roads, and municipal facilities, low-altitude aircraft typically need to perform inspection tasks along a pre-set flight plan or initial route. Since the initial route is often generated based on the connection between the mission area and waypoints, it may not fully reflect real-time environmental changes such as local buildings, trees, power lines, light poles, billboards, temporary construction facilities, and narrow passages between buildings. Therefore, local obstacle avoidance route planning is still necessary during low-altitude close-range flight.

[0003] Existing obstacle avoidance systems for low-altitude aircraft typically use a single lidar, visual sensor, or range sensor to acquire obstacle information and perform deceleration, hovering, or simple obstacle avoidance based on the obstacle distance. This approach can reduce some collision risks, but when faced with distant small targets, lateral close-range walls, blind spots below, irregular obstacles, and temporary obstacles during urban patrols, the results from a single sensing device are insufficient to simultaneously consider detection range, close-range depth information, installation weight, power consumption, and cost.

[0004] On the other hand, low-altitude aircraft need to combine perception results with flight attitude data output by the flight control unit, preset flight plans or initial route information, and local reference tracks for local route planning. If only obstacle warnings are issued without unified processing of obstacle point sets, obstacle outlines, local planning coordinate systems, candidate track evaluation, and safety envelope collision detection, it will be difficult to generate stable, executable, and safety-compliant local tracks in urban patrol environments.

[0005] Therefore, a multi-sensor fusion obstacle avoidance route planning method for low-altitude aircraft used for urban inspection is needed. This method should integrate long-range detection, blind spot filling below, lateral near-range depth perception, flight attitude alignment, obstacle contour determination, candidate trajectory generation guided by local reference trajectories, horizontal obstacle avoidance, and vertical obstacle avoidance to improve the autonomous obstacle avoidance capability and route planning reliability of low-altitude aircraft. Summary of the Invention

[0006] The purpose of this invention is to provide a multi-sensor fusion obstacle avoidance route planning method for low-altitude aircraft used in urban inspections. This method addresses the problem of insufficient handling of distant small targets, lateral and close obstacles, blind spots below, irregular obstacles, and local track feasibility in urban inspection environments. It enables low-altitude aircraft to perform stable local route planning and obstacle avoidance control based on the results of multi-sensor fusion perception.

[0007] To achieve the above objectives, the present invention employs the following technical solution: A multi-sensor fusion obstacle avoidance route planning method for low-altitude aircraft used for urban patrol includes: Step 1: Acquire multi-directional perception data collected by multiple sensing units on the low-altitude aircraft, acquire flight attitude data output by the flight control unit to the airborne computing platform, and acquire the preset flight plan; Step 2: Based on the acquisition time of the multi-directional sensing data, determine the corresponding low-altitude aircraft attitude in the flight attitude data, and based on the extrinsic parameters of each sensing unit relative to the low-altitude aircraft body, convert the multi-directional sensing data to a navigation coordinate system consistent with the flight attitude data to obtain the converted point cloud. Step 3: Filter the converted point cloud according to the preset horizontal detection range, vertical detection range and vertical detection range, and perform voxel grid-based downsampling processing on the filtered point cloud to obtain the obstacle point set composed of representative points after downsampling. Step 4: Based on the obstacle point set, perform obstacle clustering and noise cluster filtering to obtain obstacle clusters; according to the spatial distribution of each representative point in the obstacle cluster, determine the specific information of regular obstacles, and represent obstacles with irregular shapes or many internal gaps as obstacle grids composed of multiple occupied voxel units. Step 5: Generate a local reference track based on the initial route information in the preset flight plan, and establish a local planning coordinate system using the local reference track as the longitudinal reference line to describe the progress coordinates along the direction of the local reference track and the lateral offset distance relative to the local reference track. Step 6: In the local planning coordinate system, multiple sets of candidate local tracks are obtained based on sampling, and the evaluation cost of the candidate local tracks is constructed according to the trajectory change magnitude, deviation from the local reference track and target speed deviation. Step 7: Perform feasibility and collision detection on candidate local trajectories based on the kinematic constraints of the low-altitude aircraft, the dimensions of the low-altitude aircraft, and the obstacle point set, obstacle circumscribed contour, or obstacle grid. Step 8: When there are candidate local tracks that satisfy both executability and collision detection, select the candidate local track with the lowest evaluation cost as the output local track; otherwise, maintain the horizontal projection of the current local reference track and generate candidate vertical local tracks within the preset height adjustment range. Step 9: Generate executable waypoints and heading angle commands based on the output local track, and send the waypoints and heading angle commands to the flight control unit.

[0008] Furthermore, the sensing unit includes a long-range lidar, a blind spot lidar, and a binocular camera. The sensing unit is communicatively connected to the airborne computing platform. The long-range lidar is installed at the front or top of the low-altitude aircraft. The blind spot lidar can be installed under the fuselage or at an angle downwards. The binocular camera is arranged on the side of the low-altitude aircraft. Each frame of sensing data collected by the sensing unit carries a collection timestamp. The flight attitude data is output by the flight control unit to the airborne computing platform and includes position, attitude, velocity, and heading information. When a frame of sensing data arrives, the airborne computing platform searches for attitude data in the flight attitude data cache with a time difference less than a first preset time threshold. If the attitude data of two adjacent frames are available, the position, velocity, and attitude are interpolated according to the collection timestamp to obtain the flight attitude data corresponding to the frame of sensing data.

[0009] Furthermore, based on the external parameters of each sensing unit relative to the low-altitude aircraft body, the sensing data collected by each sensing unit is first converted to the aircraft coordinate system, and then combined with the flight attitude data at the corresponding moment to be converted to the navigation coordinate system consistent with the flight attitude data output by the flight control unit. Determining the attitude of the corresponding low-altitude aircraft includes: Select the flight pose data from all cached flight pose data where the time difference with the multi-directional sensing data acquisition time is less than the second preset time threshold as the low-altitude aircraft pose; or, interpolate the corresponding low-altitude aircraft pose data based on two adjacent frames of flight pose data before and after the multi-directional sensing data acquisition time. Furthermore, for point cloud points that exceed any detection range, the airborne computing platform removes them from the obstacle detection input for the current planning cycle; The three-dimensional space is divided into multiple voxel units, and a representative point is generated in each occupied voxel unit. The representative point is the centroid of each point in the voxel unit, the point closest to the voxel center, or the point with the highest confidence, and carries the number of points in the voxel unit, source identifier, or confidence information. For the lateral near-field area or the blind spot area below, as well as the perception area when the low-altitude aircraft is in a low-speed close-range inspection state, a voxel unit side length smaller than that of the far-field forward perception area is used. When points from different perception units fall into the same voxel unit, the representative point is determined according to the distance, confidence, or preset perception unit priority.

[0010] Furthermore, based on the spatial proximity between occupied voxel units, adjacent or continuously distributed occupied voxel units are grouped into the same obstacle cluster; For an obstacle cluster, if multiple representative points are distributed close to walls, columns, boxes, building facades, or linear structures, the specific information of the obstacles is determined based on the obstacle cluster, including the center position, circumscribed profile, size, and orientation angle; and the corresponding obstacles are represented by circumscribed profile, circumscribed box, or rotated circumscribed profile. For obstacles with irregular shapes or many internal gaps, multiple occupied voxel units belonging to the same obstacle cluster are retained as obstacle grids, and the actual occupied area of ​​the corresponding obstacle is represented by multiple obstacle grids. Set timestamps and validity periods for obstacle point sets, obstacle clusters, and obstacle grids; in a new planning period, if obstacles are continuously detected near the same spatial location, update the center position, outline, or obstacle grid of the obstacle.

[0011] Furthermore, the airborne computing platform selects a route segment within a preset distance ahead of the low-altitude aircraft's current position in the initial route, and performs smoothing processing based on the route segment endpoints, turning points, or route control points to obtain a local reference track used in the current planning cycle; the local planning coordinate system is the Frenet coordinate system established with the local reference track as the guide line, and the longitudinal reference line is the guide line; the position, velocity, and heading of the low-altitude aircraft in the navigation coordinate system are transformed to the Frenet coordinate system for local track planning; The airborne computing platform projects the position of the low-altitude aircraft in the navigation coordinate system onto the local reference track to obtain the progress coordinates along the direction of the local reference track, and calculates the vertical distance between the projection point and the aircraft position as the lateral offset distance relative to the local reference track; at the same time, it decomposes the velocity of the low-altitude aircraft into velocity components along the direction of the local reference track and lateral velocity components.

[0012] Furthermore, the airborne computing platform generates multiple lateral candidate target states based on the current lateral offset distance relative to the local reference track, the current lateral velocity, and a preset lateral offset distance sampling range; and generates multiple longitudinal candidate target states based on the current velocity component along the local reference track direction, the target velocity, and a preset velocity sampling range. Based on the sampling results of the lateral offset distance relative to the local reference track, the lateral candidate trajectory is generated using a fifth-order polynomial; based on the sampling results of the target velocity along the direction of the local reference track, the longitudinal candidate trajectory is generated using a fourth-order polynomial; the airborne computing platform combines different sampling times, lateral candidate trajectories and longitudinal candidate trajectories to obtain multiple candidate local tracks. For each candidate local trajectory, the airborne computing platform calculates the evaluation cost; the evaluation cost includes the trajectory change magnitude cost, the deviation from the local reference trajectory cost, the target speed deviation cost, and the stability cost relative to the output trajectory of the previous cycle.

[0013] Furthermore, the airborne computing platform first performs an executability test on the candidate local tracks: it determines whether the speed corresponding to each track point on the candidate local track exceeds the maximum speed, whether the acceleration exceeds the maximum acceleration, whether the heading change formed by adjacent track points exceeds the maximum heading change threshold, and whether the altitude of the track point is within the altitude range allowed by the inspection mission; candidate local tracks that fail the executability test will no longer enter the collision detection process. For candidate local tracks that pass the executability test, the airborne computing platform performs safety envelope collision detection. Specifically, for multiple track points on each candidate local track, the airborne computing platform determines the heading angle of the track point based on the positional relationship of adjacent track points, and generates a safety envelope based on the position of the track point, the heading angle, the length, width, and height of the low-altitude aircraft, and a preset safety extension distance. The safety envelope is a three-dimensional bounding box that moves with the track point or other spatial range that can cover the shape of the aircraft and the safety boundary. The airborne computing platform can also generate a swept safety envelope based on the direction of the line connecting adjacent track points. The swept safety envelope covers the spatial range traversed by the safety boundary of the aircraft during the movement of the low-altitude aircraft from the previous track point to the next track point.

[0014] Furthermore, the airborne computing platform determines whether the obstacle points, the circumscribed contours of obstacles, or the obstacle grids in the obstacle point set fall within or intersect with the safety envelope; if any candidate local track has an obstacle intruding into the safety envelope at any track point, the candidate local track is determined to have a collision risk and is eliminated; for the circumscribed contours of obstacles, the airborne computing platform determines whether the vertices, boundary segments, or their projected areas of the circumscribed contours intersect with the safety envelope; for irregular obstacles expressed by multiple obstacle grids, it determines whether the center point, grid boundary, or grid-occupied area of ​​the obstacle grid intersects with the safety envelope.

[0015] Furthermore, the airborne computing platform prioritizes obstacle avoidance planning in the horizontal plane; it avoids obstacles by lateral maneuvering in the horizontal plane, and then triggers vertical obstacle avoidance planning that includes altitude changes when all candidate local paths in the horizontal plane are impassable. When vertical obstacle avoidance planning is triggered, the airborne computing platform generates multiple altitude correction candidate values ​​according to a preset altitude step size while maintaining the progress coordinates along the local reference track direction. These values ​​are then recombined to form candidate local tracks that include altitude changes. For candidate local tracks that include altitude changes, the airborne computing platform also performs executability checks, safety envelope collision detection, and evaluation cost calculations. It prioritizes the candidate local track that meets vertical safety requirements, has the lowest cost, and exhibits a relatively gentle altitude change as the output local track. When there are no candidate local tracks in the horizontal plane that meet the requirements for executability detection and collision detection, multiple candidate vertical local tracks are generated within a preset altitude adjustment range based on the horizontal projection of the current local reference track or the initial route. The candidate vertical local tracks are then selected based on the climb rate, descent rate, altitude limit, and safety envelope collision detection results of the low-altitude aircraft. The candidate vertical local track that meets the conditions and has the lowest evaluation cost is then used as the output local track.

[0016] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, it implements the multi-sensor fusion obstacle avoidance route planning method for urban inspection low-altitude aircraft.

[0017] A computer-readable storage medium storing a computer program; when executed by a processor, the computer program implements the multi-sensor fusion obstacle avoidance route planning method for urban inspection low-altitude aircraft.

[0018] Compared with the prior art, the present invention has the following technical features: 1. This invention integrates multi-directional perception data from long-range lidar, blind spot lidar, and binocular camera with flight attitude data for unified processing, enabling it to simultaneously detect small targets at long distances, fill blind spots below, and perform lateral near-range depth perception.

[0019] 2. This invention improves the adaptability of obstacle detection results to irregular obstacles and noise interference by filtering the detection range, voxel grid-based point cloud downsampling, obstacle clustering, regular obstacle contour determination, and irregular obstacle grid representation.

[0020] 3. This invention generates a local reference track and establishes a local planning coordinate system based on a preset flight plan or initial route, prioritizes obstacle avoidance planning in the horizontal plane, and triggers vertical obstacle avoidance planning when there is no passable track in the horizontal plane, which can improve the stability of route planning for low-altitude aircraft flying near urban roads, building facades and inter-building passages. Attached Figure Description

[0021] Figure 1 This is a flowchart of the multi-sensor fusion obstacle avoidance route planning method provided in an embodiment of the present invention; Figure 2 This is a flowchart of obstacle detection and contour determination provided in an embodiment of the present invention; Figure 3 This is a flowchart of a local obstacle avoidance planning process guided by a reference trajectory, provided for an embodiment of the present invention. Detailed Implementation

[0022] like Figure 1As shown, this embodiment provides a multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection. This method runs on the onboard computing platform of the low-altitude aerial vehicle and cyclically executes sensing data reception, pose alignment, obstacle detection, candidate trajectory generation, collision detection, and flight control command output at a preset cycle. The low-altitude aerial vehicle can be a multi-rotor UAV, a fixed-wing UAV, a compound-wing aircraft, a tiltrotor aircraft, or an electric vertical takeoff and landing aircraft, etc. The method specifically includes the following steps: Step 1: Acquire multi-directional perception data collected by the long-range lidar, blind spot lidar and binocular camera on the low-altitude aircraft; acquire flight attitude data output by the flight control unit to the airborne computing platform; and acquire the preset flight plan.

[0023] In step 1, the sensing unit mounted on the low-altitude aircraft includes a long-range lidar, a blind-spot lidar, and a binocular camera. The sensing unit is communicatively connected to the airborne computing platform. The long-range lidar can be installed at the front or top of the low-altitude aircraft to acquire sensing data (point cloud) of obstacles such as building edges, tree branches, low-altitude power lines, and light poles at a relatively long distance in front and around the aircraft. The blind-spot lidar can be installed under the aircraft or at an angle downwards to acquire sensing data (point cloud) of the take-off and landing area, protrusions below, low guardrails, and near-field blind spots below. The binocular camera is positioned on the side of the low-altitude aircraft to acquire sensing data (near-field depth point cloud) of lateral walls, building facades, tree branches, or protrusions. The sensing data collected by the long-range lidar, the blind-spot lidar, and the binocular camera together constitute multi-directional sensing data.

[0024] The use of binocular cameras as a source of lateral near-field perception is intended to supplement lateral near-field depth information within the constraints of limited payload, power consumption, and installation space for low-altitude aerial vehicles used in urban patrols. Compared to adding lidar on both sides, binocular cameras are more suitable for small low-altitude aerial vehicles in terms of cost, weight, and power consumption. They can also provide lateral depth data for determining the safety envelope of building facades, wall edges, and nearby protrusions. The depth map output by the binocular cameras can be converted into a point cloud in the camera coordinate system based on camera intrinsic parameters, and invalid depth points can be removed by using a confidence threshold or the effective depth range.

[0025] The airborne computing platform establishes cache queues for perception data and flight pose data from long-range lidar point clouds, blind-spot lidar point clouds, and binocular camera point clouds. Each frame of perception data carries a collection timestamp. Flight pose data is output from the flight control unit to the airborne computing platform and includes position, attitude, velocity, and heading information. When a frame of perception data arrives, the airborne computing platform searches for pose data in the flight pose data cache with a time difference less than a first preset time threshold. If two adjacent frames of pose data are available (i.e., both are less than the first preset time threshold), the position, velocity, and attitude are interpolated based on the collection timestamp to obtain the flight pose data corresponding to that frame of perception data. When the time difference between the matched flight pose data and the acquisition time of the perception data is greater than the first preset time threshold, or when the change between the interpolated flight pose data and the flight pose data of the previous planning cycle exceeds a preset mutation threshold, the airborne computing platform can mark that frame of perception data as invalid perception data (to be confirmed later) and exclude it from obstacle fusion and candidate trajectory collision detection in the current planning cycle. This process can reduce the impact of timestamp anomalies, communication delays, or flight control state jumps on the stability of local route planning.

[0026] Step 2: Based on the acquisition time of the multi-directional sensing data, determine the corresponding low-altitude aircraft attitude in the flight attitude data, and based on the extrinsic parameters of each sensing unit relative to the low-altitude aircraft body, convert the multi-directional sensing data to a navigation coordinate system consistent with the flight attitude data to obtain the converted point cloud.

[0027] In step 2, the airborne computing platform, based on the extrinsic parameters of each sensing unit relative to the low-altitude aircraft body, first transforms the sensing data from the long-range lidar, the blind-spot lidar, and the binocular camera from their respective sensor coordinate systems to the aircraft coordinate system. Then, combining the flight attitude data at the corresponding moment, it transforms the sensing data in the aircraft coordinate system to a navigation coordinate system consistent with the flight attitude data output by the flight control unit. The navigation coordinate system can be the local navigation coordinate system adopted by the flight control unit, used to ensure that the obstacle position, the low-altitude aircraft position, and the subsequent local trajectory are under the same coordinate reference. In one implementation, the coordinate transformation is completed by cascading the sensor extrinsic parameter transformation matrix, the aircraft attitude rotation matrix, and the translation vector. Each point cloud point after transformation can retain the sensor source identifier, acquisition time, and confidence information, enabling subsequent obstacle clustering, contour determination, and safe envelope collision detection to distinguish the obstacle information provided by the long-range lidar, the blind-spot lidar, and the binocular camera.

[0028] Determining the attitude of the corresponding low-altitude aircraft includes: The flight pose data with a time difference of less than a second preset time threshold from all cached flight pose data is selected as the low-altitude aircraft pose; or, the corresponding low-altitude aircraft pose is obtained by interpolation of two adjacent frames of flight pose data before and after the multi-directional sensing data acquisition time; the low-altitude aircraft pose includes position, attitude, velocity and heading information.

[0029] That is, the flight attitude data output by the flight control unit refers to the attitude sequence with timestamps that the flight control system continuously records and sends at a fixed frequency, rather than the instantaneous attitude at a single moment. Since the acquisition time of the perception data of the LiDAR, binocular camera and other perception units is not strictly synchronized with the recording time of the flight control attitude data on the time axis, if the latest attitude output by the flight control is directly used for coordinate transformation, the point cloud stitching deviation may be caused by time mismatch, which will affect the accuracy of subsequent obstacle detection and collision judgment. Therefore, a second preset time threshold is used for filtering to obtain the accurate attitude of the low-altitude aircraft in the navigation coordinate system at the moment of perception data acquisition.

[0030] Step 3: Filter the converted point cloud according to the preset horizontal detection range, vertical detection range and vertical detection range, and perform voxel grid-based downsampling processing on the filtered point cloud to obtain the obstacle point set composed of representative points after downsampling.

[0031] In step 3, the airborne computing platform sets the lateral detection range, longitudinal detection range, and vertical detection range according to the flight altitude of the urban inspection mission, the fuselage size of the low-altitude aircraft, the inspection speed, and the braking distance. The converted point cloud is then filtered based on the lateral, longitudinal, and vertical detection ranges. For point cloud points that exceed any detection range, the airborne computing platform removes them from the obstacle detection input of the current planning cycle.

[0032] For point cloud points located within the detection range and belonging to the preset close-range safety zone, forward planning zone, lateral close-range zone, or lower blind spot zone, the airborne computing platform retains them and participates in subsequent downsampling, clustering, and collision detection; for point cloud points within the detection range but not belonging to any of the above functional areas, they can be downgraded or removed according to distance or task requirements.

[0033] The airborne computing platform performs voxel-based downsampling on the filtered point cloud to obtain a set of obstacle points composed of representative downsampled points, as follows: The three-dimensional space is divided into multiple voxel units, and a representative point is generated within each occupied voxel unit to reduce the number of point clouds and computational load. All representative points constitute an obstacle point set. The representative point can be the centroid of each point within the voxel unit, the point closest to the voxel center, or the point with the highest confidence level, and can carry information about the number of points within that voxel unit, its source identifier, or its confidence level. The side length of the voxel unit is determined based on the point cloud density, the flight speed of the low-altitude aircraft, the computational capability of the planning cycle, the distance to the sensing area, and the preset minimum obstacle size. For the aforementioned lateral near-field areas or lower blind-spot areas, and the sensing area when the low-altitude aircraft is in a low-speed close-range inspection state, since the obstacles are close to the aircraft and the sensing accuracy requirement is higher, a voxel unit side length smaller than that of the far-field forward sensing area is used to preserve near-field details such as wall edges, tree branches, cables, and protrusions. When points from different sensing units fall into the same voxel unit, a representative point is determined based on distance, confidence level, or preset sensing unit priority to avoid duplicate obstacle points that could lead to clustering inflated results. At this point, the original point cloud, after being filtered and downsampled, is abstracted into a set of obstacle points (i.e., a set of representative points), with each representative point corresponding to an occupied voxel unit.

[0034] Step 4: Based on the obstacle point set, perform obstacle clustering and noise cluster filtering to obtain obstacle clusters; according to the spatial distribution of each representative point in the obstacle cluster, determine the specific information of regular obstacles, and represent obstacles with irregular shapes or many internal gaps as obstacle grids composed of multiple occupied voxel units.

[0035] In step 4, the airborne computing platform groups adjacent or continuously distributed occupied voxel units into the same obstacle cluster based on the spatial proximity relationship between occupied voxel units (representative points in the corresponding obstacle point set). Obstacle clusters with fewer than a preset point number threshold, smaller than a preset size threshold, and not appearing stably within a continuous planning period can be identified as sparse noise point clusters and filtered out. For slender obstacle clusters that appear stably within a continuous planning period, even if their point number or spatial size is small, they are retained as valid obstacle clusters.

[0036] For an obstacle cluster, if multiple representative points are distributed close to walls, columns, boxes, building facades, or linear structures, the airborne computing platform determines the specific information of the obstacles based on the obstacle cluster, including the center position, circumscribed contour, size, and orientation angle; and uses the circumscribed contour, circumscribed box, or rotated circumscribed contour to represent the corresponding obstacles in order to reduce the amount of data required for collision detection.

[0037] For obstacles with irregular shapes or numerous internal gaps (a gap ratio greater than a preset gap threshold), such as L-shaped building edges, tree canopies, scaffolding, and billboard supports, using a single circumscribed contour or circumscribed box to represent them can easily misjudge spaces not actually occupied by obstacles as impassable spaces. In this case, the airborne computing platform retains multiple occupied voxel units belonging to the same obstacle cluster as obstacle grids, and multiple obstacle grids together represent the actual occupied range of the corresponding obstacle.

[0038] The airborne computing platform can also set timestamps and validity periods for obstacle clusters and obstacle grids. In a new planning period, if obstacles are continuously detected near the same spatial location, the center position, circumscribed contour, or obstacle grid of that obstacle is updated; if no obstacle is detected for several consecutive periods, it is deleted from the current obstacle set. In this way, rollingly updated obstacle inputs can be provided for local trajectory planning without building a large-scale global map.

[0039] Step 5: Generate a local reference track based on the initial route information in the preset flight plan, and establish a local planning coordinate system using the local reference track as the longitudinal reference line to describe the progress coordinates along the direction of the local reference track and the lateral offset distance relative to the local reference track.

[0040] In step 5, the airborne computing platform generates a local reference track based on the initial route in the preset flight plan. Specifically, the airborne computing platform can select a route segment within a preset distance in front of the current position of the low-altitude aircraft in the initial route, and smooth it according to the endpoints, turning points or route control points of the route segment to obtain the local reference track used in the current planning cycle. The local reference track does not replace the original preset flight plan, but is a reference line used for local obstacle avoidance and re-approaching the original route near the original preset flight plan.

[0041] In one implementation, the local planning coordinate system is a Frenet coordinate system established with a local reference track as the guide line; in the Frenet coordinate system, the longitudinal reference line is the guide line; the position, velocity and heading of the low-altitude aircraft in the navigation coordinate system are converted to the Frenet coordinate system for local track planning.

[0042] The airborne computing platform projects the position of the low-altitude aircraft in the navigation coordinate system onto the local reference track to obtain the progress coordinates along the direction of the local reference track, and calculates the vertical distance between the projection point and the aircraft position as the lateral offset distance relative to the local reference track; at the same time, it decomposes the velocity of the low-altitude aircraft into velocity components along the direction of the local reference track and lateral velocity components.

[0043] Step 6: In the local planning coordinate system, multiple sets of candidate local tracks are obtained based on sampling, and the evaluation cost of the candidate local tracks is constructed according to the trajectory change magnitude, deviation from the local reference track and the target speed deviation.

[0044] In step 6, the airborne computing platform generates multiple lateral candidate target states based on the current lateral offset distance relative to the local reference track, the current lateral velocity, and a preset lateral offset distance sampling range; and generates multiple longitudinal candidate target states based on the current velocity component along the local reference track direction, the target velocity (the mission cruise speed given in the preset flight plan), and a preset velocity sampling range. The lateral candidate target states may include the target lateral offset distance, the target lateral velocity, and the target lateral acceleration; the longitudinal candidate target states may include the target velocity along the track direction and the target acceleration along the track direction.

[0045] Based on the sampling results of the lateral offset distance relative to the local reference track, the lateral candidate trajectory can be generated using a fifth-order polynomial, ensuring that the lateral position, lateral velocity, and lateral acceleration satisfy the boundary conditions at the start and end points. Based on the sampling results of the target velocity along the direction of the local reference track, the longitudinal candidate trajectory can be generated using a fourth-order polynomial, ensuring that the position, velocity, and acceleration along the track direction satisfy the current state at the start point and that the target velocity or target acceleration requirement is met at the end point. The airborne computing platform combines different sampling times, lateral candidate trajectories, and longitudinal candidate trajectories to obtain multiple candidate local tracks.

[0046] For each candidate local track, the airborne computing platform calculates the evaluation cost; the evaluation cost may include the cost of trajectory change magnitude, the cost of deviation from the local reference track (such as the cost of lateral offset), the cost of target speed deviation, and the cost of stability relative to the output track of the previous cycle.

[0047] Among them, the trajectory change amplitude cost is used to suppress sharp turns or sharp acceleration and deceleration, the deviation from the local reference track cost is used to prompt the low-altitude aircraft to return to the vicinity of the preset flight plan after obstacle avoidance, the target speed deviation cost is used to maintain inspection efficiency, and the stability cost is used to reduce track jumps between consecutive planning cycles.

[0048] Step 7: Perform feasibility and collision detection on candidate local trajectories based on the kinematic constraints of the low-altitude aircraft, the dimensions of the low-altitude aircraft, and the obstacle point set, obstacle circumscribed contour, or obstacle grid.

[0049] In step 7, the airborne computing platform first performs an executability test on the candidate local tracks. The executability test may include: determining whether the speed corresponding to each track point on the candidate local track exceeds the maximum speed, determining whether the acceleration exceeds the maximum acceleration, determining whether the heading change formed by adjacent track points exceeds the maximum heading change threshold, and determining whether the altitude of the track point is within the altitude range allowed by the inspection task. Candidate local tracks that fail the executability test will no longer enter the collision detection.

[0050] For candidate local tracks that pass the executability test, the airborne computing platform performs safety envelope collision detection. Specifically, for multiple trackpoints on each candidate local track, the airborne computing platform determines the heading angle of the trackpoint based on the positional relationship of adjacent trackpoints, and generates a safety envelope based on the position of the trackpoint, the heading angle, the length, width, and height of the low-altitude aircraft, and a preset safety extension distance. The safety envelope can be a three-dimensional bounding box that moves with the trackpoint or other spatial range that can cover the aircraft's shape and safety boundaries. To avoid missing collisions between adjacent trackpoints due to only detecting discrete trackpoints, the airborne computing platform can also generate a swept safety envelope based on the direction of the line connecting adjacent trackpoints. The swept safety envelope covers the spatial range traversed by the aircraft's safety boundaries during the low-altitude aircraft's movement from the previous trackpoint to the next trackpoint.

[0051] The airborne computing platform determines whether obstacle points (representative points), obstacle circumscribed contours, or obstacle grids within the obstacle point set fall within or intersect with the safety envelope. If any candidate local track has an obstacle intruding into the safety envelope at any track point, the candidate local track is deemed to have a collision risk and is eliminated; if no obstacle intrusion occurs at any of the detected track points, the candidate local track passes the collision detection. For obstacle circumscribed contours, the airborne computing platform can determine whether the vertices, boundary segments, or their projected areas intersect with the safety envelope; for irregular obstacles represented by multiple obstacle grids, it can determine whether the center point, grid boundary, or grid-occupied area intersects with the safety envelope. Therefore, irregular corners, tree canopies, billboard supports, or local protruding structures near power lines can be included in collision detection.

[0052] Step 8: When there are candidate local tracks that meet the requirements of executability detection and collision detection, select the candidate local track with the lowest evaluation cost as the output local track; otherwise, maintain the horizontal projection of the current local reference track and generate candidate vertical local tracks within the preset height adjustment range.

[0053] In step 8, the airborne computing platform prioritizes obstacle avoidance planning in the horizontal plane. Urban patrol flights typically require maintaining relatively stable flight altitude, patrol viewpoint, and communication links; frequent altitude changes increase energy consumption and reduce flight stability. Simultaneously, the safe space above or below in the urban low-altitude environment may be limited by buildings, bridges, power lines, and ground infrastructure. Therefore, this embodiment prioritizes obstacle avoidance through horizontal lateral maneuvering. Only when all candidate local paths in the horizontal plane are impassable does it trigger vertical obstacle avoidance planning that includes altitude changes.

[0054] When vertical obstacle avoidance planning is triggered, the airborne computing platform can generate multiple candidate height correction values ​​according to a preset height step size while maintaining the progress coordinates along the local reference track direction, and recombine them to form a candidate local track containing height changes. For the candidate local track containing height changes, the airborne computing platform also performs executability testing, safety envelope collision detection and evaluation cost calculation, and prioritizes the candidate local track that meets the vertical safety requirements, has the lowest cost and has a relatively gentle height change (the height change is within the preset height range) as the output local track.

[0055] When there are no candidate local tracks in the horizontal plane that meet the requirements for executability detection and collision detection, multiple candidate vertical local tracks are generated within a preset altitude adjustment range based on the horizontal projection of the current local reference track or the initial route. The candidate vertical local tracks are then selected based on the climb rate, descent rate, altitude limit, and safety envelope collision detection results of the low-altitude aircraft. The candidate vertical local track that meets the conditions and has the lowest evaluation cost is then used as the output local track.

[0056] When a low-altitude aircraft bypasses an obstacle and reapproaches the local reference track, the onboard computing platform converges the lateral offset distance sampling range for the next planning cycle towards the vicinity of the local reference track. This allows the low-altitude aircraft to gradually return to the vicinity of the preset flight plan, rather than flying along a deviated route for an extended period. This process can balance the route tracking requirements of urban patrol missions with the local obstacle avoidance safety requirements.

[0057] Step 9: Generate executable waypoints and heading angle commands based on the output local track, and send the waypoints and heading angle commands to the flight control unit.

[0058] In step 9, the airborne computing platform generates executable flight control track points and heading angle commands based on the finally selected output local track. The track points can be resampled according to the time interval or spatial interval required by the flight control unit, and include at least the position, altitude, and heading angle in the navigation coordinate system. The heading angle can be calculated based on the positional relationship of adjacent track points, or it can be determined in combination with the tangential direction of the output local track. When generating executable flight control commands, the airborne computing platform can also resample the output local track according to the control cycle supported by the flight control unit, and perform continuous processing on the heading angle changes of adjacent track points. When the distance, altitude, or heading angle changes between adjacent track points exceed preset limits, the airborne computing platform interpolates, deletes, or smooths the corresponding track points to ensure that the sequence of track points sent to the flight control unit meets the requirements of the flight control execution interface and the low-altitude aircraft's motion capability.

[0059] After the airborne computing platform sends the waypoint and heading angle commands to the flight control unit, it receives multi-directional perception data and flight attitude data again in the next planning cycle and repeats the above steps. When the preset flight plan or initial route information changes, the airborne computing platform regenerates the local reference track according to the new preset flight plan or initial route information, and performs local track planning for the next mission based on the regenerated local reference track at the end of the current mission.

[0060] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-sensor fusion obstacle avoidance route planning method for low-altitude aircraft used for urban patrol, characterized in that, include: Step 1: Acquire multi-directional perception data collected by multiple sensing units on the low-altitude aircraft, acquire flight attitude data output by the flight control unit to the airborne computing platform, and acquire the preset flight plan; Step 2: Based on the acquisition time of the multi-directional sensing data, determine the corresponding low-altitude aircraft attitude in the flight attitude data, and based on the extrinsic parameters of each sensing unit relative to the low-altitude aircraft body, convert the multi-directional sensing data to a navigation coordinate system consistent with the flight attitude data to obtain the converted point cloud. Step 3: Filter the converted point cloud according to the preset horizontal detection range, vertical detection range and vertical detection range, and perform voxel grid-based downsampling processing on the filtered point cloud to obtain the obstacle point set composed of representative points after downsampling. Step 4: Based on the set of obstacle points, perform obstacle clustering and noise cluster filtering to obtain obstacle clusters; Based on the spatial distribution of representative points in the obstacle cluster, the specific information of regular obstacles is determined, and obstacles with irregular shapes or many internal gaps are represented as obstacle grids composed of multiple occupied voxel units. Step 5: Generate a local reference track based on the initial route information in the preset flight plan, and establish a local planning coordinate system using the local reference track as the longitudinal reference line to describe the progress coordinates along the direction of the local reference track and the lateral offset distance relative to the local reference track. Step 6: In the local planning coordinate system, multiple sets of candidate local tracks are obtained based on sampling, and the evaluation cost of the candidate local tracks is constructed according to the trajectory change magnitude, deviation from the local reference track and target speed deviation. Step 7: Perform feasibility and collision detection on candidate local trajectories based on the kinematic constraints of the low-altitude aircraft, the dimensions of the low-altitude aircraft, and the obstacle point set, obstacle circumscribed contour, or obstacle grid. Step 8: When there are candidate local tracks that satisfy both executability and collision detection, select the candidate local track with the lowest evaluation cost as the output local track; otherwise, maintain the horizontal projection of the current local reference track and generate candidate vertical local tracks within the preset height adjustment range. Step 9: Generate executable waypoints and heading angle commands based on the output local track, and send the waypoints and heading angle commands to the flight control unit.

2. The multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection as described in claim 1, characterized in that, The sensing unit includes a long-range lidar, a blind spot lidar, and a binocular camera. The sensing unit is communicatively connected to the airborne computing platform. The long-range lidar is installed at the front or top of the low-altitude aircraft. The blind spot lidar can be installed under the fuselage or at an angle downwards. The binocular camera is arranged on the side of the low-altitude aircraft. Each frame of sensing data collected by the sensing unit carries a collection timestamp. The flight attitude data is output by the flight control unit to the airborne computing platform and includes position, attitude, velocity, and heading information. When a frame of sensing data arrives, the airborne computing platform searches for attitude data in the flight attitude data cache with a time difference less than a first preset time threshold. If the attitude data of two adjacent frames are available, the position, velocity, and attitude are interpolated according to the collection timestamp to obtain the flight attitude data corresponding to the frame of sensing data.

3. The multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection as described in claim 1, characterized in that, Based on the external parameters of each sensing unit relative to the low-altitude aircraft body, the sensing data collected by each sensing unit is first converted to the body coordinate system, and then combined with the flight attitude data at the corresponding moment to be converted to the navigation coordinate system consistent with the flight attitude data output by the flight control unit. Determining the attitude of the corresponding low-altitude aircraft includes: Among all cached flight pose data, flight pose data with a time difference of less than the time of multi-directional perception data acquisition is selected as the low-altitude aircraft pose. Alternatively, the corresponding low-altitude aircraft attitude can be obtained by interpolating the flight attitude data of two adjacent frames before and after the multi-directional sensing data acquisition time.

4. The multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection as described in claim 1, characterized in that, For point cloud points that are outside any detection range, the airborne computing platform removes them from the obstacle detection input for the current planning cycle. The three-dimensional space is divided into multiple voxel units, and a representative point is generated in each occupied voxel unit. The representative point is the centroid of each point in the voxel unit, the point closest to the voxel center, or the point with the highest confidence, and carries the number of points in the voxel unit, source identifier, or confidence information. For the lateral near-field area or the blind spot area below, as well as the perception area when the low-altitude aircraft is in a low-speed close-range inspection state, a voxel unit side length smaller than that of the far-field forward perception area is used. When points from different perception units fall into the same voxel unit, the representative point is determined according to the distance, confidence, or preset perception unit priority.

5. The multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection according to claim 1, characterized in that, Based on the spatial proximity between occupied voxel units, adjacent or continuously distributed occupied voxel units are grouped into the same obstacle cluster; For an obstacle cluster, if multiple representative points are distributed close to walls, columns, boxes, building facades, or linear structures, the specific information of the obstacles is determined based on the obstacle cluster, including the center position, circumscribed profile, size, and orientation angle; and the corresponding obstacles are represented by circumscribed profile, circumscribed box, or rotated circumscribed profile. For obstacles with irregular shapes or many internal gaps, multiple occupied voxel units belonging to the same obstacle cluster are retained as obstacle grids, and the actual occupied area of ​​the corresponding obstacle is represented by multiple obstacle grids. Set timestamps and validity periods for obstacle point sets, obstacle clusters, and obstacle grids; in a new planning period, if obstacles are continuously detected near the same spatial location, update the center position, outline, or obstacle grid of the obstacle.

6. The multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection according to claim 1, characterized in that, The airborne computing platform selects a route segment within a preset distance ahead of the low-altitude aircraft's current position in the initial route, and performs smoothing based on the route segment endpoints, turning points, or route control points to obtain a local reference track for the current planning cycle. The local planning coordinate system is the Frenet coordinate system established with the local reference track as the guide line, and the longitudinal reference line is the guide line. The position, velocity, and heading of the low-altitude aircraft in the navigation coordinate system are transformed to the Frenet coordinate system for local track planning. The airborne computing platform projects the position of the low-altitude aircraft in the navigation coordinate system onto the local reference track to obtain the progress coordinates along the direction of the local reference track, and calculates the vertical distance between the projection point and the aircraft position as the lateral offset distance relative to the local reference track; at the same time, it decomposes the velocity of the low-altitude aircraft into velocity components along the direction of the local reference track and lateral velocity components.

7. The multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection according to claim 1, characterized in that, The airborne computing platform generates multiple lateral candidate target states based on the current lateral offset distance relative to the local reference track, the current lateral velocity, and a preset lateral offset distance sampling range; and generates multiple longitudinal candidate target states based on the current velocity component along the local reference track direction, the target velocity, and a preset velocity sampling range. Based on the sampling results of the lateral offset distance relative to the local reference track, the lateral candidate trajectory is generated using a fifth-order polynomial; based on the sampling results of the target velocity along the direction of the local reference track, the longitudinal candidate trajectory is generated using a fourth-order polynomial; the airborne computing platform combines different sampling times, lateral candidate trajectories and longitudinal candidate trajectories to obtain multiple candidate local tracks. For each candidate local trajectory, the airborne computing platform calculates the evaluation cost; the evaluation cost includes the trajectory change magnitude cost, the deviation from the local reference trajectory cost, the target speed deviation cost, and the stability cost relative to the output trajectory of the previous cycle.

8. The multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection according to claim 1, characterized in that, The airborne computing platform first performs an executability test on the candidate local tracks: it determines whether the speed corresponding to each track point on the candidate local track exceeds the maximum speed, whether the acceleration exceeds the maximum acceleration, whether the heading change formed by adjacent track points exceeds the maximum heading change threshold, and whether the altitude of the track points is within the altitude range allowed by the inspection mission; candidate local tracks that fail the executability test will no longer enter the collision detection process. For candidate local tracks that pass the executability test, the airborne computing platform performs safety envelope collision detection. Specifically, for multiple track points on each candidate local track, the airborne computing platform determines the heading angle of the track point based on the positional relationship of adjacent track points, and generates a safety envelope based on the position of the track point, the heading angle, the length, width, and height of the low-altitude aircraft, and a preset safety extension distance. The safety envelope is a three-dimensional bounding box that moves with the track point or other spatial range that can cover the shape of the aircraft and the safety boundary. The airborne computing platform can also generate a swept safety envelope based on the direction of the line connecting adjacent track points. The swept safety envelope covers the spatial range traversed by the safety boundary of the aircraft during the movement of the low-altitude aircraft from the previous track point to the next track point.

9. The multi-sensor fusion obstacle avoidance route planning method for low-altitude aerial vehicles used for urban inspection according to claim 1, characterized in that, The airborne computing platform prioritizes obstacle avoidance planning in the horizontal plane; it avoids obstacles by lateral maneuvering in the horizontal plane. When all candidate local paths in the horizontal plane are impassable, it then triggers vertical obstacle avoidance planning that includes changes in altitude. When vertical obstacle avoidance planning is triggered, the airborne computing platform generates multiple altitude correction candidate values ​​according to a preset altitude step size while maintaining the progress coordinates along the local reference track direction. These values ​​are then recombined to form candidate local tracks that include altitude changes. For candidate local tracks that include altitude changes, the airborne computing platform also performs executability checks, safety envelope collision detection, and evaluation cost calculations. It prioritizes the candidate local track that meets vertical safety requirements, has the lowest cost, and exhibits a relatively gentle altitude change as the output local track. When there are no candidate local tracks in the horizontal plane that meet the requirements for executability detection and collision detection, multiple candidate vertical local tracks are generated within a preset altitude adjustment range based on the horizontal projection of the current local reference track or the initial route. The candidate vertical local tracks are then selected based on the climb rate, descent rate, altitude limit, and safety envelope collision detection results of the low-altitude aircraft. The candidate vertical local track that meets the conditions and has the lowest evaluation cost is then used as the output local track.

10. A terminal device, comprising a processor, a memory, and a computer program stored in the memory; characterized in that, When the processor executes the computer program, it implements the multi-sensor fusion obstacle avoidance route planning method for urban inspection low-altitude aircraft as described in any one of claims 1-9.

Images