Route planning method for cooperation of air-ground heterogeneous unmanned system in complex environment
By generating accurate OSM maps using airborne drones equipped with lidar and converting them into directed graphs, the problem of inaccurate map information in air-ground collaborative path planning under complex environments is solved, enabling collaborative navigation between drones and unmanned vehicles.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING MECHANICAL EQUIP INST
- Filing Date
- 2024-11-20
- Publication Date
- 2026-05-22
AI Technical Summary
In complex environments, the map information generated by existing technologies in air-ground cooperative path planning is not accurate enough and is easily affected by changes in lighting and weather, resulting in inaccurate obstacle location information and making it difficult to achieve effective navigation.
An OSM map is generated using an aerial drone equipped with a lidar. The point cloud data is converted into a two-dimensional raster grayscale image to detect passable areas and fit the road centerline. Combined with SLAM and integrated inertial navigation fusion technology, an accurate OSM map is generated and converted into a directed graph for path planning.
The generated OSM map avoids the influence of lighting and weather conditions, reduces image stitching errors, achieves more accurate path planning, and supports collaborative navigation of heterogeneous air-ground unmanned systems.
Smart Images
Figure CN122072162A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of path planning technology, and in particular to a path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments. Background Technology
[0002] As human curiosity grows, exploration and intelligence gathering in unknown areas are becoming increasingly widespread. Traditional exploration methods often employ a combined air-ground approach. With the development of autonomous driving technology, it has become possible for drones to collaborate with unmanned vehicles (UAVs) to perform tasks. Drones, with their wide field of vision, provide terrain and topographic information to UAVs on the ground. Based on the global environmental information provided by the drones, the UAVs generate drivable path information, thereby ensuring and improving their autonomous navigation capabilities.
[0003] Currently, most aerial-ground collaborative navigation methods use image stitching to generate panoramic maps. While image stitching utilizes high-resolution visual images, its detection performance is easily affected by changes in lighting conditions and weather. In environments with poor lighting or severe weather, it cannot meet the actual work requirements. Furthermore, because the images collected by the UAV at different locations have different perspectives, cumulative errors will occur during subsequent image processing, resulting in inaccurate location information of obstacles in the map. These factors make navigation tasks in complex environments difficult to complete. Summary of the Invention
[0004] Based on the above analysis, the embodiments of the present invention aim to provide a path planning method for collaborative air-ground heterogeneous unmanned systems in complex environments, in order to solve the problem of inaccurate map information obtained by existing technologies.
[0005] This invention provides a path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments, including:
[0006] An OSM map is generated using point cloud data collected by a lidar-equipped UAV, comprising: calculating a two-dimensional raster grayscale image based on point cloud data obtained by the lidar from aerial scanning of the ground; detecting passable areas on the two-dimensional raster grayscale image; fitting road centerlines based on the passable areas to obtain a set of path points; converting the set of path points into OSM format to obtain nodes and node numbers in the OSM map, and obtaining paths in the OSM map.
[0007] After receiving the task, the ground-based unmanned vehicle performs path planning based on its current location and the generated OSM map.
[0008] A further improvement to the above method, the path planning based on the generated OSM map, includes:
[0009] Load the generated OSM map and convert it into a directed graph; perform path planning based on the obtained directed graph.
[0010] Based on a further improvement to the above method, the step of converting the OSM map into a directed graph includes:
[0011] Choose any node in the OSM map as the reference point, convert the coordinates of all other nodes to the relative coordinates of the reference point, and update the coordinates of the nodes in the OSM map.
[0012] Traverse all walkable paths in the OSM map, add the start and end points of each path as nodes to the directed graph, and save the path containing the node as information for that node.
[0013] Directed edges are constructed based on the starting and ending points of each path according to the road attributes and added to the directed graph. The weight of the directed edge is calculated from the length of the current path. The road attributes include one-way roads and two-way roads. If it is a one-way road, a one-way edge is constructed, and if it is a two-way road, a two-way edge is constructed.
[0014] A further improvement to the above method, the step of calculating a two-dimensional raster grayscale image based on point cloud data obtained from aerial scanning of the ground by lidar, includes:
[0015] Point cloud maps are generated based on point cloud data obtained from aerial scanning of the ground by lidar, using SLAM and combined inertial navigation fusion technology.
[0016] The obtained point cloud map is divided into grids, and then subjected to two-dimensional rasterization and normalization to obtain a two-dimensional raster map.
[0017] The average height of each grid cell in the two-dimensional grid map is used as the pixel value of the grayscale image to obtain the two-dimensional grid grayscale image.
[0018] Based on a further improvement of the above method, the two-dimensional rasterization process includes: obtaining the raster where each point is located based on the horizontal and vertical coordinates; and calculating the average height of all points in each raster as the value of that raster.
[0019] The normalization process includes normalizing the values of each grid cell to the range of 0 to 255.
[0020] Based on a further improvement of the above method, the detection of passable regions on the two-dimensional raster grayscale image includes:
[0021] Based on the grayscale value changes of adjacent grid cells, the passable area on the two-dimensional grid grayscale image is obtained.
[0022] A further improvement to the above method is that detecting the passable area on the two-dimensional grid grayscale image further includes performing an erosion and dilation operation on the two-dimensional grid grayscale image beforehand.
[0023] Based on a further improvement to the above method, the step of fitting the road centerline based on the passable area to obtain the path point set includes:
[0024] Along the direction of the passable area, pixel coordinates are collected and recorded one by one along the center line of the passable area. The intersection of the passable area is considered the completion of one path segment. The collection direction is changed at the intersection to start the collection of the next path segment. The first collected pixel is taken as the starting point of the path, and the last collected pixel is taken as the ending point of the path and the starting point of the next path segment. The center lines of each road are obtained by fitting the pixel points of each path using the cubic spline method. The path point set of each road is obtained by sampling the center lines of each road.
[0025] A further improvement to the above method involves converting the set of path points into OSM format to obtain nodes and node numbers in the OSM map, and obtaining paths in the OSM map, including:
[0026] Convert the waypoints of each road to global coordinates, then convert them to OSM format to obtain the nodes and node numbers in the OSM map, and obtain the path in the OSM map based on the nodes in the same path.
[0027] A further improvement to the above method, the path planning based on the obtained directed graph, includes:
[0028] Based on the obtained directed graph, path planning is performed using the A* algorithm.
[0029] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0030] 1. Using a drone equipped with a lidar to generate point clouds, converting the point cloud map into a two-dimensional raster grayscale image, and finally generating an OSM map can avoid the influence of light or weather conditions on data acquisition and the cumulative errors generated during image stitching, making the generated OSM map more accurate.
[0031] 2. The OSM map was converted into a directed graph, enabling successful subsequent path planning;
[0032] 3. A path planning method for ground-based unmanned vehicles is developed based on OSM maps generated by UAV-mounted equipment, enabling collaborative path planning for heterogeneous air-ground unmanned systems.
[0033] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description
[0034] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.
[0035] Figure 1 This is a flowchart illustrating a path planning method for collaborative operation of heterogeneous air-ground unmanned systems in a complex environment, as shown in one embodiment of the present invention.
[0036] Figure 2 This is a schematic diagram illustrating the acquisition of pixel points in a passable area according to an embodiment of the present invention;
[0037] Figure 3 This is a structural diagram of an OSM map according to an embodiment of the present invention.
[0038] Figure label:
[0039] 1-Intersection; 2-Starting point Detailed Implementation
[0040] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.
[0041] A specific embodiment of the present invention discloses a path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments, such as... Figure 1 As shown, it includes:
[0042] S1: Generate an OSM map using point cloud data collected by a lidar-equipped UAV, including: calculating a two-dimensional raster grayscale image based on point cloud data obtained by the lidar from aerial scanning of the ground; detecting passable areas on the two-dimensional raster grayscale image; fitting road centerlines based on the passable areas to obtain a set of path points; converting the set of path points into OSM format to obtain nodes and node numbers in the OSM map, and obtaining paths in the OSM map;
[0043] S2: After receiving the task, the ground-based unmanned vehicle performs path planning based on its current location and the generated OSM map.
[0044] By using a drone equipped with a lidar to generate point clouds, converting the point cloud map into a two-dimensional raster grayscale image, and finally generating an OSM map, the influence of lighting or weather conditions on data acquisition and the cumulative errors caused by image stitching can be avoided, making the generated OSM map more accurate. In addition, the OSM map generated by the drone-mounted equipment can be used for path planning of ground unmanned vehicles, realizing a path planning method for collaborative air-ground heterogeneous unmanned systems.
[0045] Specifically, step S1 includes S11-S13;
[0046] S11: A two-dimensional raster grayscale image is calculated based on point cloud data obtained from aerial scanning of the ground by lidar, including:
[0047] The UAV carries a lidar scanner to scan the ground, and the scan results are presented as point clouds. Based on the point cloud data obtained by the lidar from the air, a point cloud map is generated using SLAM and combined inertial navigation fusion technology. The coordinates of each point in the point cloud map can be represented as:
[0048] p i =(x i ,y i ,z i ), i = 0, 1, 2...n
[0049] Where, p i Let i represent the point with index i, and n represent the total number of points in the point cloud map. i ,y i ) represents point p i Geographic location information, z i Point p i Elevation information;
[0050] Furthermore, the obtained point cloud map is divided into grids, and after 2D rasterization and normalization, a 2D raster map is obtained, including:
[0051] The obtained point cloud map is subjected to coordinate statistics, and the minimum and maximum values of the x and y coordinates are obtained, respectively denoted as (x...). min ,x max ) and (y min ,y max The point cloud map is rasterized to obtain a map of size (x...). The x-axis represents the east-west coordinates of the point's geographic location; the y-axis represents the north-south coordinates of the point's geographic location; and the point cloud map is rasterized to obtain a map of size (x...). max -x min ,y max -y min A raster map of x, where x max -x minThe difference in the x-coordinates, y max -y min This represents the difference in the vertical coordinates; for example, the resolution of the raster is set to 1 meter during rasterization.
[0052] Laser point cloud 2D rasterization is the process of mapping 3D point cloud data onto a 2D raster map. This process can convert point cloud data into more easily processed 2D data. Specifically, the raster where each point is located is obtained based on the x-coordinate and y-coordinate of each point; the average height of all points in each raster is calculated as the value of that raster.
[0053] The normalization process includes: normalizing the values of each grid cell to the range of 0 to 255 using the following formula:
[0054]
[0055] in, This is the raw value of the average height of a raster cell. The normalized value of the average height of the grid;
[0056] Furthermore, the average height of each grid cell in the two-dimensional grid map is used as the pixel value of the grayscale image to obtain a two-dimensional grid grayscale image; wherein, the size of the two-dimensional grid grayscale image is the same as the size of the grid map, and each pixel in the two-dimensional grid grayscale image is in the range of 0 to 255.
[0057] S12: Detect passable areas on the two-dimensional raster grayscale image, fit a road centerline based on the passable areas, and obtain a set of path points, including:
[0058] The two-dimensional raster grayscale image is pre-processed with erosion and dilation to remove raster value anomalies caused by local point cloud anomalies, thereby eliminating noise and separating objects.
[0059] Furthermore, after removing raster value anomalies, the passable area on the two-dimensional raster grayscale image is obtained based on the grayscale value changes of adjacent rasters. It should be noted that for terrain with relatively continuous road changes, such as uphill and downhill sections of roads, the grayscale value changes are relatively uniform in the grayscale image. Thus, passable areas can be identified on the image based on the obvious continuous changes in the grayscale values of adjacent rasters. For unobstructed planar roads, the grayscale values of adjacent rasters remain basically unchanged, and these can also be considered as passable areas.
[0060] Furthermore, based on the road centerline fitted within the traversable area, a set of path points is obtained, including:
[0061] Along the direction of the passable area, pixel coordinates are collected and recorded one by one along the center line of the passable area. The intersection of the passable area is considered the completion of one path segment. At the intersection, the collection direction is changed, and the collection of the next path segment begins. The first collected pixel is taken as the starting point of the path, and the last collected pixel is taken as the ending point of the path and the starting point of the next path segment. Since the collected points are discrete, to ensure overall smoothness, cubic spline fitting is used to fit the pixel points of each path to obtain the center lines of each road. The path point sets of each road are obtained by sampling each road center line. The sampling interval can be set according to the resolution requirements; for example, the sampling interval is set to 0.5 meters.
[0062] It should be noted that collecting pixel coordinates one by one along the center line of the passable area is to avoid deep pits, ditches, and obstacles on both sides of the road; when the passable area is a straight line, the interval between the collected pixels can be appropriately increased; when the passable area is a curve, the number of collected pixels can be increased to ensure that the trend of the collected pixels matches the trend of the actual road.
[0063] like Figure 2 As shown, pixels are collected starting from the starting point. When a passable intersection is collected, the pixel at the intersection is taken as the endpoint of this collection, completing one collection cycle. When collecting to the right or forward, the pixel at the intersection is taken as the starting point, so that the pixel at the intersection can be used as the starting or ending point of three paths simultaneously. During path planning, this information can be used to query the connection relationships between the paths, thereby providing information between nodes for global path planning.
[0064] S13: Convert the set of path points into OSM format to obtain the nodes and node numbers in the OSM map, and obtain the paths in the OSM map;
[0065] It should be noted that, as Figure 3 As shown, the structure of map data includes spatial elements, namely data primitives, and attribute elements; among them, data primitives mainly include nodes used to define the location of points in space and paths used to define lines; attribute elements are used to describe vector data primitives.
[0066] Specifically, the waypoints of each road are converted into global coordinates, then converted into OSM format to obtain the nodes and node numbers in the OSM map, and the path in the OSM map is obtained based on the nodes in the same path. It is worth noting that the two-dimensional raster grayscale image uses pixel coordinates, while the OSM map uses global coordinates, so coordinate conversion is required.
[0067] S2: After receiving the task, the ground-based unmanned vehicle performs path planning based on its current location and the generated OSM map;
[0068] Specifically, step S2 includes S21-S23;
[0069] S21: Load the generated OSM map; for example, load the obtained XML file by using the map name and storage path to complete the OSM map loading;
[0070] S22: Convert the OSM map into a directed graph, including:
[0071] Choose any node in the OSM map as the reference point, convert the coordinates of all other nodes to the relative coordinates of the reference point, and update the coordinates of the nodes in the OSM map.
[0072] It should be noted that performing relative coordinate transformation can reduce the node values, save subsequent calculation time, and reduce the calculation difficulty;
[0073] Traverse all walkable paths in the OSM map, add the start and end points of each path as nodes to the directed graph, and save the path containing the node as information for that node.
[0074] Directed edges are constructed based on the start and end points of each path according to road attributes and added to the directed graph. The weight of each directed edge is calculated from the length of the current path. The road attributes include one-way roads and two-way roads. If it is a one-way road, a one-way edge is constructed; if it is a two-way road, a two-way edge is constructed. For example, the start point of a one-way edge is S1, the end point is E1, and the weight is W1; one of the two-way edges has a start point of S2, an end point of E2, and a weight of W2; the other two-way edge has a start point of E2, an end point of S2, and a weight of W2.
[0075] It should be noted that the OSM map stores information that can be used to calculate road weights, but does not provide a specific solution. When the ground-based autonomous vehicle receives a task and uses the A* algorithm to plan its path based on its current location, the input information needs to include the weights of the directed edges. Therefore, a directed graph transformation is required to obtain the weights of the directed edges.
[0076] S23: Based on the obtained directed graph, perform path planning using the A* algorithm;
[0077] Specifically, when using the A* algorithm for path planning from the starting point to the destination, the starting point is the current location of the autonomous vehicle, and the destination is the mission objective point. The parameters of the A* algorithm include:
[0078] The cost estimation function f(n) from the starting point through state point n to the target point, the actual cost function g(n) from the starting point to state point n, and the heuristic function h(n) from state point n to the target point; where f(n) is obtained using the following formula:
[0079] f(n) = g(n) + h(n)
[0080] We can obtain g(n) using the following formula:
[0081]
[0082] Where, N s N represents the starting point. s The set of nodes traversed to reach state point n is
[0083] {N1,N2,N3,...,N m}, where k represents the weight of the directed edge between the two nodes;
[0084] Using Euclidean distance, h(n) can be obtained using the following formula:
[0085]
[0086] Where: x s The x-coordinate of state point n is represented by y. s The ordinate of state point n is represented by x. d The x-coordinate of the target point, y d Represents the ordinate of the target point;
[0087] The A* algorithm takes a directed graph G = (V, E) and a starting point as input; where V represents the set of all nodes in the directed graph, and E represents the set of all edges in the directed graph; the specific path planning steps are as follows:
[0088] Add the starting point to the openList collection and repeat the following steps:
[0089] Traverse the openList to find the node with the smallest f value, set it as the current node, and move it to the closeList. For the current node's neighbors, check if they are not part of the directed graph path or are already in the closeList. If they are not in the openList, add them to the openList, set the current node as their parent, record their f, g, and h values, and compare the original g value with the g value obtained by reaching the neighbor via the current node. If the g value obtained by reaching the neighbor via the current node is smaller, change the parent node of the neighbor to the current node and recalculate the g and h values. Continue this process until the endpoint is added to the openList. Then, backtrack from the endpoint back to the starting point along the parent nodes to obtain the optimal path.
[0090] Compared with existing technologies, the method provided in this embodiment utilizes a UAV equipped with a LiDAR to generate point clouds, converts the point cloud map into a two-dimensional raster grayscale image, and finally generates an OSM map. This can avoid the influence of light or weather conditions on data acquisition and the cumulative errors generated during image stitching, making the generated OSM map more accurate. Converting the OSM map into a directed graph facilitates subsequent path planning. Based on the OSM map generated by the UAV-equipped device, path planning is performed for ground unmanned vehicles, realizing a path planning method for collaborative air-ground heterogeneous unmanned systems.
[0091] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.
[0092] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments, characterized in that, include: An OSM map is generated using point cloud data collected by a lidar-equipped UAV, comprising: calculating a two-dimensional raster grayscale image based on point cloud data obtained by the lidar from aerial scanning of the ground; detecting passable areas on the two-dimensional raster grayscale image; fitting road centerlines based on the passable areas to obtain a set of path points; converting the set of path points into OSM format to obtain nodes and node numbers in the OSM map, and obtaining paths in the OSM map. After receiving the task, the ground-based unmanned vehicle performs path planning based on its current location and the generated OSM map.
2. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 1, characterized in that, The route planning based on the generated OSM map includes: Load the generated OSM map and convert it into a directed graph; perform path planning based on the obtained directed graph.
3. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 2, characterized in that, The process of converting the OSM map into a directed graph includes: Choose any node in the OSM map as the reference point, convert the coordinates of all other nodes to the relative coordinates of the reference point, and update the coordinates of the nodes in the OSM map. Traverse all walkable paths in the OSM map, add the start and end points of each path as nodes to the directed graph, and save the path containing the node as information for that node. Directed edges are constructed based on the starting and ending points of each path according to the road attributes and added to the directed graph. The weight of the directed edge is calculated from the length of the current path. The road attributes include one-way roads and two-way roads. If it is a one-way road, a one-way edge is constructed, and if it is a two-way road, a two-way edge is constructed.
4. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 1, characterized in that, The calculation of the two-dimensional raster grayscale image based on point cloud data obtained from aerial scanning of the ground by lidar includes: Point cloud maps are generated based on point cloud data obtained from aerial scanning of the ground by lidar, using SLAM and combined inertial navigation fusion technology. The obtained point cloud map is divided into grids, and then subjected to two-dimensional rasterization and normalization to obtain a two-dimensional raster map. The average height of each grid cell in the two-dimensional grid map is used as the pixel value of the grayscale image to obtain the two-dimensional grid grayscale image.
5. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 4, characterized in that, The two-dimensional rasterization process includes: obtaining the grid cell containing each point based on the horizontal and vertical coordinates; and calculating the average height of all points in each grid cell as the value of that grid cell. The normalization process includes normalizing the values of each grid cell to the range of 0 to 255.
6. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 1, characterized in that, The detection of passable regions on the two-dimensional raster grayscale image includes: Based on the grayscale value changes of adjacent grid cells, the passable area on the two-dimensional grid grayscale image is obtained.
7. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 6, characterized in that, The detection of passable areas on the two-dimensional raster grayscale image also includes performing an erosion and dilation operation on the two-dimensional raster grayscale image beforehand.
8. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 1, characterized in that, The method of fitting the road centerline based on the passable area to obtain a set of path points includes: Along the direction of the passable area, pixel coordinates are collected and recorded one by one along the center line of the passable area. The intersection of the passable area is considered the completion of one path segment. The collection direction is changed at the intersection to start the collection of the next path segment. The first collected pixel is taken as the starting point of the path, and the last collected pixel is taken as the ending point of the path and the starting point of the next path segment. The center lines of each road are obtained by fitting the pixel points of each path using the cubic spline method. The path point set of each road is obtained by sampling the center lines of each road.
9. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 8, characterized in that, The process of converting the set of path points to OSM format to obtain nodes and node numbers in the OSM map, and obtaining paths in the OSM map, includes: Convert the waypoints of each road to global coordinates, then convert them to OSM format to obtain the nodes and node numbers in the OSM map, and obtain the path in the OSM map based on the nodes in the same path.
10. The path planning method for collaborative operation of heterogeneous air-ground unmanned systems in complex environments according to claim 3, characterized in that, The path planning based on the obtained directed graph includes: Based on the obtained directed graph, path planning is performed using the A* algorithm.