Unmanned vehicle path planning method and device based on unmanned aerial vehicle orthographic image

Abstract

The application discloses a kind of unmanned vehicle path planning method and device based on unmanned aerial vehicle orthographic image, wherein the method comprises: obtaining the orthographic image photographed by unmanned aerial vehicle, the orthographic image is handled with semantic segmentation, the object class identified in semantic segmentation result is labeled;Extract the pixel region corresponding to each semantic label, generate the corresponding semantic category image, and perform TDF truncated euclidean distance field transformation on each semantic category image;Obtain the 3D point cloud data of the laser radar sensor of unmanned vehicle, map the 3D point cloud data of unmanned vehicle to the coordinate system of unmanned aerial vehicle TDF image using particle filtering algorithm, obtain the corresponding unmanned vehicle positioning information from unmanned aerial vehicle TDF image, and obtain obstacle positioning information according to unmanned vehicle positioning information;According to unmanned vehicle positioning information and obstacle positioning information, the path planning of unmanned vehicle is carried out.The application can improve the positioning accuracy of unmanned vehicle in complex environment and the accuracy and reliability of navigation path.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned vehicles, and particularly relates to a method and device for unmanned vehicle path planning based on unmanned aerial vehicle orthographic images. BACKGROUND

[0002] This section is intended to provide background or context to the embodiments of the application recited in the claims. The description herein does not constitute admission that the prior art is prior art nor does it constitute an admission of any description in the section as prior art unless expressly so stated.

[0003] Unmanned aerial vehicle-assisted unmanned vehicle navigation is a frontier research direction in the field of intelligent systems. The core is to make up for the limitations of the ground view of the unmanned vehicle by means of the high-altitude view and dynamic sensing capability of the unmanned aerial vehicle, and then realize collaborative positioning, environment modeling and path optimization. The existing technology mainly realizes it in the following way: the unmanned aerial vehicle carries a laser radar to ascend, and a point cloud map is constructed by using SLAM technology. The unmanned vehicle navigates by combining the RTK positioning technology with the point cloud map of the unmanned aerial vehicle. However, the existing technology cannot meet the accuracy requirements of the unmanned vehicle positioning, and cannot guarantee the accuracy and reliability of the unmanned vehicle navigation path. SUMMARY

[0004] The embodiments of the present application provide a method for unmanned vehicle path planning based on unmanned aerial vehicle orthographic images, which is used to improve the positioning accuracy of the unmanned vehicle and guarantee the accuracy and reliability of the unmanned vehicle navigation path. The method comprises the following steps:

[0005] An orthographic image photographed by an unmanned aerial vehicle is obtained, semantic segmentation processing is performed on the orthographic image to obtain a semantic segmentation result, object categories recognized in the semantic segmentation result are labeled to obtain semantic labels of each object category, and the object categories include a road area, a building, vegetation, a house, a vehicle and a pedestrian;

[0006] A pixel area corresponding to each semantic label is extracted, a corresponding semantic category image is generated, TDF truncated Euclidean distance field transformation is performed on each semantic category image, and a corresponding TDF image is obtained;

[0007] 3D point cloud data of a laser radar sensor of the unmanned vehicle is obtained, the 3D point cloud data of the unmanned vehicle is mapped to a coordinate system of the TDF image of the unmanned aerial vehicle by using a particle filtering algorithm, corresponding unmanned vehicle positioning information is obtained from the TDF image of the unmanned aerial vehicle, and obstacle positioning information is obtained according to the unmanned vehicle positioning information;

[0008] According to the unmanned vehicle positioning information and the obstacle positioning information, the path of the unmanned vehicle is planned by using an A* algorithm and a Bessel curve.

[0009] The embodiments of the present application also provide a device for unmanned vehicle path planning based on unmanned aerial vehicle orthographic images, which is used to improve the positioning accuracy of the unmanned vehicle and guarantee the accuracy and reliability of the unmanned vehicle navigation path. The device comprises:

[0010] The unmanned aerial vehicle orthographic image semantic segmentation module is configured to acquire an orthographic image captured by an unmanned aerial vehicle, perform semantic segmentation on the orthographic image to obtain a semantic segmentation result, label object categories recognized in the semantic segmentation result, and obtain semantic labels of each object category.

[0011] The TDF image conversion module is configured to extract a pixel region corresponding to each semantic label, generate a corresponding semantic category image, perform TDF truncated Euclidean distance field conversion on each semantic category image, and obtain a corresponding TDF image.

[0012] The unmanned vehicle positioning information acquisition module is configured to acquire 3D point cloud data of a laser radar sensor of the unmanned vehicle, map the 3D point cloud data of the unmanned vehicle to a coordinate system of the unmanned aerial vehicle TDF image by using a particle filtering algorithm, obtain corresponding unmanned vehicle positioning information from the unmanned aerial vehicle TDF image, and obtain obstacle positioning information according to the unmanned vehicle positioning information.

[0013] The unmanned vehicle path planning module is configured to perform path planning of the unmanned vehicle by using an A* algorithm and a Bessel curve according to the unmanned vehicle positioning information and the obstacle positioning information.

[0014] The embodiment of the present application further provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the unmanned vehicle path planning method based on an unmanned aerial vehicle orthographic image when executing the computer program.

[0015] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the unmanned vehicle path planning method based on an unmanned aerial vehicle orthographic image.

[0016] The embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the unmanned vehicle path planning method based on an unmanned aerial vehicle orthographic image.

[0017] In the embodiment of the present application, by acquiring the orthographic image shot by the unmanned aerial vehicle, the orthographic image is subjected to semantic segmentation processing to obtain a semantic segmentation result, the object categories recognized in the semantic segmentation result are labeled to obtain semantic labels of each object category; the object categories include road area, building, vegetation, house, vehicle and pedestrian; a pixel area corresponding to each semantic label is extracted to generate a corresponding semantic category image, the TDF truncated Euclidean distance field transformation is performed on each semantic category image to obtain a corresponding TDF image; the 3D point cloud data of the laser radar sensor of the unmanned vehicle is acquired, the 3D point cloud data of the unmanned vehicle is mapped to the coordinate system of the unmanned aerial vehicle TDF image by using the particle filtering algorithm, the corresponding unmanned vehicle positioning information is obtained from the unmanned aerial vehicle TDF image, and the obstacle positioning information is obtained according to the unmanned vehicle positioning information; according to the unmanned vehicle positioning information and the obstacle positioning information, the A* algorithm and the Bessel curve are used for path planning of the unmanned vehicle. In the above process, the embodiment of the present application constructs the matching reference of the 3D point cloud data of the unmanned vehicle to the 2D image by the semantic segmentation processing and the TDF image of multiple semantic categories, realizes the positioning of the 3D point cloud data of the unmanned vehicle to the TDF image by combining the particle filtering algorithm, and generates a smooth obstacle avoidance path by using the A* algorithm and the Bessel curve based on the unmanned vehicle positioning information in the TDF image, so as to improve the positioning accuracy, the accuracy and the reliability of the navigation path of the unmanned vehicle in a complex environment. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort. In the drawings:

[0019] Figure 1 The flow chart of the unmanned vehicle path planning method based on the orthographic image of the unmanned aerial vehicle in the embodiment of the present application;

[0020] Figure 2 The schematic diagram of the two-class image of semantic segmentation in the embodiment of the present application;

[0021] Figure 3 The schematic diagram of the unmanned vehicle path planning device based on the orthographic image of the unmanned aerial vehicle in the embodiment of the present application;

[0022] Figure 4 The specific schematic diagram of the unmanned vehicle path planning device based on the orthographic image of the unmanned aerial vehicle in the embodiment of the present application;

[0023] Figure 5 The schematic diagram of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, further detailed description of the embodiments of the present application will be given below with reference to the drawings. Herein, the illustrative embodiments of the present application and the description thereof are used to explain the present application, but not as a limitation of the present application.

[0025] Figure 1 The flow chart of the method for unmanned vehicle path planning based on the orthographic image of the unmanned aerial vehicle in the embodiments of the present application, the method comprises:

[0026] Step 101, obtaining the orthographic image shot by the unmanned aerial vehicle, performing semantic segmentation processing on the orthographic image to obtain a semantic segmentation result, labeling the object categories recognized in the semantic segmentation result to obtain semantic labels of each object category; the object categories include road area, building, vegetation, house, vehicle and pedestrian;

[0027] Step 102, extracting the pixel area corresponding to each semantic label, generating a corresponding semantic category image, performing TDF truncated Euclidean distance field transformation on each semantic category image to obtain a corresponding TDF image;

[0028] Step 103, obtaining 3D point cloud data of the laser radar sensor of the unmanned vehicle, mapping the 3D point cloud data of the unmanned vehicle to the coordinate system of the TDF image of the unmanned aerial vehicle by using the particle filtering algorithm, obtaining corresponding unmanned vehicle positioning information from the TDF image of the unmanned aerial vehicle, and obtaining obstacle positioning information according to the unmanned vehicle positioning information;

[0029] Step 104, performing path planning of the unmanned vehicle by using A* algorithm and Bessel curve according to the unmanned vehicle positioning information and the obstacle positioning information.

[0030] Each step will be described in detail below.

[0031] In step 101, the orthographic image shot by the unmanned aerial vehicle is obtained, semantic segmentation processing is performed on the orthographic image to obtain a semantic segmentation result, the object categories recognized in the semantic segmentation result are labeled to obtain semantic labels of each object category; the object categories include road area, building, vegetation, house, vehicle and pedestrian.

[0032] In specific embodiments, the orthographic image generated by the unmanned aerial vehicle is used to realize unmanned vehicle navigation. The specific role of the orthographic image is: 1. used for cross-view positioning of the unmanned vehicle, and 2. used as a global static map for path planning of the unmanned vehicle. In the unmanned vehicle positioning algorithm simulation, the positioning is realized by matching the 3D point cloud generated by the laser radar on the unmanned vehicle with the 2D orthographic image of the unmanned aerial vehicle, without relying on RTK / GPS. In order to improve the accuracy, the orthographic image and the laser radar point cloud are subjected to semantic segmentation, the weights of buildings and roads are increased, the weight of trees is reduced, and the influence caused by the difference in the view angle of the unmanned aerial vehicle and the unmanned vehicle is weakened. In the path planning process, the orthographic image of the unmanned aerial vehicle provides a global static map for the unmanned vehicle, the laser radar point cloud on the unmanned vehicle detects obstacles in real time to form a local dynamic map, and global path and local path planning are performed in combination with the global static map.

[0033] In specific embodiments, a deep learning model is used to perform semantic segmentation on the orthographic image taken by the unmanned aerial vehicle to identify pixel-level semantic labels, and each pixel has a corresponding semantic label. In order to more accurately define the drivable road range, edge detection is also required. The edge detection method can identify the boundary lines in the image or point cloud, and these boundaries are often the dividing lines between the road and the surrounding environment. By extracting these boundary lines, the contour of the drivable road can be further refined. Unmanned aerial vehicle operation: in the disaster area, the operator controls the low-cost unmanned aerial vehicle (such as a consumer multi-rotor unmanned aerial vehicle) to ascend, takes aerial photographs of the disaster area, and obtains an orthographic image. The image processing module carried by the unmanned aerial vehicle performs semantic segmentation and edge detection on the orthographic image to identify the drivable road and the obstacle boundary, then generates a TDF image, and transmits the map to the unmanned vehicle through the wireless communication module.

[0034] In step 102, the pixel region corresponding to each semantic label is extracted to generate a corresponding semantic category image, and TDF truncated Euclidean distance field transformation is performed on each semantic category image to obtain a corresponding TDF image.

[0035] Figure 2 The schematic diagram of the two images segmented by the semantic segmentation in the embodiments of the present application is shown in the following figure. In specific embodiments, according to the semantic segmentation result, two images are generated:

[0036] One type of image, the orthographic image is binarized to generate a drivable area map, the road is set as a drivable area, and is assigned a value of 255, and the building, vegetation and other areas are set to 0, thereby providing a global drivable area map for the path planning module.

[0037] In an embodiment, it further includes:

[0038] Based on the semantic segmentation result and the object category of the unmanned aerial vehicle orthographic image, the orthographic image is binarized to generate a drivable area binary map: the road area is marked as a drivable area, and the non-road area is marked as a non-drivable area.

[0039] Another type of image is to generate a TDF image (Truncated Distance Field) for each semantic label. There are as many images as there are semantic labels. Taking the road image as an example, for the semantic segmentation orthographic image, check the semantic label of each pixel, mark the road as 255, and other labels as 0; the TDF image is used to represent the Euclidean distance between any non-target pixel point and the target pixel point on each semantic category image; wherein the pixel value of the target pixel point is the maximum value, and the pixel value of the non-target pixel point decreases with the increase of the Euclidean distance from the target pixel point; the Euclidean distance evaluates the matching degree of the laser radar 3D point cloud data of the unmanned vehicle and the target pixel point in the orthographic image of the unmanned aerial vehicle.

[0040] In step 103, 3D point cloud data of the unmanned vehicle laser radar sensor is obtained, and the particle filtering algorithm is used to map the 3D point cloud data of the unmanned vehicle to the coordinate system of the unmanned aerial vehicle TDF image, to obtain corresponding unmanned vehicle positioning information from the unmanned aerial vehicle TDF image, and to obtain obstacle positioning information according to the unmanned vehicle positioning information.

[0041] In an embodiment, the particle filtering algorithm is used to map the 3D point cloud data of the unmanned vehicle to the coordinate system of the unmanned aerial vehicle TDF image, including:

[0042] Based on the particle state and the scale parameter, the laser radar 3D point cloud data of the unmanned vehicle is mapped to the unmanned aerial vehicle TDF image.

[0043] In an embodiment, based on the particle state and the scale parameter, the laser radar 3D point cloud data of the unmanned vehicle is mapped to the unmanned aerial vehicle TDF image, including:

[0044] According to the following mapping formula, the laser radar 3D point cloud data of the unmanned vehicle is mapped to the unmanned aerial vehicle TDF image:

[0045]

[0046] wherein, is the spatial coordinates of the particle state, t is the translation, is the rotation angle of the unmanned vehicle relative to the TDF image coordinate system for coordinate conversion, is the spatial coordinates of the laser radar 3D point cloud data of the unmanned vehicle, is the scale parameter, is the pixel coordinates of the mapped TDF image.

[0047] In a specific embodiment, based on particle state and scale parameters, the spatial coordinates of the LiDAR 3D point cloud data are rotated and translated before being mapped to pixels in the TDF image. The pixel value corresponding to each pixel represents the degree of matching between the autonomous vehicle point cloud data and the target pixel, and the degree of matching is used to calculate the particle weight. The calculation of particle weight in the particle filtering algorithm includes the following steps:

[0048] 1. Determine the semantic matching degree: Construct a semantic feature matching network, inputting the semantic segmentation feature map of the UAV orthophoto and the semantic features of the UAV LiDAR point cloud; output the matching degree of each semantic category. s i ;

[0049] 2. Dynamic weight matrix calculation: based on the degree of matching. s i Calculate adaptive semantic weights: For semantic categories with high matching scores (such as buildings in urban scenes), increase their weight in the particle filter algorithm; for semantic categories with low matching scores (such as vegetation in forest scenes), decrease their weight. Through dynamic weight allocation, the localization results of the autonomous vehicle determined by the particle filter algorithm are more accurate and can adapt to different scenes, such as urban and forest scenes, reducing the degree of localization drift caused by occlusion.

[0050] The weight calculation formula is as follows:

[0051]

[0052] in, For the first Weight of class semantics For the first The degree of semantic matching;

[0053] 3. Particle weight fusion: Generate a weight matrix based on the matching degree of each semantic class, and introduce the weight matrix into the state estimation process of particle filtering to optimize the localization results.

[0054] In step 104, the unmanned vehicle's path planning is performed using the A* algorithm and Bessel curves, based on the unmanned vehicle's localization information and obstacle localization information.

[0055] In one embodiment, path planning for the autonomous vehicle is performed using the A* algorithm and Bessel curves based on the vehicle's localization information and obstacle localization information, including:

[0056] In the drivable area, according to the unmanned vehicle positioning information and the obstacle positioning information, the path planning of the unmanned vehicle is carried out by using the A* algorithm and the Bessel curve. The A* algorithm can reduce the number of searched nodes while ensuring the shortest path. The Bessel curve can ensure the curvature continuity of the local path planning curve. On the basis of ensuring the shortest path, the expected trajectory of the vehicle is planned as smooth as possible.

[0057] In the embodiment, the unmanned vehicle end operation: after the unmanned vehicle enters the disaster area, the operator sets the material dropping point on the orthographic image transmitted by the unmanned aerial vehicle as the target point. The laser radar of the unmanned vehicle acquires the point cloud data of the surrounding environment in real time, and identifies the obstacles (such as collapsed buildings, trees, etc.) and the drivable area through semantic segmentation by PointNet. In the case of no GPS signal, the unmanned vehicle uses the particle swarm filtering algorithm, combines the TDF map and the laser radar point cloud data to carry out positioning. According to the positioning result and the real-time obstacle information, a cost map is generated, and then the A* algorithm and the Bessel curve are used for global and local path planning to output the control signal to control the vehicle to drive and accurately transport the materials to the target point.

[0058] In addition, in the path planning process, based on the dynamic environment prediction model of the Transformer architecture, the historical laser radar point cloud sequence and the semantic features of the orthographic image are used to predict the motion trajectory of the obstacle in the future period of time. The specific steps are as follows:

[0059] 1. Based on the historical laser radar point cloud sequence, the spatiotemporal features are extracted by the Transformer encoder; the semantic information of the orthographic image is fused, and the future obstacle motion trajectory prediction is output through the Transformer decoder;

[0060] 2. The current state space is obtained by using the reinforcement learning algorithm: the vehicle position, speed and obstacle distribution, according to the obstacle trajectory prediction, the weight of the heuristic function of the A* algorithm and the adjustment amount of the Bessel curve control parameter in the action space are generated, and the quantified weighted values of energy consumption, path smoothness and obstacle avoidance safety are taken as the reward function to generate the optimal path planning parameter in real time;

[0061] 3. The avoidance path is generated in advance according to the predicted obstacle trajectory, and the path is optimized online in combination with the real-time laser radar point cloud data in the actual driving.

[0062] In the embodiment of the application, an unmanned vehicle path planning device based on an unmanned aerial vehicle orthographic image is also provided, as described in the following embodiment. Since the principle of solving the problem of the device is similar to the unmanned vehicle path planning method based on the unmanned aerial vehicle orthographic image, the implementation of the device can be referred to the implementation of the unmanned vehicle path planning method based on the unmanned aerial vehicle orthographic image, and the repeated parts will not be described again.

[0063] Figure 3 FIG. 1 is a schematic diagram of an unmanned vehicle path planning device based on an unmanned aerial vehicle orthographic image according to an embodiment of the present application. The device includes:

[0064] An unmanned aerial vehicle orthographic image semantic segmentation module 301 is configured to acquire an orthographic image captured by an unmanned aerial vehicle, perform semantic segmentation on the orthographic image to obtain a semantic segmentation result, label object categories identified in the semantic segmentation result to obtain semantic labels of each object category, and the object categories include a road area, a building, vegetation, a house, a vehicle, and a pedestrian.

[0065] A TDF image transformation module 302 is configured to extract a pixel area corresponding to each semantic label, generate a corresponding semantic category image, and perform TDF truncated Euclidean distance field transformation on each semantic category image to obtain a corresponding TDF image.

[0066] An unmanned vehicle positioning information acquisition module 303 is configured to acquire 3D point cloud data of a laser radar sensor of an unmanned vehicle, map the 3D point cloud data of the unmanned vehicle to a coordinate system of a TDF image of an unmanned aerial vehicle using a particle filtering algorithm, obtain corresponding unmanned vehicle positioning information from the TDF image of the unmanned aerial vehicle, and obtain obstacle positioning information according to the unmanned vehicle positioning information.

[0067] An unmanned vehicle path planning module 304 is configured to perform path planning for the unmanned vehicle using an A* algorithm and a Bessel curve according to the unmanned vehicle positioning information and the obstacle positioning information.

[0068] In an embodiment, the unmanned vehicle positioning information acquisition module 303 is further configured to:

[0069] Map the 3D point cloud data of the laser radar of the unmanned vehicle to the TDF image of the unmanned aerial vehicle based on a particle state and a scale parameter.

[0070] In an embodiment, the unmanned vehicle positioning information acquisition module 303 is specifically configured to:

[0071] Map the 3D point cloud data of the laser radar of the unmanned vehicle to the TDF image of the unmanned aerial vehicle according to the following mapping formula:

[0072]

[0073] wherein, is a spatial coordinate of the particle state, t is a translation amount, is a rotation angle for coordinate conversion of the unmanned vehicle relative to the TDF image coordinate system, is a spatial coordinate of the 3D point cloud data of the laser radar of the unmanned vehicle, is a scale parameter, For the mapped TDF image pixel coordinates.

[0074] Figure 4 For the specific schematic diagram of the unmanned vehicle path planning device based on the unmanned aerial vehicle orthographic image in the embodiment of the application, in an embodiment, the orthographic image binarization processing module 401 is further included, and is specifically used for:

[0075] Based on the object category of the semantic segmentation result and the unmanned aerial vehicle orthographic image, the orthographic image is binarized to generate a drivable area binary map: the road area is marked as a drivable area, and the non-road area is marked as a non-drivable area.

[0076] In an embodiment, the unmanned vehicle path planning module 304 is specifically used for:

[0077] In the drivable area, according to the unmanned vehicle positioning information and the obstacle positioning information, the A* algorithm and the Bessel curve are used for path planning of the unmanned vehicle.

[0078] The embodiment of the application further provides a computer device, Figure 5 For the schematic diagram of the computer device in the embodiment of the application, the computer device 500 includes a memory 510, a processor 520, and a computer program 530 stored in the memory 510 and capable of running on the processor 520, and the processor 520 implements the above-mentioned unmanned vehicle path planning method based on the unmanned aerial vehicle orthographic image when executing the computer program 530.

[0079] The embodiment of the application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned unmanned vehicle path planning method based on the unmanned aerial vehicle orthographic image.

[0080] The embodiment of the application further provides a computer program product, the computer program product includes a computer program, and the computer program is executed by a processor to implement the above-mentioned unmanned vehicle path planning method based on the unmanned aerial vehicle orthographic image.

[0081] In the embodiment of the present application, by acquiring the orthographic image shot by the unmanned aerial vehicle, the orthographic image is subjected to semantic segmentation processing to obtain a semantic segmentation result, object categories recognized in the semantic segmentation result are labeled to obtain semantic labels of each object category; the object categories include road area, building, vegetation, house, vehicle and pedestrian; a pixel area corresponding to each semantic label is extracted to generate a corresponding semantic category image, the TDF truncated Euclidean distance field transformation is performed on each semantic category image to obtain a corresponding TDF image; 3D point cloud data of the unmanned vehicle laser radar sensor is acquired, the 3D point cloud data of the unmanned vehicle is mapped to the coordinate system of the unmanned aerial vehicle TDF image by using the particle filtering algorithm, corresponding unmanned vehicle positioning information is obtained from the unmanned aerial vehicle TDF image, and obstacle positioning information is obtained according to the unmanned vehicle positioning information; according to the unmanned vehicle positioning information and the obstacle positioning information, the A* algorithm and the Bessel curve are used for path planning of the unmanned vehicle. In the above process, by using the semantic segmentation processing and the TDF image of multiple semantic categories, the embodiment of the present application constructs a matching reference of the 3D point cloud data of the unmanned vehicle to the 2D image, realizes the positioning of the 3D point cloud data of the unmanned vehicle to the TDF image by combining the particle filtering algorithm, and generates a smooth obstacle avoidance path by using the A* algorithm and the Bessel curve based on the unmanned vehicle positioning information in the TDF image, so as to improve the positioning accuracy of the unmanned vehicle in a complex environment and the accuracy and reliability of the navigation path.

[0082] Those skilled in the art will understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0083] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The device that implements the functions specified in one or more flows and / or blocks.

[0084] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 The functions of a flow or multiple flows and / or a block or multiple blocks in conjunction with the disclosed aspects can be implemented on practitioners' computers in an interactive mode or in a batch mode. Figure 1

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow Figure 1 The functions of a flow or multiple flows and / or a block or multiple blocks in conjunction with the disclosed aspects can be implemented on practitioners' computers in an interactive mode or in a batch mode. Figure 1

[0086] The above detailed description merely describes specific embodiments of the application, and the purpose of the above detailed description is to further illustrate the purpose, technical solutions and beneficial effects of the application. It should be understood that the above detailed description is only a specific embodiment of the application and is not intended to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application should be included in the protection scope of the application.​​

Claims

1. A method for unmanned vehicle path planning based on UAV orthophotos, characterized in that, include: The process involves acquiring orthophotos taken by a drone, performing semantic segmentation on the orthophotos to obtain semantic segmentation results, and labeling the object categories identified in the semantic segmentation results to obtain semantic labels for each object category. The object categories include road areas, buildings, vegetation, houses, vehicles, and pedestrians. Extract the pixel region corresponding to each semantic label to generate the corresponding semantic category image, and perform TDF truncation Euclidean distance field transformation on each semantic category image to obtain the corresponding TDF image. Acquire 3D point cloud data from the LiDAR sensor of the unmanned vehicle, use a particle filter algorithm to map the 3D point cloud data of the unmanned vehicle to the coordinate system of the TDF image of the UAV, obtain the corresponding unmanned vehicle positioning information from the UAV TDF image, and obtain obstacle positioning information based on the unmanned vehicle positioning information. Based on the autonomous vehicle's localization information and obstacle localization information, the A* algorithm and Bessel curve are used to plan the path of the autonomous vehicle. The particle filter algorithm is used to map the 3D point cloud data of the unmanned vehicle to the coordinate system of the UAV TDF image, including: Based on particle state and scale parameters, the 3D point cloud data of the LiDAR of the unmanned vehicle is mapped to the TDF image of the UAV. Based on particle state and scale parameters, the 3D point cloud data of the autonomous vehicle's LiDAR is mapped to the TDF image of the UAV, including: The following mapping formula is used to map the 3D point cloud data of the autonomous vehicle's LiDAR to the TDF image of the drone: in, The spatial coordinates of the particle state. t The translation amount, This represents the rotation angle used for coordinate transformation of the autonomous vehicle relative to the TDF image coordinate system. The spatial coordinates of the 3D point cloud data of the LiDAR for autonomous vehicles. For scale parameters, These are the pixel coordinates of the mapped TDF image.

2. The method as described in claim 1, characterized in that, Also includes: Based on the semantic segmentation results and the object categories of the UAV orthophoto, the orthophoto is binarized to generate a binary map of the drivable area: road areas are marked as drivable areas, and non-road areas are marked as non-drivable areas.

3. The method as described in claim 2, characterized in that, Based on the autonomous vehicle's localization information and obstacle localization information, path planning for the autonomous vehicle is performed using the A* algorithm and Bessel curves, including: Within the drivable area, the autonomous vehicle's path planning is performed using the A* algorithm and Bessel curves, based on the vehicle's localization information and obstacle localization information.

4. A path planning device for unmanned vehicles based on orthophotos from unmanned aerial vehicles, characterized in that, include: The UAV orthophoto semantic segmentation module is used to acquire orthophotos taken by the UAV, perform semantic segmentation processing on the orthophotos to obtain semantic segmentation results, and label the object categories identified in the semantic segmentation results to obtain semantic labels for each object category; the object categories include road areas, buildings, vegetation, houses, vehicles and pedestrians. The TDF image transformation module is used to extract the pixel region corresponding to each semantic label, generate the corresponding semantic category image, and perform TDF truncated Euclidean distance field transformation on each semantic category image to obtain the corresponding TDF image. The unmanned vehicle positioning information acquisition module is used to acquire 3D point cloud data from the unmanned vehicle's lidar sensor. It uses a particle filter algorithm to map the unmanned vehicle's 3D point cloud data to the coordinate system of the UAV's TDF image, obtains the corresponding unmanned vehicle positioning information from the UAV's TDF image, and obtains obstacle positioning information based on the unmanned vehicle positioning information. The unmanned vehicle path planning module is used to plan the path of the unmanned vehicle based on the unmanned vehicle's location information and obstacle location information, using the A* algorithm and Bessel curves. The autonomous vehicle location information acquisition module is also used for: Based on particle state and scale parameters, the 3D point cloud data of the LiDAR of the unmanned vehicle is mapped to the TDF image of the UAV. The autonomous vehicle location information acquisition module is specifically used for: The following mapping formula is used to map the 3D point cloud data of the autonomous vehicle's LiDAR to the TDF image of the drone: in, The spatial coordinates of the particle state. t The translation amount, This represents the rotation angle used for coordinate transformation of the autonomous vehicle relative to the TDF image coordinate system. The spatial coordinates of the 3D point cloud data of the LiDAR for autonomous vehicles. For scale parameters, These are the pixel coordinates of the mapped TDF image.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 3.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 3.

Images