Methods, systems, storage media, and products for unmanned vehicle path planning in field environments in collaboration with drones

Abstract

This invention discloses a method, system, storage medium, and product for unmanned vehicle (UAV) path planning in field environments under UAV collaboration, belonging to the field of UAV path planning, and solving the problem of low data accuracy in the global path planning of UAVs in existing technologies. This invention acquires a top-down visible light image of the passable area and performs image pose calculation and sparse point cloud data generation; then generates a two-dimensional orthophoto and digital surface model of the passable area; a semantic segmentation model performs semantic segmentation on the two-dimensional orthophoto and extracts ground feature classification data; after gridding the digital surface model and ground feature classification data, slope values ​​and ground feature classification values ​​are calculated; based on the slope value and ground feature classification value of each grid cell, the passage cost of each grid cell is calculated; the starting and ending coordinates of the UAV are input, and UAV path planning is performed based on the path planning algorithm and the obtained passage cost map. This invention is used for UAV-collaborated path planning in field environments.

Description

Technical Field

[0001] A method, system, storage medium, and product for unmanned vehicle path planning in field environments under UAV collaboration are disclosed, which are used for unmanned vehicle path planning in field environments under UAV collaboration and belong to the field of unmanned vehicle path planning. Background Technology

[0002] Autonomous vehicles are playing an increasingly important role in emergency rescue, disaster assessment, material transportation, and medical assistance due to their rapid response and high reliability. They can conduct search and rescue operations in dangerous environments, assess disaster situations, and even assist firefighting efforts, greatly improving rescue efficiency and reducing the risk of casualties. However, in complex and ever-changing wilderness environments, path planning for autonomous vehicles still faces many challenges: for example, how to quickly plan the optimal route on damaged roads after a disaster, how to avoid collisions at congested rescue sites, and how to maintain navigation accuracy under conditions of limited communication. These are all problems that urgently need to be solved, requiring more intelligent and robust algorithms and more reliable sensor technologies.

[0003] Existing autonomous vehicle path planning algorithms can be broadly classified into two categories: global path planning and local path planning algorithms. Common global path planning algorithms include A* and Dijkstra, which are used to plan the optimal path from the starting point to the destination when the complete map information (static environment) is known, and are usually completed before the vehicle departs. Common local path planning algorithms include Dynamic Window (DWA) and reinforcement learning, which are used to plan the vehicle's driving path in real time based on the local environmental information obtained by sensors when the map information is unknown or partially known (dynamic environment), and are usually completed during the vehicle's driving process.

[0004] When conducting global path planning for unmanned vehicles over a large area, satellite imagery can typically be used for ground feature classification, combined with digital elevation model data to calculate the passage cost of unmanned vehicles and obtain feasible and optimal unmanned vehicle paths. However, this approach suffers from low data accuracy and low global planning efficiency, resulting in low path accuracy and untimely path planning. When using onboard sensors for environmental perception and autonomous planning, the limited observation distance makes it difficult to achieve global optimization.

[0005] As can be seen from the above, existing autonomous vehicle path planning methods have the following technical problems: 1. Using satellite imagery for land feature classification and combining it with digital elevation model data to calculate the passage cost of unmanned vehicles for global path planning results in low data accuracy, which can lead to problems such as low global planning efficiency, low path accuracy, and poor real-time performance of path planning. 2. When using onboard sensors for environmental perception and local path planning in autonomous vehicles, conventional onboard sensors have limited observation range, making it difficult to achieve global optimization. Summary of the Invention

[0006] The purpose of this invention is to provide a method, system, storage medium, and product for unmanned vehicle path planning in field environments under the collaboration of unmanned aerial vehicles (UAVs), which solves the problems of low data accuracy, low global planning efficiency, low path accuracy, and poor real-time performance of existing technologies for global path planning of unmanned vehicles; or the problem that conventional vehicle-mounted sensors have limited observation distance and are difficult to achieve global optimization when using unmanned vehicle onboard sensors for environmental perception and local path planning.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for unmanned vehicle path planning in a field environment in collaboration with drones includes the following steps: Step 1: Use a drone equipped with a visible light payload to cruise and photograph the area where the unmanned vehicle travels, obtain an overhead visible light image of the area, and transmit the overhead visible light image back to the ground station in real time via a wireless link; Step 2: The ground station receives the overhead visible light image transmitted back by the UAV, performs image pose calculation on the overhead visible light image, and generates sparse point cloud data in real time using an incremental sparse reconstruction algorithm; Step 3: Based on the image pose obtained in Step 2, the incremental map stitching method is used to continuously stitch the overhead visible light image transmitted by the UAV to generate a two-dimensional orthophoto of the passable area. Simultaneously, a digital surface model is generated based on the sparse point cloud data obtained in step 2 and a fast dense reconstruction algorithm, and incremental updates are performed to obtain the final digital surface model; Step 4: Use a semantic segmentation model to perform semantic segmentation on the two-dimensional orthophoto of the passage area in Step 3, and extract land cover classification data; Step 5: Based on the size parameters of the unmanned vehicle, establish a grid for the passage area. Based on the grid for the passage area, perform gridding processing on the digital surface model obtained in Step 3 and the land feature classification data obtained in Step 4 to obtain grid cells. Calculate the slope value and land feature classification value of the grid cells. Establish a comprehensive traffic cost model for unmanned vehicles, calculate the traffic cost of each grid cell based on the slope value and land feature classification value of the grid cell, and generate a traffic cost map for unmanned vehicles. Step 6: Input the coordinates of the starting point and ending point of the autonomous vehicle, and perform path planning for the autonomous vehicle based on the path planning algorithm and the toll cost map obtained in Step 5.

[0008] Furthermore, the specific steps of step 3 are as follows: Step 3.1: Based on the pose parameters of the first top-view visible light image obtained in Step 2, perform geometric correction and projection transformation on the top-view visible light image according to a unified spatial reference system to generate the corresponding local two-dimensional orthophoto, and use it as an existing two-dimensional orthophoto. Meanwhile, based on the sparse point cloud data of the first overhead visible light image obtained in step 2, a fast dense reconstruction algorithm is used to generate the corresponding local three-dimensional point cloud and generate a digital surface model. Step 3.2: Based on the image pose parameters obtained in Step 2, the top-view visible light image of the new ungenerated image area is geometrically corrected and projected according to a unified spatial reference system to generate a corresponding new local two-dimensional orthophoto. Meanwhile, based on the sparse point cloud data of the top-view visible light image of the ungenerated image area obtained in step 2, the region is divided, and a fast dense reconstruction algorithm is used to generate the corresponding new local three-dimensional point cloud. Step 3.3: Register and stitch the newly generated local 2D orthophoto with the existing 2D orthophoto. That is, in the overlapping area of ​​the images, according to the image pose relationship and spatial consistency constraints, the stitching boundary is smoothly fused to form a continuous 2D map result without obvious seams. If the incremental update does not reach the iteration end condition, the 2D map result is used as the new existing 2D orthophoto, and step 3.2 is executed again. Otherwise, the final 2D map result is obtained, that is, the 2D orthophoto of the passable area is generated. Meanwhile, the digital surface model is updated based on the new local 3D point cloud. If the incremental update does not reach the iteration termination condition, step 3.2 is executed again based on the updated digital surface model; otherwise, the final digital surface model is obtained.

[0009] Furthermore, the semantic segmentation model in step 4 includes a multi-scale image pyramid input layer, an encoder, and a decoder connected in sequence; The multi-scale image pyramid input layer constructs a scale space coding set for the multi-scale image pyramid from the two-dimensional orthophoto of the input passage area using the Laplacian pyramid, and then performs multi-scale high-order pyramid iteration based on the scale space coding set to obtain the multi-scale image. The formula for the scale space coding set of the multi-scale image pyramid is: In the formula, It is the first of the Laplace Pyramids The scale of the layer , For scale parameters Downsampled images, For fine scale, For a coarser scale, Indicates low-pass residual, It is the first Gradient magnitude of the layer Indicates the first Layer-scale layer coding, Indicates the total number of floors; Multi-scale higher-order pyramids are iteratively defined by the following encoding set: in, It is a first-order coding set, mainly describing the basic scale structure of the image. For the first The multi-scale high-order pyramid encoding set represents the high-order multi-scale feature geometry obtained through recursive encoding. The encoding set consists of encoding... Logarithmic mapping of gradient magnitude at each scale Together constitute This indicates the position of the pixel at the current scale within the previous scale. Based on multi-scale high-order pyramids, pyramid images of arbitrary scales are reconstructed to obtain multi-scale images; The formula for reconstructing pyramid images at any scale is: in, This represents the pyramid image obtained by direct reconstruction based on the first-order coding set. Indicates the use of the first The image obtained by inverse transformation of the 1st order encoding set. For first-order coding set The first in Each encoded component For higher-order coding sets The coded components corresponding to the scale in the middle, For first-order coding set The first in One coded component; The encoder adopts a deep convolutional neural network structure with dilated convolutions to extract features from multi-scale images input to the multi-scale image pyramid input layer. The encoder first extracts basic features through a deep convolutional neural network, and then introduces a dilated spatial pyramid pooling module to expand the receptive field and obtain contextual information of multi-scale feature representation. The dilated spatial pyramid pooling module is an ASPP module. The ASPP module processes multi-scale images in parallel using 1×1 convolutions, 3×3 dilated convolutions with a dilation rate of 6, 3×3 dilated convolutions with a dilation rate of 12, 3×3 dilated convolutions with a dilation rate of 18, and global average pooling layers. After feature concatenation of the outputs of each branch, the images are further fused using 1×1 convolutions to form multi-scale feature representations. The decoder performs a 1×1 convolution on the low-level features retained in the encoder to reduce the channel dimension, upsamples the multi-scale features output by the encoder by 4 times, and concatenates them with the processed low-level features. The concatenated features are then convolved by 3×3, and finally upsampled again by 4 times to restore the original image resolution, thereby obtaining the final feature representation result.

[0010] Furthermore, the specific steps of step 4 are as follows: Step 4.1: Construct a semantic segmentation model for UAV images; Step 4.2: Based on the semantic segmentation model, perform semantic segmentation on the two-dimensional orthophoto of the passable area obtained in Step 3, and extract the land feature classification data from the overhead visible light image, including roads, buildings, vehicles, surface vegetation and water bodies.

[0011] Furthermore, the specific steps of step 5 are as follows: Step 5.1: Set the grid size based on the unmanned vehicle's dimensions, and establish a passage area grid according to the grid size. Perform gridding processing on the digital surface model obtained in Step 3 and the land cover classification data obtained in Step 4, specifically as follows: When performing gridding on a digital surface model, the digital surface model within the grid cells is first resampled to obtain a digital surface model with the same grid size. Then, the slope is calculated based on the resampled digital surface model to obtain slope information with the same grid size. When performing gridding on land cover classification data, if all land cover classification pixel values ​​are consistent within each grid cell, the grid cell value is adopted as the pixel value of that land cover type. If the corresponding land cover classification pixel values ​​are inconsistent within each grid cell, the pixel value with the highest frequency is adopted as the grid cell value, thus obtaining land cover types consistent with the grid size. Step 5.2: Establish a comprehensive traffic cost model for unmanned vehicles. Integrate the land cover type and slope information obtained in Step 5.1 to obtain the comprehensive traffic cost. Reassign values ​​to the grid to obtain the traffic cost map for unmanned vehicles. Specifically: To address the impact of slope information on the passage cost of autonomous vehicles, a slope cost function is established: in, The slope safety threshold is defined as follows: within this threshold, the autonomous vehicle can drive normally; beyond it, the cost of passage becomes infinitely high. Points in a grid cell The slope value, i.e., slope information. This is the slope weighting coefficient, which is usually an empirical value. When the slope is 0, the cost of passage is the lowest; as the slope increases, the cost of passage increases. To address the impact of terrain feature type on autonomous vehicle traffic, a terrain feature type cost function is established: in, Category of land features The passage cost is established based on the semantic classification of geographical features; The autonomous vehicle comprehensive traffic cost model is used to represent the difficulty of an autonomous vehicle passing through each grid cell. The formula is: in, Points in a grid cell The overall cost of passage, Points in a grid cell The cost of land cover type Points in a grid cell The cost of passage due to the slope, This is a weighting coefficient used to balance the impact of land cover type and slope on access costs.

[0012] A path planning system for unmanned vehicles in a field environment under the cooperation of unmanned aerial vehicles (UAVs) includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the path planning method for unmanned vehicles in a field environment under the cooperation of unmanned aerial vehicles (UAVs).

[0013] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for unmanned vehicle path planning in a field environment in collaboration with unmanned aerial vehicles.

[0014] A computer program product includes a computer program that, when executed by a processor, implements the steps of a method for unmanned vehicle path planning in a field environment in collaboration with unmanned aerial vehicles.

[0015] Compared with the prior art, the advantages of the present invention are as follows: This invention utilizes unmanned aerial vehicles (UAVs) to assist unmanned vehicles (UAVs) in path planning and employs a real-time data transmission and processing strategy to improve environmental perception and path planning efficiency. Simultaneously, based on grid-organized UAV data, a UAV passage cost model is established to achieve unmanned path planning in roadless environments. Its advantages are specifically reflected as follows: First, this invention uses UAVs as a data collection method to provide data for UAV path planning. Compared to relying on sensors mounted on the UAV itself to obtain environmental data, it offers a wider field of view and can achieve optimal planning results over a larger area. Second, this invention utilizes UAVs to acquire visible light image data, which has a higher resolution than satellite remote sensing image data, and can provide more refined geometric information for path planning of unmanned vehicles in the field. Third, the present invention adopts a strategy of real-time image transmission from UAVs and real-time construction of digital orthophotos and digital surface models, which can effectively improve the timeliness of unmanned vehicle path planning under rapidly changing environmental conditions such as disasters. Fourth, the path planning method of the present invention does not rely on on-board visible light, lidar or other sensors to obtain environmental information, and can be applied to unmanned vehicles that do not carry any sensors, effectively expanding its application scope. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a detailed flowchart of the present invention; Figure 2 This is a schematic diagram of the multi-scale image pyramid input layer in the semantic segmentation model of this invention; Figure 3 This is a schematic diagram of the encoder and decoder in the semantic segmentation model of this invention; Figure 4 This is a reference value for the passage cost corresponding to the land feature type in this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] A method for unmanned vehicle path planning in a field environment in collaboration with drones includes the following steps: Step 1: Use a drone equipped with a visible light payload to cruise and photograph the area where the unmanned vehicle travels, obtain an overhead visible light image of the area, and transmit the overhead visible light image back to the ground station in real time via a wireless link; Step 2: The ground station receives the overhead visible light images transmitted back by the UAV, performs image pose calculation on the overhead visible light images, realizes the spatial geometric constraint recovery between the overhead visible light images, and generates sparse point cloud data in real time by combining incremental sparse reconstruction algorithm; The specific steps are as follows: Step 2.1: Based on the classic collinearity equation in photogrammetry, establish the spatial geometric constraint relationship between UAV image points, spatial points and camera projection center. By solving the collinearity equation, obtain the camera pose parameters of each top-view visible light image in a unified spatial coordinate system, that is, obtain the image pose solution and realize the recovery of spatial geometric constraints between images. Step 2.2: Under the solved image pose constraints, the corresponding feature points obtained by SIFT feature extraction, feature description matching and geometric consistency constraint screening of the top-view visible light image are used to perform spatial forward intersection calculation to generate corresponding three-dimensional spatial points, and the three-dimensional spatial points are gradually added to the sparse point cloud set; as new top-view visible light images are continuously added, the sparse point cloud data is incrementally updated.

[0020] Step 3: Based on the image pose obtained in Step 2, the incremental map stitching method is used to continuously stitch the overhead visible light image transmitted by the UAV to generate a two-dimensional orthophoto of the passable area. Simultaneously, a digital surface model is generated based on the sparse point cloud data obtained in step 2 and a fast dense reconstruction algorithm, and incremental updates are performed to obtain the final digital surface model; The specific steps are as follows: Step 3.1: Based on the pose parameters of the first top-view visible light image obtained in Step 2, perform geometric correction and projection transformation on the top-view visible light image according to a unified spatial reference system to generate the corresponding local two-dimensional orthophoto (DOM), and use it as an existing two-dimensional orthophoto. Meanwhile, based on the sparse point cloud data of the first overhead visible light image obtained in step 2, a fast dense reconstruction algorithm is used to generate the corresponding local 3D point cloud and generate a digital surface model (DSM). Step 3.2: Based on the image pose parameters obtained in Step 2, the top-view visible light image of the new ungenerated image area is geometrically corrected and projected according to a unified spatial reference system to generate a corresponding new local two-dimensional orthophoto. Meanwhile, based on the sparse point cloud data of the top-view visible light image of the ungenerated image area obtained in step 2, the region is divided, and a fast dense reconstruction algorithm is used to generate the corresponding new local three-dimensional point cloud. Step 3.3: Register and stitch the newly generated local 2D orthophoto (as the UAV continuously transmits new overhead visible light images) with the existing 2D orthophoto to continuously expand the image. That is, in the overlapping area of ​​the images, according to the image pose relationship and spatial consistency constraints, the stitching boundary is smoothly fused (that is, the corresponding area is locally replaced or weighted and updated without re-stitching the entire map, so as to realize the incremental update of the map), forming a continuous 2D map result without obvious seams. If the incremental update does not reach the iteration end condition, the 2D map result is used as the new existing 2D orthophoto, and step 3.2 is executed again. Otherwise, the final 2D map result is obtained, that is, the 2D orthophoto of the passable area is generated. Simultaneously, the digital surface model is updated based on the new local 3D point cloud, that is, the new local 3D point cloud is merged with the existing digital surface model, thereby realizing the incremental construction of the digital surface model of the passable area. If the incremental update does not reach the iteration end condition, step 3.2 is executed again based on the updated digital surface model; otherwise, the final digital surface model (such as roads, grasslands, buildings, water bodies, etc.) is obtained.

[0021] Step 4: Use a semantic segmentation model to perform semantic segmentation on the two-dimensional orthophoto of the passage area in Step 3, and extract land cover classification data; The semantic segmentation model for UAV images used in this invention includes a multi-scale fusion encoder-decoder, i.e., the encoder of the deep neural network model. Its main body is a DCNN with dilated convolutions, which can use a commonly used classification network such as ResNet. Then there is an Atrous Spatial Pyramid Pooling (ASPP) module with dilated convolutions, mainly to introduce multi-scale information. Compared with the classic DeepLabv3 model, this model introduces a decoder module, which further fuses low-level features with high-level features to improve the accuracy of segmentation boundaries.

[0022] To adapt to multi-scale image and feature inputs, this invention employs a multi-scale image pyramid construction method and embeds it into a multi-scale fusion encoder-decoder deep neural network model framework. When the framework loads images, it adaptively learns and selects stacked features at each scale. The multi-scale image pyramid construction reduces scale variation loss by introducing higher-order pyramid representations.

[0023] The semantic segmentation model consists of a multi-scale image pyramid input layer, an encoder, and a decoder connected sequentially. The multi-scale image pyramid input layer constructs a scale space coding set for the multi-scale image pyramid from the two-dimensional orthophoto of the input passage area using the Laplacian pyramid, and then performs multi-scale high-order pyramid iteration based on the scale space coding set to obtain the multi-scale image. The formula for the scale space coding set of the multi-scale image pyramid is: In the formula, It is the first of the Laplace Pyramids The scale of the layer , For scale parameters Downsampled images, For fine scale, For a coarser scale, Indicates low-pass residual, It is the first Gradient magnitude of the layer Indicates the first Layer-scale layer coding, Indicates the total number of floors; Multi-scale higher-order pyramids are iteratively defined by the following encoding set: in, It is a first-order coding set, mainly describing the basic scale structure of the image. For the first The multi-scale high-order pyramid encoding set represents the high-order multi-scale feature geometry obtained through recursive encoding. The encoding set consists of encoding... Logarithmic mapping of gradient magnitude at each scale Together constitute This indicates the position of the pixel at the current scale within the previous scale. Based on multi-scale high-order pyramids, pyramid images of arbitrary scales are reconstructed to obtain multi-scale images; The formula for reconstructing pyramid images at any scale is: in, This represents the pyramid image obtained by direct reconstruction based on the first-order coding set. Indicates the use of the first The image obtained by inverse transformation of the 1st order encoding set. For first-order coding set The first in Each encoded component For higher-order coding sets The coded components corresponding to the scale in the middle, For first-order coding set The first in One coded component; The encoder adopts a deep convolutional neural network structure with dilated convolutions to extract features from multi-scale images input to the multi-scale image pyramid input layer. The encoder first extracts basic features through a deep convolutional neural network, and then introduces a dilated spatial pyramid pooling module to expand the receptive field and obtain contextual information of multi-scale feature representation. The dilated spatial pyramid pooling module is an ASPP module. The ASPP module processes multi-scale images in parallel using 1×1 convolutions, 3×3 dilated convolutions with a dilation rate of 6, 3×3 dilated convolutions with a dilation rate of 12, 3×3 dilated convolutions with a dilation rate of 18, and global average pooling layers. After feature concatenation of the outputs of each branch, the images are further fused using 1×1 convolutions to form multi-scale feature representations. The decoder performs a 1×1 convolution on the low-level features retained in the encoder to reduce the channel dimension, upsamples the multi-scale features output by the encoder by 4 times, and concatenates them with the processed low-level features. The concatenated features are then convolved by 3×3, and finally upsampled again by 4 times to restore the original image resolution, thereby obtaining the final feature representation result.

[0024] The specific steps are as follows: Step 4.1: Construct a semantic segmentation model for the drone images; Step 4.2: Based on the semantic segmentation model, perform semantic segmentation on the two-dimensional orthophoto of the passable area obtained in Step 3, and extract the land feature classification data in the overhead visible light image, including land feature classification information such as roads, buildings, vehicles, surface vegetation and water bodies, other shading objects and background.

[0025] Step 5: Based on the size parameters of the unmanned vehicle, establish a grid for the passage area. Based on the grid for the passage area, perform gridding processing on the digital surface model obtained in Step 3 and the land feature classification data obtained in Step 4 to obtain grid cells. Calculate the slope value and land feature classification value of the grid cells. Establish a comprehensive traffic cost model for unmanned vehicles, calculate the traffic cost of each grid cell based on the slope value and land feature classification value of the grid cell, and generate a traffic cost map for unmanned vehicles. The specific steps are as follows: Step 5.1: Set the grid size according to the unmanned vehicle size parameters (e.g., each grid cell is 5 meters × 5 meters), and establish a passage area grid according to the grid size. Perform gridding processing on the digital surface model obtained in Step 3 and the land cover classification data obtained in Step 4, specifically as follows: DSM data constructed from overhead, wind- and light-reflecting images obtained by UAVs typically have high spatial resolution (centimeter-level). Therefore, when performing gridding on the digital surface model, the digital surface model within the grid cells is first resampled to obtain a digital surface model with the same grid size. Then, slope calculation is performed based on the resampled digital surface model to obtain slope information consistent with the grid size. Land cover classification data share the same challenge: mapping from high-resolution data to lower-resolution grid cells. Therefore, when gridding land cover classification data, if all corresponding land cover class pixel values ​​within a grid cell are identical, the grid cell value is set to that class of land cover pixel value. If the corresponding land cover class pixel values ​​within different grid cells are inconsistent, the pixel value with the highest frequency is used as the grid cell value, resulting in land cover types consistent with the grid size. Step 5.2: Establish a comprehensive traffic cost model for unmanned vehicles. Integrate the land cover type and slope information obtained in Step 5.1 to obtain the comprehensive traffic cost. Reassign values ​​to the grid to obtain the traffic cost map for unmanned vehicles. Specifically: To address the impact of slope information on the passage cost of autonomous vehicles, a slope cost function is established: in, The slope safety threshold is defined as follows: within this threshold, the autonomous vehicle can drive normally; beyond it, the cost of passage becomes infinitely high. Points in a grid cell The slope value, i.e., slope information. This is the slope weighting coefficient, which is usually an empirical value. When the slope is 0, the cost of passage is the lowest; as the slope increases, the cost of passage increases. To address the impact of terrain feature type on autonomous vehicle traffic, a terrain feature type cost function is established: in, Category of land features The passage cost is established based on the semantic classification of geographical features; like Figure 4 The figure shows the reference values ​​for the access costs corresponding to different land cover types, which can be expanded based on the semantic segmentation categories.

[0026] The autonomous vehicle comprehensive traffic cost model is used to represent the difficulty of an autonomous vehicle passing through each grid cell. The formula is: in, Points in a grid cell The overall cost of passage, Points in a grid cell The cost of land cover type Points in a grid cell The cost of passage due to the slope, The weighting coefficients are used to balance the impact of terrain features and slope on passage costs, and can be adjusted according to the actual performance of the autonomous vehicle and the requirements of the passage mission. Specific weights are set based on actual needs and scene characteristics, aiming to reflect the greater difficulty of passage in areas with higher slopes. Combining semantic and slope information, each grid cell is comprehensively evaluated, and a final passage cost is assigned.

[0027] Step 6: Input the coordinates of the autonomous vehicle's starting point and destination. Based on the path planning algorithm and the traffic cost map obtained in Step 5, perform path planning for the autonomous vehicle. Specifically: Based on the traffic cost map obtained in Step 5, input the coordinates of the autonomous vehicle's starting point and destination. Convert these coordinates into grid index values ​​within the travel area. Use the path planning algorithm to perform optimal cost search on the grid, select the set of grid cells with the lowest overall traffic cost, and then convert them into latitude and longitude coordinates to obtain the optimal path for the autonomous vehicle within the area.

[0028] In summary, this invention utilizes UAVs to collect high-precision, real-time visible light images of the passable area. Compared to using satellites to acquire image data of the passable area, this provides autonomous vehicles with higher precision and more real-time environmental information, better supporting autonomous vehicle path planning. The invention employs a strategy of real-time UAV image transmission and real-time construction of digital orthophotos and digital surface models, which effectively improves the efficiency of environmental information acquisition and is more conducive to path planning tasks for autonomous vehicles in different application scenarios. Combined with digital surface models for global path planning, the invention achieves high data accuracy, high global planning efficiency, high path accuracy, and excellent real-time path planning performance. Furthermore, the path planning method of this invention does not rely on onboard visible light or lidar sensors to acquire environmental information, and can be applied to autonomous vehicles without any sensors, effectively expanding its application scope and achieving global optimization.

Claims

1. A method for unmanned vehicle path planning in a field environment in collaboration with unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Step 1: Use a drone equipped with a visible light payload to cruise and photograph the area where the unmanned vehicle travels, obtain an overhead visible light image of the area, and transmit the overhead visible light image back to the ground station in real time via a wireless link; Step 2: The ground station receives the overhead visible light image transmitted back by the UAV, performs image pose calculation on the overhead visible light image, and generates sparse point cloud data in real time using an incremental sparse reconstruction algorithm; Step 3: Based on the image pose obtained in Step 2, the incremental map stitching method is used to continuously stitch the overhead visible light images transmitted by the UAV to generate a two-dimensional orthophoto of the passable area. Simultaneously, a digital surface model is generated based on the sparse point cloud data obtained in step 2 and a fast dense reconstruction algorithm, and incremental updates are performed to obtain the final digital surface model; Step 4: Use a semantic segmentation model to perform semantic segmentation on the two-dimensional orthophoto of the passage area in Step 3, and extract land cover classification data; Step 5: Based on the size parameters of the unmanned vehicle, establish a grid for the passage area. Based on the grid for the passage area, perform gridding processing on the digital surface model obtained in Step 3 and the land feature classification data obtained in Step 4 to obtain grid cells. Calculate the slope value and land feature classification value of the grid cells. Establish a comprehensive traffic cost model for unmanned vehicles, calculate the traffic cost of each grid cell based on the slope value and land feature classification value of the grid cell, and generate a traffic cost map for unmanned vehicles. Step 6: Input the coordinates of the starting point and ending point of the autonomous vehicle, and perform path planning for the autonomous vehicle based on the path planning algorithm and the toll cost map obtained in Step 5.

2. The method for unmanned vehicle path planning in a field environment under UAV collaboration as described in claim 1, characterized in that, The specific steps of step 3 are as follows: Step 3.1: Based on the pose parameters of the first top-view visible light image obtained in Step 2, perform geometric correction and projection transformation on the top-view visible light image according to a unified spatial reference system to generate the corresponding local two-dimensional orthophoto, and use it as an existing two-dimensional orthophoto. Meanwhile, based on the sparse point cloud data of the first overhead visible light image obtained in step 2, a fast dense reconstruction algorithm is used to generate the corresponding local three-dimensional point cloud and generate a digital surface model. Step 3.2: Based on the image pose parameters obtained in Step 2, perform geometric correction and projection transformation on the top-view visible light image of the new ungenerated image area according to a unified spatial reference system to generate the corresponding new local two-dimensional orthophoto. Meanwhile, based on the sparse point cloud data of the top-view visible light image of the ungenerated image area obtained in step 2, the region is divided, and a fast dense reconstruction algorithm is used to generate the corresponding new local three-dimensional point cloud. Step 3.3: Register and stitch the newly generated local 2D orthophoto with the existing 2D orthophoto. That is, in the overlapping area of ​​the images, according to the image pose relationship and spatial consistency constraints, the stitching boundary is smoothly fused to form a continuous 2D map result without obvious seams. If the incremental update does not reach the iteration end condition, the 2D map result is used as the new existing 2D orthophoto, and step 3.2 is executed again. Otherwise, the final 2D map result is obtained, that is, the 2D orthophoto of the passable area is generated. Meanwhile, the digital surface model is updated based on the new local 3D point cloud. If the incremental update does not reach the iteration termination condition, step 3.2 is executed again based on the updated digital surface model; otherwise, the final digital surface model is obtained.

3. A method for unmanned vehicle path planning in a field environment under UAV collaboration as described in claim 1 or 2, characterized in that, The semantic segmentation model in step 4 includes a multi-scale image pyramid input layer, an encoder, and a decoder connected in sequence. The multi-scale image pyramid input layer constructs a scale space coding set for the multi-scale image pyramid from the two-dimensional orthophoto of the input passage area using the Laplacian pyramid, and then performs multi-scale high-order pyramid iteration based on the scale space coding set to obtain the multi-scale image. The formula for the scale space coding set of the multi-scale image pyramid is: In the formula, It is the first of the Laplace Pyramids The scale of the layer , For scale parameters Downsampled images, For fine scale, For a coarser scale, Indicates low-pass residual, It is the first Gradient magnitude of the layer Indicates the first Layer-scale layer coding, Indicates the total number of floors; Multi-scale higher-order pyramids are iteratively defined by the following encoding set: in, It is a first-order coding set, mainly describing the basic scale structure of the image. For the first The multi-scale high-order pyramid encoding set represents the high-order multi-scale feature geometry obtained through recursive encoding. The encoding set consists of encoding... Logarithmic mapping of gradient magnitude at each scale Together constitute This indicates the position of the pixel at the current scale in the previous scale. Based on multi-scale high-order pyramids, pyramid images of arbitrary scales are reconstructed to obtain multi-scale images; The formula for reconstructing pyramid images at any scale is: in, This represents the pyramid image obtained by direct reconstruction based on the first-order coding set. Indicates the use of the first The image obtained by inverse transformation of the 1st order encoding set. For first-order coding set The first in Each encoded component For higher-order coding sets The coded components corresponding to the scale in the middle, For first-order coding set The first in One coded component; The encoder adopts a deep convolutional neural network structure with dilated convolutions to extract features from multi-scale images input to the multi-scale image pyramid input layer. The encoder first extracts basic features through a deep convolutional neural network, and then introduces a dilated spatial pyramid pooling module to expand the receptive field and obtain contextual information of multi-scale feature representation. The dilated spatial pyramid pooling module is an ASPP module. The ASPP module processes multi-scale images in parallel using 1×1 convolutions, 3×3 dilated convolutions with a dilation rate of 6, 3×3 dilated convolutions with a dilation rate of 12, 3×3 dilated convolutions with a dilation rate of 18, and global average pooling layers. After feature concatenation of the outputs of each branch, the images are further fused using 1×1 convolutions to form multi-scale feature representations. The decoder performs a 1×1 convolution on the low-level features retained in the encoder to reduce the channel dimension, upsamples the multi-scale features output by the encoder by 4 times, and concatenates them with the processed low-level features. The concatenated features are then convolved by 3×3, and finally upsampled again by 4 times to restore the original image resolution, thereby obtaining the final feature representation result.

4. The method for unmanned vehicle path planning in a field environment under UAV collaboration as described in claim 3, characterized in that, The specific steps of step 4 are as follows: Step 4.1: Construct a semantic segmentation model for UAV images; Step 4.2: Based on the semantic segmentation model, perform semantic segmentation on the two-dimensional orthophoto of the passable area obtained in Step 3, and extract the land feature classification data from the overhead visible light image, including roads, buildings, vehicles, surface vegetation and water bodies.

5. The method for unmanned vehicle path planning in a field environment under UAV collaboration as described in claim 4, characterized in that, The specific steps of step 5 are as follows: Step 5.1: Set the grid size based on the unmanned vehicle's dimensions, and establish a passage area grid according to the grid size. Perform gridding processing on the digital surface model obtained in Step 3 and the land cover classification data obtained in Step 4, specifically as follows: When performing gridding on a digital surface model, the digital surface model within the grid cells is first resampled to obtain a digital surface model with the same grid size. Then, the slope is calculated based on the resampled digital surface model to obtain slope information with the same grid size. When performing gridding on land cover classification data, if all land cover classification pixel values ​​are consistent within each grid cell, the grid cell value is adopted as the pixel value of that land cover type. If the corresponding land cover classification pixel values ​​are inconsistent within each grid cell, the pixel value with the highest frequency is adopted as the grid cell value, thus obtaining land cover types consistent with the grid size. Step 5.2: Establish a comprehensive traffic cost model for unmanned vehicles. Integrate the land cover type and slope information obtained in Step 5.1 to obtain the comprehensive traffic cost. Reassign values ​​to the grid to obtain the traffic cost map for unmanned vehicles. Specifically: To address the impact of slope information on the passage cost of autonomous vehicles, a slope cost function is established: in, The slope safety threshold is defined as follows: within this threshold, the autonomous vehicle can drive normally; beyond it, the cost of passage becomes infinitely high. Points in a grid cell The slope value, i.e., slope information. This is the slope weighting coefficient, which is usually an empirical value. When the slope is 0, the cost of passage is the lowest; as the slope increases, the cost of passage increases. To address the impact of terrain feature type on autonomous vehicle traffic, a terrain feature type cost function is established: in, Category of land features The passage cost is established based on the semantic classification of geographical features; The autonomous vehicle comprehensive traffic cost model is used to represent the difficulty of an autonomous vehicle passing through each grid cell. The formula is: in, Points in a grid cell The overall cost of passage, Points in a grid cell The cost of land cover type Points in a grid cell The cost of passage due to the slope, This is a weighting coefficient used to balance the impact of land cover type and slope on access costs.

6. A path planning system for unmanned vehicles in a field environment in collaboration with unmanned aerial vehicles (UAVs), comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1-5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-5.

8. A computer program product, comprising a computer program, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-5.

Images