An unmanned aerial vehicle assisted unmanned vehicle global environment exploration route planning method

Abstract

The application discloses a kind of unmanned vehicle global environment exploration route planning method assisted by unmanned aerial vehicle, belong to the field of environment modeling and path planning.It includes: firstly, in outdoor environment, two-dimensional overhead picture of environment is acquired by unmanned aerial vehicle from air with large viewing angle, high mobility, rapidity and low cost mode, and the picture stream is obtained by panoramic splicing based on FREAK feature, obstacle segmentation is carried out based on Unet3+ neural network to obtain environment obstacle binary graph;Secondly, the passable area is extracted for the obstacle binary map, and the improved hierarchical Vino graph road network modeling method is further used to obtain the environment hierarchical passable network topology node graph;Then, the environment hierarchical passable network node graph constructed is converted into Euler graph by minimum weight matching mode;Finally, the exploration path of ground unmanned vehicle global environment is obtained by solving Euler loop for any starting point through improved Fleury algorithm.

Description

Technical Field

[0001] This invention belongs to the field of environmental modeling and path planning, specifically relating to a method for global environmental exploration route planning for unmanned vehicles assisted by drones. Background Technology

[0002] Drone-assisted unmanned vehicle global environment exploration is a new exploration mode based on artificial intelligence technology and unmanned systems. Its main purpose is to quickly and comprehensively detect and analyze environmental information of a specific area through the combination of drones and unmanned vehicles to support scientific research, emergency rescue, military reconnaissance, urban planning and other fields.

[0003] Comprehensive environmental situational awareness requires unmanned platforms to efficiently explore every corner of the environment. Currently, single-vehicle autonomous exploration often relies on the local perception perspective of its own sensor system, following a greedy strategy to explore the unknown environment step by step, which can only guarantee local optimality. Given the disadvantages of single-vehicle systems in environmental exploration, it is essential to propose a UAV-assisted unmanned vehicle global environment exploration route planning method, leveraging the advantages of UAVs' high maneuverability, high field of view, and speed. Summary of the Invention

[0004] To address the aforementioned issues, this invention discloses a method for planning a global environment exploration route for unmanned vehicles (UAVs) assisted by drones. Based on the fact that UAVs can acquire global environmental information from the air in outdoor environments with a wide field of view, high maneuverability, speed, and low cost, this method can provide high-value information gains to ground-based UAV systems. Through further modeling and processing, a global environment exploration route for the UAV can be planned, allowing the UAV to efficiently explore the environment and complete related tasks.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A method for unmanned vehicle-assisted global environment exploration and route planning includes the following steps:

[0007] S1: The drone collects a stream of environmental images from above, following a pre-defined route, either remotely or autonomously.

[0008] S2: The onboard computer receives the photo stream captured by the drone in step S1 and performs photo stitching based on FREAK features to obtain a panoramic overhead image of the environment of interest.

[0009] S3: The panoramic top-down image obtained by stitching in step S3 is used to perform obstacle segmentation to obtain a binary map of environmental obstacles using a neural network based on Unet3+.

[0010] S4: Extract connected components of obstacles, extract boundary vertices and centroids of obstacles, triangulate and pair the centroids of obstacles, and determine the adaptive sampling step size of obstacle boundaries from the binarized environment map obtained in step S3 to obtain a sparse set of obstacle boundary points.

[0011] S5: Input the set of obstacle boundary points obtained in step S4 into the Veno graph generation algorithm, and obtain the environment passable hierarchical network topology node graph by matching, merging, pruning and smoothing the graph nodes.

[0012] S6: Convert the environment node network graph into an Eulerian graph; determine whether the environment passable hierarchical network topology node graph obtained in step S5 is an Eulerian graph. If not, proceed to S7; otherwise, proceed to S8.

[0013] S7: Convert the environmental network node graph obtained in step S5 into an Eulerian graph using an algorithm based on least binary matching;

[0014] S8: Based on the Eulerian graph obtained in step S6, the improved Fleury algorithm is used to obtain the Eulerian circuit for any starting point, thus obtaining the route planned by the autonomous vehicle for exploring the entire environment, and finally returning to the origin.

[0015] Furthermore, step S1 specifically includes the following process:

[0016] (1-1) Set flight path and shooting parameters: Set the drone's flight path and shooting parameters, such as altitude, angle, and focal length, according to the area and purpose to be shot;

[0017] (1-2) Conduct flight missions: The UAV is controlled by remote or automatic means to conduct flight missions according to the preset flight path, and transmits or saves image streams to the ground UAV host in real time.

[0018] Furthermore, step S2 specifically includes the following process:

[0019] (2-1) Read the image stream according to the time label. For each image, use the FREAK algorithm to extract key points and features.

[0020] (2-2) Match key points and features using algorithms such as FLANN-based feature matchers;

[0021] (2-3) Use the RANSAC-based algorithm to calculate the transformation matrix between adjacent images;

[0022] (2-4) Register adjacent images according to the transformation matrix and stitch the images together;

[0023] (2-5) Optimize and smooth the stitched map to obtain a clearer stitching effect;

[0024] Furthermore, step S3 specifically includes the following process:

[0025] (3-1) Create a dataset by collecting a set of labeled image datasets, where each image contains two parts: buildings and background. Preprocess the images, including resizing, cropping, rotating, flipping, and normalizing, so that they can be input into the Unet3+ network for training.

[0026] (3-2) Construct the Unet3+ network and train it using the prepared dataset so that the network can learn the differences between buildings and the background and perform effective building obstacle segmentation.

[0027] (3-3) Deploy the trained network, input the panoramic map obtained by stitching in step S2, complete the building obstacle segmentation task, and obtain the binary map of environmental obstacles.

[0028] Furthermore, step S4 specifically includes the following process:

[0029] (4-1) Extract the obstacle connected components from the binary map obtained by segmentation in step S3;

[0030] (4-2) Extract the boundary vertices and centroids of each obstacle connected domain in step (4-1);

[0031] (4-3) Triangulate and pair all the obstacle material centers from step (4-2) to construct neighbor relationships;

[0032] (4-4) Based on the obstacle material neighbor relationship constructed in step (4-3), the adaptive sampling step size of each obstacle boundary is calculated and determined;

[0033] (4-5) Based on the adaptive step size of each obstacle determined in step (4-4), perform discrete sampling of the obstacle boundary to obtain the set of boundary sequence points of all obstacles.

[0034] Furthermore, step S5 specifically includes the following process:

[0035] (5-1) Input the obstacle boundary sequence point set obtained in step S4 into the Vinio map generation algorithm to obtain Vinio point pairs, and delete the point pairs that occupy the obstacles;

[0036] (5-2) Based on the Vino point pairs obtained in step (5-1), integrate and pair them to obtain the initial traversable area network node map;

[0037] (5-3) Select nodes with three or more neighbors as hub nodes, obtain road points between hub nodes by searching, establish neighbor relationships between hub nodes, and merge hub nodes that are close in distance.

[0038] (5-4) Smooth the road nodes between hub nodes to obtain the hierarchical network topology node diagram of the environment that is accessible.

[0039] Furthermore, step S6 specifically includes the following process:

[0040] (6-1) Based on the environmental network topology node graph obtained in step S5, determine whether the graph is an Eulerian graph by calculating the degree of each node. If it is, go to step S8; otherwise, go to step S7.

[0041] Furthermore, step S7 specifically includes the following process:

[0042] (7-1) Construct a bipartite graph model based on the set of singularities obtained in step S6;

[0043] (7-2) For the bipartite graph established in step (7-1), establish an integer programming model and find the matching relationship between singularities according to the principle of optimal distance.

[0044] Furthermore, step S8 specifically includes the following process:

[0045] (8-1) Choose any point in the graph as the starting point;

[0046] (8-2) Find its neighboring edge as the next edge to be visited, and give the following priority: ① Unless there are no other alternative edges, the next edge to be visited cannot be a cut edge of the remaining graph; ② For non-cut edge neighboring edges, prioritize the shortest edge as the next edge to be visited.

[0047] (8-3) When step (8-2) cannot continue, the resulting trace is an Eulerian circuit starting from the selected starting point. The unmanned vehicle can efficiently and orderly explore every corner of the environment and eventually return to the starting point by following this circuit.

[0048] The beneficial effects of this invention are:

[0049] This invention proposes a method for planning a global environment exploration route for unmanned vehicles (UAVs) assisted by drones. This method leverages the high maneuverability, high field of view, and rapid acquisition of prior maps by UAVs to plan a global environment exploration route for the UAV. The UAV then efficiently explores the environment and completes related tasks based on this route. Structurally, high-altitude UAVs can be portable, covert, and safely adapted to complex environments, providing more comprehensive two-dimensional environmental perception. This prior environmental perception provides ground-based UAVs with high-value information gains, helping them make more accurate decisions. Methodologically, a hierarchical network topology node graph is obtained by stitching, segmenting, and modeling passable areas from a stream of UAV-viewed images. The UAV can then plan the shortest Eulerian circuit based on this graph and efficiently explore the entire environment by following this path. Attached Figure Description

[0050] Figure 1 A schematic diagram of the method flow of this invention;

[0051] Figure 2 Example of photo stitching results based on FREAK features;

[0052] Figure 3 Example of obstacle segmentation results based on Unet3+ neural network;

[0053] Figure 4 Results of obstacle connected domains and centroid triangulation;

[0054] Figure 5 The set of boundary sequence points of the obstacle after adaptive sampling;

[0055] Figure 6 Example of initial Vinyasa plot results;

[0056] Figure 7 The Vinio diagram after pairing, merging, and pruning;

[0057] Figure 8 The environment smoothed by Bézier curves allows for a permissible hierarchical network topology node diagram.

[0058] Figure 9 Eulerian graph constructed based on least binary matching;

[0059] Figure 10 The paths between singularities need to be repeated. Detailed Implementation

[0060] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0061] Example 1: As Figure 1As shown, a method for global environment exploration and route planning for unmanned vehicles assisted by unmanned aerial vehicles (UAVs) includes the following steps:

[0062] S1: The drone collects a stream of environmental images from above, following a pre-defined route, either remotely or autonomously.

[0063] (1-1) Set flight path and shooting parameters: Set the drone's flight path and shooting parameters, such as altitude, angle, and focal length, according to the area and purpose to be shot;

[0064] (1-2) Conduct flight missions: The UAV is controlled by remote or automatic means to conduct flight missions according to the preset flight path, and transmits or saves image streams to the ground UAV host in real time.

[0065] S2: The onboard computer receives the photo stream captured by the drone in step S1 and performs photo stitching based on FREAK features to obtain a panoramic overhead image of the environment of interest.

[0066] (2-1) Read the image stream according to the time label. For each image, use the FREAK algorithm to extract key points and features.

[0067] (2-2) Keypoints and features are matched using algorithms such as a FLANN-based feature matcher. The nearest neighbor is searched in the feature descriptors of the previous image, and the second nearest neighbor is searched in the feature descriptors of the next image. Then, based on the distance ratio between the nearest and second nearest neighbors, it is determined whether to match the nearest neighbor to the second nearest neighbor. If the distance ratio is less than a certain threshold, the match is established.

[0068] (2-3) Use a RANSAC-based algorithm to calculate the transformation matrix between adjacent images. Among them (H) 13 H 23 ) represents the translation vector, (H) 11 H 12 H 21 H 22 H 31 H 32 ) represents the rotation and scaling parameters, H 33 Indicates the scale parameter.

[0069] (2-4) Register adjacent images according to the transformation matrix and stitch them together; let two adjacent images be I1 and I2, and their transformation matrix be H. Use the following formula to stitch I2 to the right of I1:

[0070]

[0071] Where w1 is the width of I1, and It is the Gaussian weighted function and the Gaussian weighted function of the rest, where (x c ,y c ) represents the center point coordinates, and σ is the standard deviation of the Gaussian function. This formula is actually a weighted fusion of I2, where the weights are calculated using the Gaussian function. For the stitched image I, the projected coordinates corresponding to I1 and I2 are calculated using the following formula:

[0072]

[0073] Where (x, y) are the pixel coordinates in I, (x1', y1', w1') are the projected coordinates corresponding to I1, and (x2', y2', w2') are the projected coordinates corresponding to I2. Concatenating I1 and I2 together yields the concatenated image I.

[0074] (2-5) Optimize and smooth the stitched map to obtain a clearer stitching effect. Figure 2 This is an example of a photo stitching result based on the FREAK feature.

[0075] S3: The panoramic top-down image obtained by stitching in step S3 is used to perform obstacle segmentation to obtain a binary map of environmental obstacles using a neural network based on Unet3+.

[0076] (3-1) Create a dataset by collecting a set of labeled image datasets, where each image contains two parts: buildings and background. Preprocess the images, including resizing, cropping, rotating, flipping, and normalizing, so that they can be input into the Unet3+ network for training.

[0077] (3-2) Construct the Unet3+ network, which consists of an encoder and a decoder. The encoder extracts image features, while the decoder restores the feature maps to their original size. A skip connection mechanism is incorporated into the Unet3+ network to help it learn feature information at different resolutions. The Unet3+ network is trained using a prepared dataset, enabling it to learn the differences between buildings and the background and perform effective building obstacle segmentation.

[0078] (3-3) such as Figure 3 As shown, the trained network is deployed, and the panoramic map obtained by stitching in step S2 is input to complete the building obstacle segmentation task and obtain a binary map of environmental obstacles.

[0079] S4: Extract connected components of obstacles, extract boundary vertices and centroids of obstacles, triangulate and pair the centroids of obstacles, and determine the adaptive sampling step size of obstacle boundaries from the binarized environment map obtained in step S3 to obtain a sparse set of obstacle boundary points.

[0080] (4-1) Extract the obstacle connected components from the binary map obtained by segmentation in step S3;

[0081] (4-2) Extract the boundary of each obstacle in step (4-1) using the boundary tracing algorithm, and calculate the centroid of the connected domain of each obstacle;

[0082] The formula for calculating the centroid is: Where (x) i ,y i ) represents the coordinates of each pixel within the connected component, and n represents the number of pixels within the connected component.

[0083] (4-3) such as Figure 4 As shown, triangulation and pairing are performed on all obstacle material centers in step (4-2) to construct neighbor relationships;

[0084] (4-4) Based on the obstacle material neighbor relationship constructed in step (4-3), the adaptive sampling step size of each obstacle boundary is calculated and determined;

[0085] (4-5) such as Figure 5 As shown, based on the adaptive step size of each obstacle determined in step (4-4), the obstacle boundary is discretely sampled to obtain the set of boundary sequence points of all obstacles.

[0086] S5: Input the set of obstacle boundary points obtained in step S4 into the Veno graph generation algorithm, and obtain the environment passable hierarchical network topology node graph by matching, merging, pruning and smoothing the graph nodes.

[0087] (5-1) such as Figure 6 As shown, the obstacle boundary sequence point set obtained in step S4 is input into the Vinio map generation algorithm to obtain Vinio point pairs, and then the point pairs occupying the obstacles are deleted;

[0088] (5-2) Based on the Vino point pairs obtained in step (5-1), integrate and pair them to obtain the initial traversable area network node map;

[0089] (5-3) such as Figure 7 As shown, nodes with three or more neighbors are selected as hub nodes. Road points between hub nodes are obtained by searching, and neighbor relationships between hub nodes are established. Hub nodes with close proximity are merged.

[0090] (5-4) such as Figure 8As shown, the road nodes between hub nodes are smoothed using Bézier curves to obtain the hierarchical network topology node diagram of the accessible environment.

[0091] S6: Convert the environment node network graph into an Eulerian graph; determine whether the environment passable hierarchical network topology node graph obtained in step S5 is an Eulerian graph. If not, proceed to S7; otherwise, proceed to S8.

[0092] (6-1) Based on the environmental network topology node graph obtained in step S5, the method in graph theory is used to identify the singularity in the graph by calculating the degree of each node. The number of singularities determines whether the graph is an Eulerian graph. If it is, go to step S8; otherwise, go to step S7.

[0093] S7: As Figure 9 As shown, the environmental network node graph obtained in step S5 is converted into an Eulerian graph using an algorithm based on least binary matching.

[0094] (7-1) Construct a bipartite graph model based on the set of singularities obtained in step S6. Let V1 be the set of singularities in graph G. By the basics of graph theory, it can be proved that the number of singularities in a simple graph is even, denoted as n. Construct a bipartite graph B = (S, T, E'), where S + T = V1. The relationship E' between nodes is constructed as follows:

[0095]

[0096] Introducing decision variable r ij =0,1 to represent S i With T j The matching relationship, if r ij =1 indicates a match; otherwise, it does not match.

[0097] (7-2) For the bipartite graph established in step (7-1), establish an integer programming model and find the matching relationship between singular vertices according to the distance-optimal principle. Figure 10 It shows the information about the edges that need to be traversed repeatedly between singularities;

[0098] minw(e i ′ j )r ij

[0099]

[0100] S8: Based on the Eulerian graph obtained in step S6, the improved Fleury algorithm is used to obtain the Eulerian circuit for any starting point, thus obtaining the route planned by the autonomous vehicle for exploring the entire environment, and finally returning to the origin.

[0101] (8-1) Take any point in the graph as the starting point v0∈V;

[0102] (8-2) Find neighboring edges as the next edge to be visited, and give the following priority: ① Unless there are no other alternative edges, the next edge to be visited cannot be a cut edge of the remaining graph; ② For non-cut edges, prioritize the shortest edge as the next edge to be visited.

[0103] (8-3) When step (8-2) cannot continue, the resulting trace is an Eulerian circuit starting from the selected starting point. The unmanned vehicle can efficiently and orderly explore every corner of the environment and eventually return to the starting point by following this circuit.

[0104] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A method for global environment exploration and route planning for unmanned vehicles assisted by unmanned aerial vehicles, characterized in that, The method includes the following steps: S1: The drone collects a stream of environmental images from above, following a pre-defined route, either remotely or autonomously. S2: The onboard computer receives the photo stream captured by the drone in step S1 and performs photo stitching based on FREAK features to obtain a panoramic overhead image of the environment of interest. S3: The panoramic top-down image obtained by stitching in step S2 is used to perform obstacle segmentation to obtain a binary map of environmental obstacles using a neural network based on Unet3+. S4: Extract connected components of obstacles, extract boundary vertices and centroids of obstacles, triangulate and pair the centroids of obstacles, and determine the adaptive sampling step size of obstacle boundaries from the binarized environment map obtained in step S3 to obtain a sparse set of obstacle boundary points. S5: Input the set of obstacle boundary points obtained in step S4 into the Veno graph generation algorithm, and obtain the environment passable hierarchical network topology node graph by matching, merging, pruning and smoothing the graph nodes. S6: Convert the environment node network graph into an Eulerian graph; determine whether the environment passable hierarchical network topology node graph obtained in step S5 is an Eulerian graph. If not, proceed to S7; otherwise, proceed to S8. S7: Convert the environmental network node graph obtained in step S5 into an Eulerian graph using an algorithm based on least binary matching; S8: Based on the Eulerian graph obtained in step S6, the improved Fleury algorithm is used to obtain the Eulerian circuit for any starting point, thus obtaining the route planned by the autonomous vehicle for exploring the entire environment, and finally returning to the origin.

2. The method for global environment exploration and route planning of an unmanned vehicle assisted by a drone according to claim 1, characterized in that, In step S2, the specific details are as follows: Includes the following processes: (2-1) Read the image stream according to the time label. For each image, use the FREAK algorithm to extract key points and features. (2-2) Match key points and features using a FLANN-based feature matcher algorithm; (2-3) Calculate the transformation matrix between adjacent images using a RANSAC-based algorithm; (2-4) Register adjacent images according to the transformation matrix and stitch the images together; (2-5) Optimize and smooth the stitched map to obtain a clearer stitching effect.

3. The method for global environment exploration and route planning of an unmanned vehicle assisted by a drone according to claim 1, characterized in that, In step S3, the specific details are as follows: Includes the following processes: (3-1) Create a dataset by collecting a set of labeled image datasets, where each image contains two parts: buildings and background. Preprocess the images, including resizing, cropping, rotating, flipping, and normalizing, so that they can be input into the Unet3+ network for training. (3-2) Construct the Unet3+ network and train it using the prepared dataset so that the network can learn the differences between buildings and the background and perform effective building obstacle segmentation; (3-3) Deploy the trained network, input the panoramic map obtained by stitching in step S2, complete the building obstacle segmentation task, and obtain the binary map of environmental obstacles.

4. The method for global environment exploration and route planning of an unmanned vehicle assisted by a drone according to claim 1, characterized in that, In step S4, the specific details are as follows: Includes the following processes: (4-1) Extract the obstacle connected components from the binary map obtained by segmentation in step S3; (4-2) Extract the boundary vertices and centroids for each obstacle connected domain in step (4-1); (4-3) Triangulate and pair all the obstacle material centers from step (4-2) to construct neighbor relationships; (4-4) Based on the obstacle material neighbor relationship constructed in step (4-3), the adaptive sampling step size of each obstacle boundary is calculated and determined; (4-5) Based on the adaptive step size of each obstacle determined in step (4-4), perform discrete sampling of the obstacle boundary to obtain the set of boundary sequence points of all obstacles.

5. The method for global environment exploration and route planning of an unmanned vehicle assisted by a drone according to claim 1, characterized in that, In step S5, the specific details are as follows: Includes the following processes: (5-1) Input the set of obstacle boundary sequence points obtained in step S4 into the Vinio map generation algorithm to obtain Vinio point pairs, and delete the point pairs that occupy obstacles; (5-2) Based on the Vino point pairs obtained in step (5-1), integrate and pair them to obtain the initial traversable area network node map; (5-3) Select nodes with three or more neighbors as hub nodes, obtain road points between hub nodes by searching, establish neighbor relationships between hub nodes, and merge hub nodes that are close in distance. (5-4) Smooth the road nodes between hub nodes to obtain the environmentally accessible hierarchical network topology node diagram.

6. The method for global environment exploration and route planning of an unmanned vehicle assisted by a drone according to claim 1, characterized in that, In step S6, the specific details are as follows: Includes the following processes: (6-1) Based on the environmental network topology node graph obtained in step S5, the singularity in the graph is identified by calculating the degree of each node. The number of singularities determines whether the graph is an Eulerian graph. If it is, proceed to step S8; otherwise, proceed to step S7.

7. The method for global environment exploration and route planning of a drone-assisted unmanned vehicle according to claim 1, characterized in that, In step S7, the specific details are as follows: Includes the following processes: (7-1) Construct a bipartite graph model based on the set of singularities obtained in step S6; (7-2) For the bipartite graph established in step (7-1), establish an integer programming model and find the matching relationship between singularities according to the principle of optimal distance.

8. The method for global environment exploration and route planning of an unmanned vehicle assisted by a drone according to claim 1, characterized in that, In step S8, the specific details are as follows: Includes the following processes: (8-1) Choose any point in the graph as the starting point; (8-2) Find its neighboring edge as the next edge to be visited, and give the following priority: ① Unless there are no other alternative edges, the next edge to be visited cannot be a cut edge of the remaining graph; ② For non-cut edge neighboring edges, prioritize the shortest edge as the next edge to be visited. (8-3) When step (8-2) cannot continue, the resulting trace is an Eulerian circuit starting from the selected starting point. The unmanned vehicle efficiently and orderly explores every corner of the environment according to this circuit and eventually returns to the starting point.

Images