Multi-robot autonomous exploration enhanced hierarchical planning framework

Abstract

The invention discloses a multi-robot autonomous exploration enhanced hierarchical planning framework, and solves the problems of short visibility and difficult convergence of a high-dimensional learning space of an existing system in scenes such as a large environment, an increased number of agents and a limited transmission bandwidth. A three-layer planning framework is provided, and low-dimensional action space efficiency of a boundary method and hyperopia and optimality of deep reinforcement learning are combined. First, a boundary is determined in a free space, a sparse map representation is created to mitigate a data transmission burden, and a dimension of a deep reinforcement learning action space is reduced. Secondly, developing a multi-graph neural network (mGNN), incorporating a target state and a robot state, and calculating affinity by utilizing reinforcement learning based on a strategy to replace a traditional heuristic utility value; and finally, local routing planning is realized through subsequence search, and exhaustive traversal is avoided. Wide verification and simulation results prove that the framework can realize environment exploration with less time step length and data transmission quantity, and shows the superiority of efficiency and performance.

Description

Technical Field

[0001] The present invention relates to the field of multi-robot environment exploration, and in particular to a method for preprocessing environmental information and extracting and allocating target points in a multi-machine large environment. Specifically, the present invention uses a boundary point-based method to extract candidate targets and a sparse representation of a local map, and uses a multi-graph neural network to calculate the utility between the robot and the candidate targets. Background Art

[0002] Exploring unknown environments is a fundamental task in robotic applications. It refers to observing and mapping the environment using sensor data without prior knowledge of the environment, estimating its own state, and making decisions and planning for the exploration task based on this. The goal is to scan the specified unknown environment as thoroughly and accurately as possible in the shortest time to establish conditions for subsequent robotic tasks. Traditional methods often rely on greedy frontier selection or end-to-end deep reinforcement learning methods, but these methods are often limited, such as short-sighted decisions or convergence difficulties due to high-dimensional learning space. Therefore, it is necessary to design a decision-making method that can consider long-term utility and is easy to learn. In addition, most existing multi-robot exploration systems integrate local information by acquiring sensor data or complete map data, but this consumes a lot of communication resources, especially in systems using 3D LiDAR. Therefore, how to enable the computing center to obtain the required data with lower transmission bandwidth is an urgent problem that needs to be solved. Summary of the invention

[0003] The purpose of the present invention is to propose a multi-robot autonomous exploration enhanced hierarchical planning framework to solve the suboptimality and short-sightedness of boundary-based methods in traditional approaches and the problem of too high learning space dimension in end-to-end deep learning methods.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A multi-robot autonomous exploration enhanced hierarchical planning framework, characterized by comprising: an environmental information preprocessing layer, a global planning layer and a local planning layer:

[0006] The environmental information preprocessing layer preprocesses the local environmental information obtained by the sensors of each robot, and calculates a sparse representation that can provide complete environmental information; the computing center collects the local information data, and obtains a complete global map representation through a merging algorithm and a map reconstruction algorithm; the candidate targets are calculated based on the global information and provided to the planning layer;

[0007] The global planning layer establishes a multi-graph neural network based on the candidate targets and the robot posture information, updates it through deep reinforcement learning, and obtains the utility matrix between each robot and each candidate target through model calculation. The Hungarian algorithm is used for binary allocation to obtain the exploration target allocation that maximizes the overall utility.

[0008] When the number of boundary points within the assigned target reaches a threshold, the local planning layer performs local clustering to obtain local candidate target points, uses the subsequence transposition traversal method to search for the local routing plan with the highest utility, avoids exhaustive enumeration of the routing sequence, and after completing the planning, sends the target to the navigation algorithm to control the robot to move to the target.

[0009] In addition, a hierarchical exploration planning method using the above multi-robot autonomous exploration enhanced hierarchical planning framework is proposed, including:

[0010] 1) The processing flow of the environmental information preprocessing layer is as follows:

[0011] The robot obtains local environment information and represents it with a grid map;

[0012] The local grid map is denoised by a closing operation, and the boundary extraction algorithm proposed by the present invention is used to extract the dense boundaries of occupied space and free space;

[0013] The dense boundary information is transmitted as a sparse representation of the map to the computing center, which obtains the global representation through merging and reconstruction algorithms;

[0014] The computing center uses a clustering algorithm to calculate candidate targets and calculates the distance between each robot and each candidate target;

[0015] 2) The processing flow of the global planning layer is as follows:

[0016] Establish robot self-graph, candidate target self-graph and cross-graph structure according to robot information and candidate target information;

[0017] Build a multi-graph neural network model and perform deep reinforcement learning by calculating the increase in map area after each decision as the reward value;

[0018] The model is used to calculate the utility between each robot and each candidate target, and the Hungarian algorithm is used to perform binary allocation;

[0019] 3) The processing flow of the local planning layer is as follows:

[0020] When the number of boundary points within the candidate target exceeds the threshold, secondary clustering is performed to obtain local candidate targets;

[0021] Calculate the distance between targets and the number of cluster members of the targets, and calculate the utility function based on this;

[0022] The highest utility route is calculated based on the utility, and the target point is sent to the navigation module to make the robot move to the target point.

[0023] Furthermore, the specific steps of preprocessing by the environmental information preprocessing layer are as follows:

[0024] S1: Grid map created by SLAM algorithm Use closing operation for denoising;

[0025] S2: The denoised map is divided into three channels according to the grid state and right and Use kernel convolution to calculate the boundary points and then Perform AND operation to obtain the occupied boundary F occ_i and the unknown boundary F un_i ;

[0026] S3: Let F i =F occ_i ∪F un_i (1≤i≤N), using robot r i The position is used as the seed coordinate to fill the scan line and obtain the reconstructed raster map

[0027] S4: F i For each point in The proportion of unknown grids within the middle radius RAD, reaching the threshold means that the robot r j Explored, kept until unp list Otherwise, it means it has been used by a robot j explored, discarded;

[0028] S5: For unp list Mean-Shift clustering is performed on the points in where c k Represents cluster C k The center of all clusters C k Form a global set C.

[0029] Furthermore, the specific steps of global goal planning are as follows:

[0030] S1: Constructing the robot graph G r =(V r , E r ) and candidate cluster center graph G c =(V c , E c ), and the cross graph G connecting the robots and the cluster centersrc =(V r , V c , E rc ), where V r and E r represents the characteristics of the robot and the weights of their relationships, while V c and E c Represents the characteristics of cluster centers and the weights of their relationships, E rc Represents the matching utility between the robot and the cluster center to be learned;

[0031] S2: For robot status and candidate targets Building a multi-layer perceptron to extract high-dimensional features

[0032] S3: Through three linear projections L query , To calculate the query of the node key Sum

[0033] S4: Iterate l layers to calculate G through the attention mechanism r , G c , G rc The attention weights and feature values ​​of

[0034] S5: Use the Hungarian algorithm to perform binary allocation and obtain the allocation r between the robot and the target point i ~c k .

[0035] The training model is set up as follows: Calculate the map area increase after this decision in is the occupied area of ​​the grid map at time t, and the time step R consumed by the task completion after this decision time , the overall reward is R = R time +λR area , where λ is a hyperparameter that balances the two rewards. PPO is used as the policy optimizer to update the hyperparameters of the multi-graph neural network model.

[0036] Furthermore, the local routing is planned as follows:

[0037] S1: For each r i ~c k , C is transformed by Mean-Shift algorithm k Members are clustered to obtain local candidate targets f kj ∈C k ;

[0038] S2: Define ρk =[f k0 , f k1 , f k2 ..f kn ] is the path of the local candidate points;

[0039] where f k0 For robot i Current location, G kj f kj The information gain is equal to the number of class members, L kj f k0 to f kj The distance from A * Calculated by the algorithm;

[0040] S3: To G kj and L kj , the following calculation is performed to expand the information gain variance and reduce the distance cost variance:

[0041] Among them, k g , k l is the attenuation rate, U G and U L As the normalized upper limit, epsilon is a very small positive number to prevent division by zero;

[0042] S4: Calculate ρ k The sum of the utilities in Find the optimal route with the highest utility through subsequence transposition traversal

[0043] S5: Send the target point of the route to the navigation module, and the robot ri moves to the target point to complete the task.

[0044] Furthermore, the subsequence transposition traversal process is as follows:

[0045] S1: Press G kj As the initial sorting,

[0046] S2: Randomly select two non-robot indexes x and y, reverse the subsequence between them to generate a new sequence, calculate all possible situations, and take the path with the highest utility as the new initial sequence;

[0047] S3: Repeat S1 and S2 until a certain initial sequence does not produce a path with higher utility after any reverse subsequence, then the initial sequence is the best route BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the implementation of the present invention or the existing technical solutions, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art:

[0049] Figure 1 Explore a three-layer planning architecture for multi-robot environments;

[0050] Figure 2 This is the flow chart of environment preprocessing, where (a) is denoising, boundary detection and map reconstruction, and (b) is boundary point merging and clustering;

[0051] Figure 3 It is the boundary detection map of the three-channel occupancy grid map;

[0052] Figure 4 To develop a global planning graph using robots and clustering information; DETAILED DESCRIPTION

[0053] The present invention is specifically introduced below with reference to the accompanying drawings and specific examples, and the specific implementation process is as follows:

[0054] Phase 1: Three-layer planning architecture for multi-robot environment exploration

[0055] like Figure 1 As shown in the figure, the design of a three-layer planning framework including a global planning layer can provide a theoretical reference for multi-robots to explore unknown environments.

[0056] The preprocessing layer calculates the global grid map and candidate target points. Each robot r i Get the local environment grid map and boundary. Then at the calculation center, merge the local frontiers to get the global frontier F un and F occ , calculate the reconstructed global grid map g. Use the Mean-Shift algorithm to get the cluster center and cluster members f k ∈c k . The cluster center is used as the candidate target of the robot;

[0057] The global planning layer calculates the robot's allocation of candidate points. Based on the reconstructed global map Robot Status ri and cluster center state c k , calculate the matching target position c through the cross graph k ~s ri ;

[0058] The local planning layer performs local routing planning for the cluster members assigned to the robot. k ∈C k , we get a route ρ that maximizes the utility U *, and then navigate and control to enable the robot to complete the task.

[0059] Stage 2: Dense boundary detection and sparse map construction

[0060] Figure 2 The algorithm for calculating candidate targets and reconstructing environmental information based on boundary point extraction realizes the preprocessing of local environmental information and the integration of global environmental information, ensures the acquisition of global candidate target information, and reduces the amount of transmitted data. The grid map established by the SLAM algorithm It often contains a large number of noise points, and directly extracting boundary points will result in high discreteness. Therefore, the dilation operation is first used to expand the free space, and then the dilation and erosion operations (closing operations) are used simultaneously to denoise the grid map.

[0061] like Figure 3 The present invention proposes to use multi-channel kernel convolution to extract F un and F occ For the denoised raster map The traditional boundary point extraction algorithm can only obtain the boundary points between the free space and the unknown space, which requires a lot of calculation. First, use the Boolean index to extract the boundary points between the free space and the unknown space. Divided into three channel matrices and Then, the boundary points of the occupied space and the unknown space are calculated through kernel convolution, and then their intersection in the free space is calculated. Since the extracted boundary points are dense, a closed area is formed inside, which can support subsequent map reconstruction in the center.

[0062] After completing the boundary detection, the local occupancy grid map is converted into a sparse boundary map through the sparse graph structure and transmitted to the computing center. i (1≤i≤N), the sparse graph is reconstructed into an occupancy grid map through the occupancy grid map reconstruction algorithm. Specifically, s ri The coordinates of F are the seed coordinates, and the scan line filling algorithm is used to fill F un_i and F occ_i The internal feasible region is filled with free space. Since the current robot position s is used ri are the seed coordinates, and some external boundary noise points may be located in unreachable areas. After removing the noise points using the closing operation, we can get the sparse graph The constructed raster map.

[0063] In order to operate the global planning layer, it is necessary to obtain the coordinates of the global candidate area. In order to avoid redundant global boundary points when multiple robots are close to each other, it is necessary to map the local map. Specifically, for a given detection radius RAD, a detection matrix C is established. R , the grid map G reconstructed by other robots k ′(r k ∈R, k≠i), check each robot r i The local boundary point f un_i ∈F un_i In C R Whether the unknown space in (PRC_UN) is sufficient, that is, whether the point is close to the boundary. If so, the point is considered to be on the global boundary and is added to unp list If not, then the robot's boundary point f un_i Already explored by other robots in the global map, will not be used as a global border.

[0064] If the boundary points are directly assigned as candidate targets, the action space is too large and the utility of boundary point aggregation is ignored. Therefore, Mean-Shift is used to cluster the frontier points, and the center of each cluster is used to calculate the robot's target to improve the efficiency of target assignment. Mean-Shift iteratively moves the core to a higher density area until convergence. However, the cluster center can easily be generated in an unreachable area, so the coordinates of the boundary point closest to the cluster center are selected as candidate target points. where c k Represents cluster C k The center of all clusters C k Form a set C.

[0065] Stage 3: Target Assignment Based on Multi-Graph Neural Network

[0066] Figure 4 It is a multi-graph neural network based on deep reinforcement learning. According to the candidate target information and robot information, the reasonable allocation of each robot and candidate target is realized to ensure that the system decision can maximize the long-term benefits of exploration.

[0067] Through the reconstructed grid occupancy map, robot clustering information and location information, the target point assignment problem is described as the association problem between robot nodes and target nodes. The association relationship is represented by a bipartite graph. Specifically, in the robot graph G r Node V r With the cluster center graph G c Node V c Create a cross graph G between rc , and estimate the utility edge E through a multi-graph neural network based on the attention mechanism rc .

[0068] Specifically, for and Build a multi-layer perceptron to extract high-dimensional features Through three linear projections L query , To calculate the query of the node key Sum Make similarity Based on this, the attention weight from node j to node i can be calculated:

[0069] The attention value can then be calculated

[0070] Next, through the multi-layer perceptron f V Calculate node eigenvalues:

[0071] For the candidate target point graph G c The same process is performed to update the node features. After completing the node feature update of the self-graph, the feature update operation of the cross-graph can be performed. * The algorithm finds the node r on the occupancy grid map i and c k The shortest distance between ik , the information gain g from i to k ik Use the number of members of the corresponding cluster |C k | indicates that these two pieces of information are considered when computing the directed edge features from node i to k. In cross-graph operations, the similarity is represented as a multi-layer perception The edge attention weight is calculated as:

[0072] Attention value It can be calculated as:

[0073] Then replace the attention value in the above calculation node feature value with Thus, the node features are updated twice. The self-graph and cross-graph constitute an operation block in turn, and each time the l-layer operation block is used to calculate the features of the robot node and the cluster center node. The graph operation is executed in the computing center, using the proximal policy optimization (PP0) as the policy optimizer. Specifically, the reward value is calculated by consuming the time step R time and the exploration area growth R area Represents, where the former is a fixed negative real number and the latter is R area =a(G i )-A(Gi-1 ), where a is the occupied area of ​​the grid map at time t, so the total reward is: R=R time +λR area

[0074] Where λ is a hyperparameter that balances the two rewards. After the graph operation is completed, a weight matrix is ​​obtained, corresponding to the robot r i For cluster center c k At this point, the problem is transformed into a bipartite graph G rc Each robot node in the cluster is assigned a cluster center node to maximize the allocation utility. This paper uses the Hungarian algorithm to calculate the allocation r i ~c k Since the Hungarian algorithm requires the same number of frontier points as goal points, a dummy robot with negative infinite weight is added and subsequently skipped in the final output.

[0075] Phase 4: Local routing planning based on subsequence transposition index

[0076] After obtaining the cluster centers, merely visiting these centers is not enough to conduct local exploration in a large scene. The present invention sets that when the average number of cluster members reaches a threshold value θ, local routing planning and control planning are performed.

[0077] For robot ri and its designated cluster center C k , cluster its members through the Mean-Shift algorithm to obtain the local candidate target point f kj ∈C k In order to effectively calculate the local exploration route, both the benefit and the cost should be considered, so the present invention proposes a routing scheme that determines the best route to visit all local candidate target points.

[0078] Define ρ k =[f k0 , f k1 , f k2 ...f kn ] is the path of the local candidate point, where f k0 For robot i Current location, G kj f kj The information gain is equal to the number of subclass members, L kj f k0 to f kj The distance from A * The algorithm calculates the optimal path. This maximizes the following overall utility function:

[0079] Where c is a balancing factor used to balance the interests of both parties. This problem is similar to the traveling salesman problem (TSP), but the standard TSP only considers distance cost without considering information gain, and needs to return to the starting point after completing the exploration of all target points. Many methods typically use the number of unknown areas or boundaries around the target point as a measure of information gain. While this can provide substantial benefits, it often leads to a preference for targets with higher information gain because the scanning range of the sensor affects nearby candidate targets. Therefore, before determining the route, the following function needs to be calculated:

[0080] Among them, k g , k l is the attenuation rate, U G and U L As the normalized upper limit, epsilon is a very small positive number to prevent division by zero. The purpose of this calculation is to expand the information gain gap and narrow the distance gap.

[0081] After preprocessing, we need to find the subsequence of the best route. However, when there are many target points, it is difficult to find the subsequence. Therefore, we use the method of reversing the subsequence. Specifically, (1) press G kj As the initial sorting; (2) Randomly select two non-robot indexes x and y, reverse the subsequence between them to generate a new sequence, calculate all possible situations, and take the path with the highest utility as the new initial sequence; (3) Repeat steps (1) and (2) until no path with higher utility can be found in the reversed subsequence of a certain initial sequence. After calculating ρ * Finally, the robot uses the navigation stack of the Robot Operating System (ROS) to navigate to the target point.

[0082] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A multi-robot autonomous exploration enhanced hierarchical planning framework, characterized in that: include: Environmental information preprocessing layer, global planning layer and local planning layer: The environmental information preprocessing layer preprocesses the local environmental information obtained by each robot sensor and calculates a sparse representation that can provide complete environmental information; the computing center collects each local information data and obtains a complete global map representation through a merging algorithm and a map reconstruction algorithm; Calculate candidate targets based on global information and provide them to the planning layer; The global planning layer establishes a multi-graph neural network based on the candidate targets and the robot posture information, updates it through deep reinforcement learning, and obtains the utility matrix between each robot and each candidate target through model calculation. The Hungarian algorithm is used for binary allocation to obtain the exploration target allocation that maximizes the overall utility. When the number of boundary points within the assigned target reaches a threshold, the local planning layer performs local clustering to obtain local candidate target points, uses the subsequence transposition traversal method to search for the local routing plan with the highest utility, avoids exhaustive enumeration of the routing sequence, and after completing the planning, sends the target to the navigation algorithm to control the robot to move to the target.

2. A hierarchical exploration planning method using the multi-robot autonomous exploration enhanced hierarchical planning framework as claimed in claim 1, comprising: 1) The processing flow of the environmental information preprocessing layer is as follows: The robot obtains local environment information and represents it with a grid map; The local grid map is denoised by a closing operation, and the boundary extraction algorithm proposed by the present invention is used to extract the dense boundaries of occupied space and free space; The dense boundary information is transmitted as a sparse representation of the map to the computing center, which obtains the global representation through merging and reconstruction algorithms; The computing center uses a clustering algorithm to calculate candidate targets and calculates the distance between each robot and each candidate target; 2) The processing flow of the global planning layer is as follows: Establish robot self-graph, candidate target self-graph and cross-graph structure according to robot information and candidate target information; Build a multi-graph neural network model and perform deep reinforcement learning by calculating the increase in map area after each decision as the reward value; The model is used to calculate the utility between each robot and each candidate target, and the Hungarian algorithm is used to perform binary allocation; 3) The processing flow of the local planning layer is as follows: When the number of boundary points within the candidate target exceeds the threshold, secondary clustering is performed to obtain local candidate targets; Calculate the distance between targets and the number of cluster members of the targets, and calculate the utility function based on this; The highest utility route is calculated based on the utility, and the target point is sent to the navigation module to make the robot move to the target point.

3. The hierarchical exploration planning method according to claim 2, characterized in that: The specific steps of preprocessing in the environmental information preprocessing layer are as follows: S1: Grid map created by SLAM algorithm Use closing operation for denoising; S2: The denoised map is divided into three channels according to the grid state and right and Use kernel convolution to calculate the boundary points and then Perform AND operation to obtain the occupied boundary F occ_i and the unknown boundary F un_i ; S3: Let F i =F occ_i ∪F un_i (1≤i≤N), using robot r i The position is used as the seed coordinate to fill the scan line and obtain the reconstructed raster map S4: F i For each point in The proportion of unknown grids within the middle radius RAD, reaching the threshold means that the robot r j Explored, reserved until unp list Otherwise, it means it has been used by a robot j explored, discarded; S5: For unp list Mean-Shift clustering is performed on the points in where c k Represents cluster C k The center of all clusters C k Form a global set C.

4. The hierarchical exploration planning method according to claim 2, characterized in that: The specific steps of global goal planning are as follows: S1: Constructing the robot graph G r =(V r , E r ) and candidate cluster center graph G c =(V c , E c ), and the cross graph G connecting the robots and the cluster centers rc =(V r , V c , E rc ), where V r and E r represents the characteristics of the robot and the weights of their relationships, while V c and E c Represents the characteristics of cluster centers and the weights of their relationships, E rc Represents the matching utility between the robot and the cluster center to be learned; S2: For robot status and candidate targets Building a multi-layer perceptron to extract high-dimensional features S3: Through three linear projections L query , To calculate the query of the node key Sum S4: Iterate l layers to calculate G through the attention mechanism r , G c , G rc The attention weights and feature values ​​of S5: Use the Hungarian algorithm to perform binary allocation and obtain the allocation r between the robot and the target point i ~c k ; The training model is set up as follows: Calculate the map area increase after this decision in is the occupied area of ​​the grid map at time t, and the time step R consumed by the task completion after this decision time , the overall reward is R = R time +λR area , where λ is a hyperparameter that balances the two rewards. PPO is used as the policy optimizer to update the hyperparameters of the multi-graph neural network model.

5. The hierarchical exploration planning method according to claim 2, characterized in that: The local routing plan is as follows: S1: For each r i ~c k , C is transformed by Mean-Shift algorithm k Members are clustered to obtain local candidate targets f kj ∈C k ; S2: Define ρ k =[f k0 , f k1 , f k2 ...f kn ] is the path of the local candidate points; where f k0 For robot i Current location, G kj f kj The information gain is equal to the number of class members, L kj f k0 to f kj The distance from A * Calculated by the algorithm; S3: To G kj and L kj , the following calculation is performed to expand the information gain variance and reduce the distance cost variance: Among them, k g , k l is the attenuation rate, U G and U L As the normalized upper limit, epsilon is a very small positive number to prevent division by zero; S4: Calculate ρ k The sum of the utilities in Find the optimal route with maximum utility through subsequence transposition traversal method S5: Send the route target point to the navigation module, the robot i Move to the target point to complete the mission.

6. The hierarchical exploration planning method according to claim 5, characterized in that: The subsequence transposition traversal method process is as follows: S1: Press G kj As the initial sorting, S2: Randomly select two non-robot indexes x and y, reverse the subsequence between them to generate a new sequence, calculate all possible situations, and take the path with the highest utility as the new initial sequence; S3: Repeat S1 and S2 until a certain initial sequence does not produce a path with higher utility after any reverse subsequence, then the initial sequence is the best route

Images