A method for exploring unknown scenes with multi-robot collaboration

Abstract

The present invention relates to a method for exploring unknown scenes in a collaborative multi-robot environment. The method comprises a local exploration phase and a global exploration phase. In the local exploration phase, each robot explores within a local exploration area, generates a semantic-frontier viewpoint, and generates a local path for each semantic-frontier viewpoint by solving a traveling salesman problem. In the global exploration phase, a dynamic topological graph is constructed, the semantic-frontier viewpoints are stored in leaf nodes of the graph, and the graph is shared with all robots. When leaving the local exploration area, each robot considers the leaf nodes within the graph and generates a global path by solving multiple traveling salesman problems. The robot is then coordinated to enter the nearest new local exploration area, thereby achieving collaborative multi-robot exploration. The present invention enables efficient collaborative multi-robot exploration.

Description

Technical Field

[0001] The present invention relates to the field of robot autonomous exploration technology, and in particular to an unknown scene exploration method for multi-robot collaboration. Background Art

[0002] Autonomous robotic exploration involves planning and controlling robot movements to fully and accurately model the surrounding environment. This typically relies on Simultaneous Localization and Mapping (SLAM) technology. When exploring unknown scenarios, collaborative exploration by multiple robots can leverage their numerical superiority over a single robot, expanding the scope of exploration while improving efficiency.

[0003] Among existing single-robot and multi-robot autonomous exploration methods, lidar and depth cameras are the two main sensor types, and the maps used for exploration are mostly grid or voxel maps. There are two classic exploration methods based on these maps: frontier-based and sample-based. In recent years, methods that combine frontier and sample-based methods have emerged and are becoming mainstream. These methods typically use a set of frontier viewpoints to cover a grid or voxel frontier, with the robot completing autonomous exploration by traversing each frontier viewpoint.

[0004] However, the selection of frontier viewpoints in this method often relies solely on the geometric information of the scene. For example, for a robot using lidar for exploration, its frontier is composed of discrete point clouds. The density of the point cloud (i.e., the lidar resolution) directly affects the quality of the frontier representation. Therefore, the frontier viewpoint is tightly coupled to the quality of the point cloud generation. If the sensor resolution is low or there are dynamic obstacles blocking the frontier, the frontier representation will be incomplete, resulting in inaccurate or even missing frontier viewpoints. This can cause the robot to backtrack, repeatedly exploring the same area, reducing exploration efficiency. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an unknown scene exploration method for multi-robot collaboration, which can realize efficient multi-robot collaborative exploration.

[0006] The technical solution adopted by the present invention to solve its technical problems is: to provide an unknown scene exploration method for multi-robot collaboration, dividing the area to be explored into several sub-areas of fixed shapes, each sub-area of ​​fixed shape as a local exploration area, and the robot moves to a new local exploration area for exploration after completing the exploration of the local exploration area until the exploration of all local exploration areas is completed. Specifically, it includes a local exploration stage and a global exploration stage. In the local exploration stage, each robot explores in the local exploration area, each robot generates a semantic-frontier viewpoint, and generates a local path for the semantic-frontier viewpoint by solving the traveling salesman problem; in the global exploration stage, by constructing a dynamic topological map, the semantic-frontier viewpoint is stored in the leaf node of the topological map and shared with all robots. When each robot leaves the local exploration area, it considers the leaf node in the topological map, generates a global path by solving the multi-traveling salesman problem, coordinates each robot to enter the new local exploration area nearby, and realizes multi-robot collaborative exploration.

[0007] When each robot generates a semantic-frontier viewpoint, the surrounding environment image obtained by the robot is used to extract the semantic information of the image in the frontier direction through a pre-trained network, and the semantic information is used as the information gain to measure the frontier, and is combined with the frontier viewpoint to obtain the semantic-frontier viewpoint.

[0008] When generating a local path for the semantic-frontier viewpoint by solving the traveling salesman problem, the end point is selected as the semantic-frontier viewpoint with the largest information gain.

[0009] In the local exploration phase, each robot updates the position, viewing angle, corresponding robot number and semantic-frontier viewpoint information gain of the unexplored frontier viewpoint at the boundary of the local exploration area to the leaf node of the topological graph.

[0010] The specific method of constructing a dynamic topology map is as follows: each robot starts to explore the unknown environment from the initial point, and at the same time constructs a topology map T = (V, E), where V is the node of the topology map, E is the edge connecting the nodes, the root node of the topology map is the initial position of the robot, and the nodes of the topology map are divided into internal nodes and leaf nodes. The internal node is the center point of the explored area. When the local exploration area is completed, it will be added to the topology map; the leaf node is the front viewpoint at the boundary of the local exploration area, which constitutes the exploration frontier of all robots. The front viewpoint contains four pieces of information, namely position, viewing angle, semantic information gain and robot number.

[0011] The dynamic update process of the topology map includes:

[0012] When the i-th robot has finished exploring the current local exploration area, the edges of the topological graph connect the explored areas;

[0013] For several robots that have explored the local exploration area, a target point is selected from all leaf nodes of the topological graph by solving the multi-traveling salesman problem, and the target point is removed from the topological graph. The robot then enters a new local exploration area for exploration.

[0014] When the robot completes the exploration of a new local exploration area, the center point of the new local exploration area is added to the topological graph, and an edge is added. At the same time, the frontier viewpoint currently at the boundary of the local exploration area is updated to a new leaf node.

[0015] The technical solution adopted by the present invention to solve its technical problems is: to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the steps of the above-mentioned unknown scene exploration method for multi-robot collaboration are implemented.

[0016] The technical solution adopted by the present invention to solve its technical problem is: providing a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned unknown scene exploration method for multi-robot collaboration are implemented.

[0017] Beneficial effects

[0018] Due to the adoption of the above-mentioned technical solution, the present invention has the following advantages and positive effects compared with the prior art: the present invention installs a camera on the robot to obtain image information of the robot's surrounding environment, extracts semantic information in the frontier direction through a pre-trained network, uses the semantic information as the information gain to measure the frontier, and combines it with the frontier viewpoint to generate a semantic-frontier viewpoint, which effectively solves the inaccuracy caused by relying solely on geometric information; in addition, the present invention stores the semantic-frontier viewpoints through the leaf nodes of the topological tree, and all robots can share these frontier viewpoints. By solving the multi-traveling salesman problem (MTSP), each robot collaboratively explores these shared frontier viewpoints, giving priority to exploring the frontier with large information gain, thereby improving the efficiency of multi-machine collaborative exploration. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a framework diagram of the unknown scene exploration method for multi-robot collaboration according to the first embodiment of the present invention;

[0020] Figure 2 is a screening graph of semantic-frontier viewpoints in the first embodiment of the present invention;

[0021] Figure 3 Schematic diagram of a topological diagram in the first embodiment of the present invention. DETAILED DESCRIPTION

[0022] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the appended claims of the application.

[0023] The first embodiment of the present invention relates to a method for exploring unknown scenes for multi-robot collaboration, such as Figure 1 As shown, multiple robots start from an initial point to explore the area to be explored, and the area to be explored is divided into multiple square areas of equal area, where each square area represents a local exploration area. After completing the exploration of a local exploration area, the robot moves to a new local exploration area for exploration until the exploration of all local exploration areas is completed.

[0024] This implementation adopts a hierarchical exploration approach, divided into local exploration and global exploration phases. The local exploration phase is responsible for exploration within a local exploration area. Each robot generates a semantic-frontier viewpoint and generates a local path for each semantic-frontier viewpoint by solving the traveling salesman problem. The global exploration phase constructs a dynamic topological graph, stores the semantic-frontier viewpoints in the leaf nodes of the topological graph, and shares it with all robots. When each robot leaves a local exploration area, it simultaneously considers the leaf nodes within the topological graph and generates a global path by solving the multi-traveling salesman problem. This coordinates each robot to enter the nearest new local exploration area, enabling multi-robot collaborative exploration.

[0025] Since it is difficult to accurately characterize the frontier and the frontier viewpoint by relying solely on the geometric information of the sensor, in order to overcome this problem, this embodiment installs a camera on the robot, and uses the camera to obtain image information of the robot's surrounding environment. The obtained surrounding environment image is used through a pre-trained network to extract the semantic information of the image in the frontier direction, and the semantic information is used as the information gain to measure the frontier. It is combined with the frontier viewpoint to obtain the semantic-frontier viewpoint. Compared with the traditional frontier viewpoint that only relies on geometric information, it uses the semantic information in the frontier direction as the information gain of the frontier viewpoint, so that the selection of the frontier viewpoint has priority, which is of great significance for exploration. Therefore, for the semantic-frontier viewpoint, when the robot obtains a local path by solving the traveling salesman problem, the end point will select the frontier viewpoint with the largest information gain, such as Figure 2 As shown in the figure, the process continues until the local area is explored. For the frontier viewpoints that are at the boundary of the local exploration area and have not been explored, the robot updates its position, viewpoint, corresponding robot number, and semantic-frontier viewpoint information gain to the leaf nodes of the topology graph.

[0026] The topological map constructed in this embodiment is based on the regional decomposition of the unknown scene: in order to avoid multiple robots repeatedly exploring an area and give full play to the advantages of multi-robot collaborative exploration, the area to be explored is subdivided into square areas of equal area, and the square areas represent local exploration areas, such as Figure 3 As shown in the figure, in a local exploration area, the robot will complete the exploration of the area as much as possible independently, and will not enter the next local exploration area until it has explored the local exploration area, thus ensuring the complete exploration of the area. Figure 3 In the figure, the yellow squares are explored areas, the gray squares are unexplored areas, and the center point of each area is p = (x, y).

[0027] The process of constructing the topology diagram in this embodiment is as follows: Figure 3 Taking the collaborative exploration of three robots as an example, first, multiple robots start exploring the unknown environment from the initial point and simultaneously build a topology map T = (V, E), where V is the node of the topology map, E is the edge connecting the nodes, and the root node of the topology map is the initial position of the robot. The nodes of the topology map are divided into internal nodes and leaf nodes, with the following characteristics: 1) The internal node is the center point v of the explored area. inter =(robot_id,p), when the local exploration area is completed, it will be added to the topology map; 2) The leaf node is the frontier viewpoint v at the boundary of the local exploration area leaf , which constitutes the exploration frontier of all robots, the frontier viewpoint v leaf Contains four pieces of information, including position p = (x, y), perspective θ, semantic information gain I and robot number robot_id.

[0028] The dynamic update process of the topological map represents the collaborative exploration process of multiple robots in the global exploration layer, as follows:

[0029] 1) When the i-th robot has completed its exploration of the current local exploration area, the edges of its topological graph connect the explored areas. At this point, the topological graph represents the explored areas and all current frontier viewpoints;

[0030] 2) For several robots that have completed their local exploration area, the multi-traveling salesman problem is solved to select target points from all leaf nodes of the topology graph. The selected target points are then removed from the topology graph. The robots then enter new local exploration areas for exploration using the generated global planning path.

[0031] 3) When the robot completes the exploration of the new local exploration area, the center point of the local exploration area is added to the topological graph and an edge is added. At the same time, the frontier viewpoint currently at the boundary of the local exploration area is updated to a new leaf node.

[0032] It is worth noting that the robot's frontier viewpoint is changing dynamically during the exploration process, which will also affect the choice of exploration area for each robot. The dynamic update of the topology map can effectively cope with such dynamic changes. Through the continuous updating of the above topology map, multiple robots can gradually complete the exploration of unknown environments.

[0033] It is not difficult to find that the nodes of the topological graph in this embodiment represent the explored areas and frontier viewpoints, the node update process reflects the collaborative exploration of multiple robots, and the directed edges represent the exploration order of the local exploration area. This embodiment uses the topological graph structure to allow all robots to share all the frontier viewpoints, thereby achieving efficient collaborative exploration.

Claims

1. A method for exploring unknown scenes for multi-robot collaboration, characterized in that: The area to be explored is divided into several sub-areas of fixed shapes, each of which serves as a local exploration area. After completing the exploration of a local exploration area, the robot moves to a new local exploration area for exploration until the exploration of all local exploration areas is completed. Specifically, the process includes a local exploration stage and a global exploration stage. In the local exploration stage, each robot explores within the local exploration area, generates a semantic-frontier viewpoint, and generates a local path for the semantic-frontier viewpoint by solving the traveling salesman problem. In the global exploration stage, a dynamic topological graph is constructed, and the semantic-frontier viewpoint is stored in the leaf node of the topological graph and shared with all robots. When leaving the local exploration area, each robot considers the leaf node in the topological graph, generates a global path by solving the multi-traveling salesman problem, coordinates each robot to enter the new local exploration area nearby, and realizes multi-robot collaborative exploration.

2. The unknown scene exploration method for multi-robot collaboration according to claim 1 is characterized in that: When each robot generates a semantic-frontier viewpoint, the surrounding environment image obtained by the robot is used to extract the semantic information of the image in the frontier direction through a pre-trained network, and the semantic information is used as the information gain to measure the frontier, and is combined with the frontier viewpoint to obtain the semantic-frontier viewpoint.

3. The unknown scene exploration method for multi-robot collaboration according to claim 1 is characterized in that: When generating a local path for the semantic-frontier viewpoint by solving the traveling salesman problem, the end point is selected as the semantic-frontier viewpoint with the largest information gain.

4. The unknown scene exploration method for multi-robot collaboration according to claim 1 is characterized in that: In the local exploration phase, each robot updates the position, viewing angle, corresponding robot number and semantic-frontier viewpoint information gain of the unexplored frontier viewpoint at the boundary of the local exploration area to the leaf node of the topological graph.

5. The unknown scene exploration method for multi-robot collaboration according to claim 1 is characterized in that: The specific method of constructing a dynamic topology map is as follows: each robot starts to explore the unknown environment from the initial point, and at the same time constructs a topology map T = (V, E), where V is the node of the topology map, E is the edge connecting the nodes, the root node of the topology map is the initial position of the robot, and the nodes of the topology map are divided into internal nodes and leaf nodes. The internal node is the center point of the explored area. When the local exploration area is completed, it will be added to the topology map; the leaf node is the front viewpoint at the boundary of the local exploration area, which constitutes the exploration frontier of all robots. The front viewpoint contains four pieces of information, namely position, viewing angle, semantic information gain and robot number.

6. The unknown scene exploration method for multi-robot collaboration according to claim 5 is characterized in that: The dynamic update process of the topology map includes: When the i-th robot has finished exploring the current local exploration area, the edges of the topological graph connect the explored areas; For several robots that have explored the local exploration area, a target point is selected from all leaf nodes of the topological graph by solving the multi-traveling salesman problem, and the target point is removed from the topological graph. The robot then enters a new local exploration area for exploration. When the robot completes the exploration of a new local exploration area, the center point of the new local exploration area is added to the topological graph, and an edge is added. At the same time, the frontier viewpoint currently at the boundary of the local exploration area is updated to a new leaf node.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the unknown scene exploration method for multi-robot collaboration as described in any one of claims 1-6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the unknown scene exploration method for multi-robot collaboration as described in any one of claims 1 to 6 are implemented.

Images