Multi-robot cooperation-oriented unknown scene exploration method

Abstract

The invention relates to a multi-robot cooperation-oriented unknown scene exploration method, which specifically comprises a local exploration stage and a global exploration stage, in the local exploration stage, each robot explores in a local exploration area, each robot generates a semantic-leading viewpoint, and each robot generates a semantic-leading viewpoint; aiming at the semantic-leading viewpoint, solving a traveling salesman problem to generate a local path; in the global exploration stage, a dynamic topological graph is constructed, the semantic-leading viewpoint is stored to leaf nodes of the topological graph and shared to all robots, when each robot leaves the local exploration area, the leaf nodes in the topological graph are considered, a global path is generated by solving a multi-traveling salesman problem, and the global path is obtained. And the robots are coordinated to enter a new local exploration area nearby, so that multi-robot collaborative exploration is realized. According to the invention, efficient multi-robot collaborative exploration can be realized.

Description

Technical Field

[0001] The present invention relates to the field of robot autonomous exploration technology, and in particular to an unknown scene exploration method oriented to multi-robot collaboration. Background Art

[0002] Robotic autonomous exploration is to plan and control the movement of robots to fully and accurately model the surrounding environment, which usually relies on Simultaneous Localization and Mapping (SLAM) technology. In the exploration of unknown scenes, compared with a single robot, multi-robot collaborative exploration can give full play to their numerical advantages, expand the exploration range and improve the exploration efficiency.

[0003] Among the existing single-robot and multi-robot autonomous exploration methods, LiDAR and depth cameras are two mainstream sensors, and the types of maps used for exploration are mostly grid maps or voxel maps. There are two classic exploration methods based on this type of map, namely frontier-based and sample-based methods. In recent years, a method combining frontier and sampling has emerged and gradually become the mainstream method. Its characteristic is that a set of frontier viewpoints are used to cover the frontier composed of grids or voxels, and the robot completes autonomous exploration by traversing each frontier viewpoint.

[0004] However, the selection of the frontier viewpoint in this method often only depends on the geometric information in the scene. For example, for a robot using a lidar for exploration, its frontier is composed of discrete point clouds, and the density of the point cloud (i.e., the resolution of the lidar) directly affects the quality of the frontier representation. Therefore, the frontier viewpoint is tightly coupled with the quality of point cloud generation. If the sensor resolution is low or there are dynamic obstacles blocking the frontier, the frontier representation is incomplete, which will lead to inaccurate or even missing frontier viewpoints. This will cause the robot to backtrack and repeatedly explore an area, reducing the exploration efficiency. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an unknown scene exploration method for multi-robot collaboration, which can realize efficient multi-robot collaborative exploration.

[0006] The technical solution adopted by the present invention to solve its technical problems is: to provide an unknown scene exploration method for multi-robot collaboration, divide the area to be explored into several sub-areas of fixed shapes, each sub-area of ​​fixed shape is used as a local exploration area, and after completing the exploration of the local exploration area, the robot transfers to a new local exploration area for exploration until the exploration of all local exploration areas is completed. Specifically, it includes a local exploration stage and a global exploration stage. In the local exploration stage, each robot explores in the local exploration area, each robot generates a semantic-frontier viewpoint, and generates a local path for the semantic-frontier viewpoint by solving the traveling salesman problem; in the global exploration stage, by constructing a dynamic topological map, the semantic-frontier viewpoint is stored in the leaf node of the topological map and shared with all robots. When each robot leaves the local exploration area, the leaf node in the topological map is considered, and a global path is generated by solving multiple traveling salesman problems, and each robot is coordinated to enter a new local exploration area nearby, so as to realize multi-robot collaborative exploration.

[0007] When each robot generates a semantic-frontier viewpoint, the surrounding environment image acquired by the robot is used through a pre-trained network to extract the semantic information of the image in the frontier direction, and the semantic information is used as the information gain to measure the frontier, and is combined with the frontier viewpoint to obtain the semantic-frontier viewpoint.

[0008] When generating a local path by solving the traveling salesman problem for the semantic-frontier viewpoint, the end point selects the semantic-frontier viewpoint with the largest information gain.

[0009] In the local exploration phase, each robot updates the position, viewing angle, corresponding robot number and semantic-frontier viewpoint information gain of the frontier viewpoint that is at the boundary of the local exploration area and has not been explored to the leaf node of the topological graph.

[0010] The specific method of constructing a dynamic topological map is as follows: each robot starts to explore the unknown environment from an initial point, and constructs a topological map T = (V, E), wherein V is a node of the topological map, and E is an edge connecting the nodes. The root node of the topological map is the initial position of the robot. The nodes of the topological map are divided into internal nodes and leaf nodes. The internal node is the center point of the explored area. When the local exploration area is completed, it will be added to the topological map; the leaf node is a frontier viewpoint at the boundary of the local exploration area, which constitutes the exploration frontier of all robots. The frontier viewpoint contains four pieces of information, namely, position, viewing angle, semantic information gain and robot number.

[0011] The dynamic update process of the topology map includes:

[0012] When the i-th robot has finished exploring the current local exploration area, the edges of the topological graph connect the explored areas;

[0013] For several robots that have explored the local exploration area, by solving the multi-traveling salesman problem, a target point is selected from all leaf nodes of the topological graph, and the target point is removed from the topological graph, and the robot enters a new local exploration area for exploration;

[0014] When the robot completes the exploration of a new local exploration area, the center point of the new local exploration area is added to the topological graph, and edges are added. At the same time, the frontier viewpoint currently at the boundary of the local exploration area is updated to a new leaf node.

[0015] The technical solution adopted by the present invention to solve its technical problem is: to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned unknown scene exploration method for multi-robot collaboration when executing the computer program.

[0016] The technical solution adopted by the present invention to solve its technical problem is: providing a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned unknown scene exploration method for multi-robot collaboration are implemented.

[0017] Beneficial Effects

[0018] Due to the adoption of the above technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention installs a camera on the robot to obtain image information of the robot's surrounding environment, extracts semantic information in the frontier direction through a pre-trained network, uses the semantic information as the information gain to measure the frontier, and combines it with the frontier viewpoint to generate a semantic-frontier viewpoint, which effectively solves the inaccuracy caused by relying solely on geometric information; in addition, the present invention stores the semantic-frontier viewpoints through the leaf nodes of the topological tree, and all robots can share these frontier viewpoints. By solving the multi-traveling salesman problem (MTSP), each robot collaboratively explores these shared frontier viewpoints, giving priority to exploring the frontier with large information gain, thereby improving the efficiency of multi-machine collaborative exploration. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a framework diagram of the unknown scene exploration method for multi-robot collaboration according to the first embodiment of the present invention;

[0020] Figure 2 is a screening graph of semantic-frontier viewpoints in the first embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram of a topological diagram in the first embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0023] The first embodiment of the present invention relates to an unknown scene exploration method for multi-robot collaboration, such as Figure 1 As shown, multiple robots start from an initial point to explore the area to be explored, and the area to be explored is divided into multiple square areas of equal area, where each square area represents a local exploration area. After completing the exploration of a local exploration area, the robot moves to a new local exploration area for exploration until the exploration of all local exploration areas is completed.

[0024] This embodiment adopts a hierarchical exploration method, which is divided into a local exploration stage and a global exploration stage. The local exploration stage is responsible for the exploration in the local exploration area. Each robot generates a semantic-frontier viewpoint and generates a local path for the semantic-frontier viewpoint by solving the traveling salesman problem. The global exploration stage builds a dynamic topological map, stores the semantic-frontier viewpoint in the leaf node of the topological map, and shares it with all robots. When each robot leaves the local exploration area, it will simultaneously consider the leaf nodes in the topological map, generate a global path by solving the multi-traveling salesman problem, coordinate each robot to enter the new local exploration area nearby, and realize multi-robot collaborative exploration.

[0025] Since it is difficult to accurately characterize the frontier and frontier viewpoints by relying solely on the geometric information of the sensor, in order to overcome this problem, this embodiment installs a camera on the robot, and obtains image information of the robot's surrounding environment through the camera. The acquired surrounding environment image is used through a pre-trained network to extract the semantic information of the image in the frontier direction, and the semantic information is used as the information gain to measure the frontier, and combined with the frontier viewpoint to obtain the semantic-frontier viewpoint. Compared with the traditional frontier viewpoint that only relies on geometric information, it uses the semantic information in the frontier direction as the information gain of the frontier viewpoint, so that the selection of the frontier viewpoint has priority, which is of great significance for exploration. Therefore, for the semantic-frontier viewpoint, when the robot obtains a local path by solving the traveling salesman problem, the end point will select the frontier viewpoint with the largest information gain, such as Figure 2 As shown in the figure, the exploration of the local area is completed. For the frontier viewpoints that are at the boundary of the local exploration area and have not been explored, the robot updates its position, viewing angle, corresponding robot number, and semantic-frontier viewpoint information gain to the leaf nodes of the topology graph.

[0026] The topological map constructed in this embodiment is based on the regional decomposition of the unknown scene: in order to avoid multiple robots from repeatedly exploring an area and give full play to the advantages of multi-robot collaborative exploration, the area to be explored is subdivided into square areas of equal area, and the square areas represent local exploration areas, such as Figure 3 In the local exploration area, the robot will complete the exploration of the area as much as possible alone, and will enter the next local exploration area only after exploring the local exploration area, thus ensuring the complete exploration of the area. Figure 3 In the figure, the yellow squares are explored areas, the gray squares are unexplored areas, and the center point of each area is p = (x, y).

[0027] The process of constructing the topology map in this embodiment is as follows: Figure 3 Taking the collaborative exploration of three robots as an example, first, multiple robots start to explore the unknown environment from the initial point, and at the same time build a topological map T = (V, E), where V is the node of the topological map, E is the edge connecting the nodes, and the root node of the topological map is the initial position of the robot. The nodes of the topological map are divided into internal nodes and leaf nodes, and their characteristics are: 1) The internal node is the center point v of the explored area inter = (robot_id, p), when the local exploration area is completed, it will be added to the topology map; 2) The leaf node is the frontier viewpoint v at the boundary of the local exploration area leaf , which constitutes the exploration frontier of all robots. The frontier viewpoint v leaf Contains four pieces of information, including position p = (x, y), viewing angle θ, semantic information gain I and robot number robot_id.

[0028] The dynamic update process of the topological map represents the collaborative exploration process of multiple robots in the global exploration layer, as follows:

[0029] 1) When the i-th robot has finished exploring the current local exploration area, the edges of its topological graph connect the explored areas. At this time, the topological graph represents the explored areas and all current frontier viewpoints;

[0030] 2) For several robots that have explored the local exploration area, the target points are selected from all leaf nodes of the topological graph by solving the multi-traveling salesman problem, and the selected target points are removed from the topological graph. The robots enter the new local exploration area for exploration through the generated global planning path;

[0031] 3) When the robot completes the exploration of the new local exploration area, the center point of the local exploration area is added to the topological map and an edge is added. At the same time, the frontier viewpoint currently at the boundary of the local exploration area is updated to a new leaf node.

[0032] It is worth noting that the frontier viewpoint of the robot is changing dynamically during the exploration process, which will also affect the choice of exploration area for each robot. The dynamic update of the topological map can effectively cope with such dynamic changes. Through the continuous updating of the above topological map, multiple robots can gradually complete the exploration of unknown environments.

[0033] It is not difficult to find that the nodes of the topological graph in this embodiment represent the explored areas and frontier viewpoints, and the node update process reflects the collaborative exploration of multiple robots. The directed edges represent the exploration order of the local exploration area. Through the topological graph structure, this embodiment allows all robots to share all the frontier viewpoints, thereby achieving efficient collaborative exploration.

Claims

1. A method for exploring unknown scenes for multi-robot collaboration, characterized in that: The area to be explored is divided into several sub-areas of fixed shapes, each of which is used as a local exploration area. After completing the exploration of the local exploration area, the robot moves to a new local exploration area for exploration until the exploration of all local exploration areas is completed. Specifically, it includes a local exploration stage and a global exploration stage. In the local exploration stage, each robot explores in the local exploration area, generates a semantic-frontier viewpoint, and generates a local path for the semantic-frontier viewpoint by solving the traveling salesman problem; in the global exploration stage, by constructing a dynamic topological map, the semantic-frontier viewpoint is stored in the leaf node of the topological map and shared with all robots. When leaving the local exploration area, each robot considers the leaf node in the topological map, generates a global path by solving the multi-traveling salesman problem, coordinates each robot to enter the new local exploration area nearby, and realizes multi-robot collaborative exploration.

2. The unknown scene exploration method for multi-robot collaboration according to claim 1 is characterized in that: When each robot generates a semantic-frontier viewpoint, the surrounding environment image acquired by the robot is used through a pre-trained network to extract the semantic information of the image in the frontier direction, and the semantic information is used as the information gain to measure the frontier, and is combined with the frontier viewpoint to obtain the semantic-frontier viewpoint.

3. The unknown scene exploration method for multi-robot collaboration according to claim 1 is characterized in that: When generating a local path by solving the traveling salesman problem for the semantic-frontier viewpoint, the end point selects the semantic-frontier viewpoint with the largest information gain.

4. The unknown scene exploration method for multi-robot collaboration according to claim 1 is characterized in that: In the local exploration phase, each robot updates the position, viewing angle, corresponding robot number and semantic-frontier viewpoint information gain of the frontier viewpoint that is at the boundary of the local exploration area and has not been explored to the leaf node of the topological graph.

5. The unknown scene exploration method for multi-robot collaboration according to claim 1 is characterized in that: The specific method of constructing a dynamic topological map is as follows: each robot starts to explore the unknown environment from an initial point, and constructs a topological map T = (V, E), wherein V is a node of the topological map, and E is an edge connecting the nodes. The root node of the topological map is the initial position of the robot. The nodes of the topological map are divided into internal nodes and leaf nodes. The internal node is the center point of the explored area. When the local exploration area is completed, it will be added to the topological map; the leaf node is a frontier viewpoint at the boundary of the local exploration area, which constitutes the exploration frontier of all robots. The frontier viewpoint contains four pieces of information, namely, position, viewing angle, semantic information gain and robot number.

6. The unknown scene exploration method for multi-robot collaboration according to claim 5 is characterized in that: The dynamic update process of the topology map includes: When the i-th robot has finished exploring the current local exploration area, the edges of the topological graph connect the explored areas; For several robots that have explored the local exploration area, by solving the multi-traveling salesman problem, a target point is selected from all leaf nodes of the topological graph, and the target point is removed from the topological graph, and the robot enters a new local exploration area for exploration; When the robot completes the exploration of a new local exploration area, the center point of the new local exploration area is added to the topological graph, and edges are added. At the same time, the frontier viewpoint currently at the boundary of the local exploration area is updated to a new leaf node.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the unknown scene exploration method for multi-robot collaboration as described in any one of claims 1-6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the unknown scene exploration method for multi-robot collaboration as described in any one of claims 1 to 6 are implemented.

Images