A multi-robot collaborative exploration method with limited communication

Abstract

This paper discloses a multi-robot collaborative exploration method under communication-constrained conditions. This method takes into account the limitations of real-world communication networks, specifically addressing these limitations in the exploration decision-making phase and the efficient representation of explored area information. The robots intermittently return to a designated location to complete their previous mission and proceed with the next, without relying on an additional central base station. A star-convex-based representation of the explored area is also proposed, significantly reducing the required communication bandwidth while maintaining accurate spatial representation.

Description

Technical Field

[0001] The present invention relates to the field of robot communication technology, and in particular to a multi-robot collaborative exploration method with limited communication. Background Art

[0002] Currently, robotic exploration systems are widely used in mining exploration, emergency rescue, agricultural monitoring, geographic mapping, and other fields. Faced with increasingly complex and large-scale tasks, multi-robot collaborative exploration offers superior performance, better fault tolerance, and reduced uncertainty compared to individual robots. Multiple robots have a strong need for information sharing. However, in the real world, especially in large-scale environments, global communication is often not possible, and large amounts of sensor data can exceed the capacity of communication networks.

[0003] Combined with the research published by Burgard, Andre et al. in the top journals of robotics, we can conclude that the multi-robot collaborative exploration method is generally as follows: Figure 1 In the process shown, a multi-robot system consists of N robots (N ≥ 2). Each robot uses its own sensors to perceive the environment, collects environmental information, and constructs a map. From this map, it extracts information about the frontier and explored space. Frontier information represents the boundary between known, obstacle-free space and unknown space. Setting the frontier in the current space as the robot's target drives the robot to explore the unknown environment. Robots connect to each other via a communication network and share information. Typically, a host is selected from among all connected robots to aggregate information from all robots, assign tasks based on this information, and distribute them to the robots.

[0004] Most current multi-robot collaborative exploration methods make idealized assumptions about communication networks. However, in the real world, the environments being explored often lack global communication guarantees, and inter-robot communication is subject to significant distance and bandwidth limitations. Distance limitations impact task allocation, as multiple robots must maintain communication to share information while completing exploration tasks. Bandwidth limitations also impact the representation of information about the explored environment. Frontier information itself is relatively small and is largely unaffected by bandwidth. To provide the context for decision-making by the host, a lightweight representation of the explored environment is necessary to reduce bandwidth usage during transmission.

[0005] There are two main difficulties in multi-robot collaborative exploration methods under communication-constrained conditions: (1) multi-robot collaborative exploration decision-making with limited communication distance; (2) efficient representation of explored environment information with limited communication bandwidth.

[0006] At present, the decision-making schemes for multi-robot exploration with (1) limited communication distance are: a) scheme that does not require communication maintenance; b) scheme that requires continuous communication; c) scheme that requires intermittent communication maintenance. Among them, the scheme of type a will make the system have greater uncertainty. The information sharing between multiple robots is sporadic and opportunistic. Task allocation based on bidding or market method is only carried out when the robots can communicate with each other. This method cannot guarantee that the robots can always communicate with each other. Once the communication range is exceeded, it is difficult to re-establish communication. It cannot guarantee that information is effectively shared among multiple robots, which may lead to repeated exploration and unprofitable energy consumption. The scheme of type b requires that multiple robots maintain complete communication connection with each other during the entire exploration process and share data at any time. The robots will explore in groups and are only allowed to select the target points to be explored within the communication range. This will introduce strong decision-making restrictions, resulting in a large area of repeated exploration and limited improvement in exploration efficiency. The method of type c allows robots to actively disconnect and seek to reconnect after completing part of the exploration. Currently, this type of method mostly has a central base station and is role-based. That is, in the multi-robot system, some robots act as explorers to collect environmental information, and some robots act as transmitters to transmit information between explorers and the central base station. However, the base station requirements still limit the flexibility of the system.

[0007] Traditional environmental information representation methods include: grid maps, octree maps, and point cloud maps. These maps are rich in information, but the data volume is large. Once the environment range is too large, it is easy to exceed the communication bandwidth, which is not conducive to data sharing. For (2) environmental information representation methods under limited communication bandwidth, there are: a. Gaussian mixture model representation; b. Convex hull representation. Among them, a can represent obstacles in the environment with a high compression rate, but because it does not have the ability to record obstacle-free space, it is necessary to reconstruct the obstacle-free area when making exploration decisions. The reconstruction process is computationally intensive and has low accuracy. b uses the union of a series of convex polyhedrons to represent the obstacle-free space in the environment. The amount of data required for transmission is small, and it contains information on whether the area has been explored, which is more suitable for exploration decisions. In order to make the convex hull representation more powerful, a variant of the convex hull, star convex, was proposed. The convex restriction of star convex is slightly weaker than that of the convex hull, and it can represent the environment more accurately. However, this technology has not yet been used in multi-robot exploration systems.

[0008] In general, the current multi-robot collaborative exploration method under communication-restricted conditions is subject to the above two difficulties. Without the introduction of additional information-assisted equipment, such as a central base station, it is not possible to maintain a certain level of communication connectivity while avoiding large-scale repeated exploration and improving exploration efficiency. Summary of the Invention

[0009] In response to the shortcomings of the existing technology, the present invention aims to provide a multi-robot collaborative exploration method with limited communication, which solves the problem of multi-robot collaborative exploration under the communication distance limitation and communication bandwidth limitation in the real world. It is applicable to any model of multi-rotor, fixed-wing or hybrid-wing UAV, ground robot, and can be used to explore unknown environments under the conditions of limited communication bandwidth and communication distance.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] A multi-robot collaborative exploration method under communication-constrained conditions, wherein each robot has an environment information database for storing environment information and two planners, an independent planner and a joint planner.

[0012] When the robots start, they gather together and can communicate with each other. This is the joint assembly phase. During the joint assembly phase, the joint planners of each robot are triggered to plan. The robots will select a host and upload all the information in their environmental information database to the host. After integrating the information, the host will assign tasks, assigning each robot the target points to visit and the final agreed location and time for the next assembly. At this point, the joint assembly phase ends, and each robot will actively disconnect and enter the independent exploration phase. During the independent exploration phase, each robot triggers an independent planner to plan, collect new environmental information through sensors, generate new barrier-free space and frontier information, and update the old environmental information. Then, independent planning is carried out based on the assigned target points and the final assembly location and time as constraints, generating a new target sequence. At the agreed assembly time, each robot arrives at the agreed joint assembly location to share environmental data, and then enters the joint planning phase again from the independent exploration phase.

[0013] Furthermore, all the environmental data generated each time the independent planner is triggered is called a frame of data. The planning process of the independent planner is:

[0014] Step A1, sampling obstacle points: Sample obstacle points to generate a point set S for generating star convexity star ; Point set S used to generate star convex star It needs to include two parts: the currently observed obstacle point S obs and the sampling point set S on the maximum observation radius of the sensor free ;

[0015] Step A1.1: Generate obstacle point cloud S obs : Sample the local occupancy grid map generated by fusing multi-frame sensor data to generate an obstacle point cloud S that can be observed at the current robot position obs; Take the current robot position P r is the origin of the cylindrical coordinate system, at multiple radii Perform sampling at different densities, where n represents the number of different radii used, R senor is the maximum sensing radius of the sensor; for each sampling point P s , from P r As a starting point, The direction of the light projection is to be set. If it collides with an obstacle during the projection process, the collision point is recorded as P hit , then P hit Join S obs gather;

[0016] Step A1.2: Generate the maximum observation area point cloud S free :When sampling the cylindrical coordinate system in step A1.1, when the sampling radius r=R sen When the light projection does not collide with the obstacle, the sampling point P s Join S free point set;

[0017] Step A2: Star convex generation: Use the point set generated in step A1 to form the point set S for generating star convex star =S obs ∪S free , and use the HPR method to calculate the star convexity; the star convexity S of the current frame k and the current robot position R k Record it in the environment information database; the star convex of the kth frame is recorded as S k , then the obstacle-free space that the current robot has explored is expressed as S1∪S2∪...∪S N , where N is the number of spatial convexities recorded in the current robot's environment information;

[0018] Step A3: Update the frontier surface element: Since the known obstacle-free area is represented by a star convex, the surface elements that make up the star convex are the boundaries of the known obstacle-free area. As long as the star convex surface elements generated entirely by obstacles are removed, the remaining surface elements are the boundaries between the known obstacle-free space and the unknown space, which are called frontier surfaces.

[0019] Step A3.1. Generate the frontier surface element of the current frame: All the surfaces are judged. If the surface element meets any of the following conditions, the surface element is not a frontier surface element: a. All vertices of the surface element belong to S obs ; b. The area of the bin is less than the minimum effective area threshold η area c. The absolute value of the z-axis component of the normal vector of the face element is greater than the z-axis component threshold z thr ; After filtering all the facets of the current frame, the frontier facet set F of the current frame will be obtained k ;

[0020] Step A3.2, inter-frame update of the frontier surface element: Assuming that the current frame is the k-th frame, this step will complete two aspects of updates: (1) Delete the frontier surface element {S1, S2, ..., S k-1}, update F k ; (2) Delete the star convex S in the current frame k The frontier surface element of the previous frame in the update {F1, F2, ..., F k-1};

[0021] Step A4: Frontier surface element clustering: cluster the frontier surface elements, generate a corresponding viewpoint for each frontier cluster, and fuse the viewpoints with close distances into a super viewpoint.

[0022] Furthermore, the sampling process of step A1.1 is adaptive to the obstacle density. For two adjacent sampling points P i , P j , will be in P i , P j The intermediate angles are sampled recursively until one of the following conditions is met:

[0023]

[0024] ||P i -P j ||<D thr .

[0025] Furthermore, in step A3.2, a facet table is created for each star convex, and the facet table is used to query whether a facet is in a star convex;

[0026] 1) The process of establishing the facet table is as follows:

[0027] 1.1) Create the object of the facet table: Convex S i All the face elements of are projected onto a gridded unit ball, and each grid on the unit ball is denoted as g i，j ; For a projected surface element f i , we can get its axial boundary rectangle on the unit sphere, which will be in the B i All grids in are denoted as G B ={g i，j |g i，j ∈B i};

[0028] 1.2) G B The grid and f i The corresponding relationship is recorded in the facet table; each grid on the final gridded unit sphere will correspond to one or more facets;

[0029] (2) The process of using the facet table for quick query is as follows:

[0030] For the face element f to be queried i Hexing Convex S j , first f i The center of the projection is projected onto the star convex S j The corresponding robot position R j On the grid unit sphere with the center as the sphere, the projection point falls on g i，j In the grid, in S j Query g in the facet table i，j The corresponding facet {f′ j |j∈{1,2,...,J}},a total of J; each face element f′ found j The three vertices of R j Connect to form a triangular pyramid, forming a total of J triangular pyramids, if f i The center of is in one of the triangular pyramids, it is judged to be f i In the Star Convex S j middle.

[0031] Furthermore, the specific process of step A4 is as follows:

[0032] Step A4.1. Spectral clustering of frontier facets: Spectral clustering requires constructing two matrices: the degree matrix D and the similarity matrix S. For the degree matrix D, connect all facets with their k nearest neighbors by Euclidean distance to form a graph, and calculate the degree matrix of this graph as D. For the similarity matrix S, calculate it as follows:

[0033] S=(exp(-s(f i , f j ) 2 / 2σ 2 )) i，j＝1...J

[0034] Among them, s(f i , f j ) is f i , f j The total distance between two surface elements, s(F i , F j )=||[d t , d n , δ n ]|| ω , σ is the parameter of the preset Gaussian function; d t , d n , δ n There are three distances to measure similarity, d t (F i , Fj )=||(c i -c j )·n j ||2 is the tangential distance, d n (F i , F j )=||(c i -c j )×n j ||2 is the normal distance, δ n (F i , F j )=||(n i -n j ) / 2||2 is the normal angle difference;

[0035] After calculating D and S, the frontier clusters can be obtained through spectral clustering;

[0036] Step A4.2, viewpoint selection: Generate a corresponding optimal viewpoint for each frontier cluster. When generating the viewpoint, in order to determine the optimal position of the viewpoint relative to the frontier cluster, the frontier cluster is first modeled as an elliptical surface; its position is described by three mutually orthogonal unit vectors Where N0 is equal to the vector of the normal vectors of all the facets in the cluster after normalization; then by performing PCA principal component analysis on the center points of all the facets in the cluster, an eigenvector matrix N is obtained. PCA and the corresponding eigenvalue vector λ PCA = [λ0, λ1, λ2], the eigenvalues in the vector are stored in descending order; N PCA The first and second columns of the matrix are projected onto the plane determined by N0, respectively, to obtain N1 and N2; within a certain distance range from the elliptical surface, a series of sampling points are generated at an angle relatively perpendicular to the surface:

[0037]

[0038] Among them, VP is the generated sampling point, C is the cluster center, R i is the observation distance of the sampling; ω=[1, ω1, ω2], where ω1∈[0, ω max ], Different sampling points can be obtained by selecting different ω, where ω1 and ω2 represent the weights of vectors N1 and N2 when generating sampling points, respectively. First, a sampling viewpoint is selected in the direction perpendicular to the ellipsoid. If the sampling viewpoint is within the known obstacle-free space, it is selected as the viewpoint corresponding to the cluster. Otherwise, sampling is continued until a viewpoint within the known obstacle-free space is sampled. If no viewpoint that meets the requirements can be found in the sampling space, the cluster is considered unobservable and is directly deleted from the environment information.

[0039] Step A4.3, super viewpoint generation: A super viewpoint is generated from the viewpoints contained in the same sphere with a given threshold radius; the super viewpoint will serve as the target point that the robot needs to reach in the exploration decision.

[0040] Furthermore, the planning process of the joint planner is as follows:

[0041] Step B1, Moderator Selection: The rules for determining the moderator robot are as follows: a. The robot determines the moderator of a new meeting by detecting the timestamps of other robots; b. When a robot enters a meeting led by a moderator, it will follow the other existing participants and remain the moderator; c. If the moderator leaves the meeting, the remaining participants will re-determine the moderator based on the timestamp;

[0042] Step B2, information integration: The robots participating in the assembly upload all environmental information in the environmental information database to the host. Information integration is divided into two aspects: (1) integration of barrier-free space information; (2) integration of frontier information; for (1), all barrier-free space information of all robots are directly aggregated, and the union of all spatial convexities is all known barrier-free spaces; for (2), the frontier face element of each robot needs to be checked to see if it is in the spatial convexity of other robots. If so, the frontier face element is deleted; in order to find the convexity that needs to be checked corresponding to any frontier face element, a KD-tree is established for the key pose points corresponding to all spatial convexities, and all key poses of the sphere with the face element center as the origin and the maximum sensor radius as the radius are found, and then the face element is checked to see if it is in the convexity corresponding to these key poses; in the process of information integration, if all face elements in a frontier cluster are deleted, the frontier cluster and its corresponding viewpoint are deleted; if all viewpoints in a super viewpoint are deleted, the super viewpoint is deleted;

[0043] Step B3, centralized planning: All superviewpoints after information integration are assigned to each robot as the target points to be reached. A time and place for the next meeting is agreed upon. After passing through the assigned superviewpoints, the robots return to the meeting place for a joint meeting. Each superviewpoint is assigned to only one robot. The cost between two points is defined as:

[0044]

[0045] Among them, v max is the maximum speed of the robot;

[0046] The position of each robot is denoted as R c = {1, ..., n}, each super viewpoint position is recorded as S c ={n+1, ..., m}, define three 0-1 decision variables: If and only if robot k chooses to go from point i to point j, the value is 1; The value of node i is 1 if and only if it is passed by robot k; (3) t i :The value is 1 if and only if node i is selected as the next meeting place; then the centralized decision planning problem can be expressed as:

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056] Among them, N c =R c ∪S c , d ij =T m (p i , p j )

[0057] Before solving the above problem, first select the next gathering location, that is, determine t i ;

[0058] After the rendezvous location is determined, the decision-making planning problem is transformed into a vehicle routing problem. First, a heuristic function is used to search for an initial path, and then a meta-heuristic method is used to search for a local path. Starting from the robot's position, the path is extended by iteratively adding arcs with the minimum cost. This method obtains an initial solution. Finally, an extension-guided local search algorithm is used to find an improved solution. At this point, the next rendezvous location P is determined. c and the path of each robot Ψ = {1, ..., n}; then determine the assembly time and use the solution as the base time T b , set extra time for the robot to explore freely T e , the last unfinished meeting place and time of robot k is T cur is the current time, then the next meeting time is:

[0059]

[0060] On this basis, after tasks and paths are assigned to robots, they disperse to explore independently.

[0061] Furthermore, the node with the largest distance to all robots is selected as the gathering location.

[0062] Furthermore, as the environment is explored and new superviewpoints are generated, each robot continuously replans its path through an independent planner to pass through each superviewpoint and arrive at the next gathering point on time. In order to reduce repeated exploration, a penalty is introduced for the area allocated to other robots for exploration. For robot r, its position is recorded as P r , and record the next meeting place as P m , the super viewpoint set is denoted as S d , then define the penalty a of node i i for

[0063]

[0064] Among them, v is the path assigned to robot v; then the planning problem is:

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071] Among them, N d =P r ∪P m ∪S d , T m It's time for the next gathering, T cur is the current moment; when the robot replans, it performs a local search based on the previous solution to generate a new solution.

[0072] The beneficial effects of this invention lie in its consideration of the limitations of real-world communication networks, specifically addressing these limitations in the exploration decision-making process and the efficient representation of explored area information. The robot will intermittently return to a designated location to complete the previous mission and proceed to the next, without relying on an additional central base station. Furthermore, a star-convex-based representation of the explored area is proposed, significantly reducing the required communication bandwidth while maintaining accurate spatial representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A flow chart of a method of the prior art;

[0074] Figure 2 is an overall flow chart of the method in an embodiment of the present invention;

[0075] Figure 3 This is an example diagram of multi-robot collaborative exploration in an embodiment of the present invention;

[0076] Figure 4 Schematic diagram of a planning method for an independent planner according to an embodiment of the present invention;

[0077] Figure 5 Schematic diagram of the process of generating barrier-free space and frontier information in an embodiment of the present invention;

[0078] Figure 6 Schematic diagram of the facet table method and result diagram in an embodiment of the present invention;

[0079] Figure 7 This is the frontier facet clustering method and results in an embodiment of the present invention;

[0080] Figure 8 Schematic diagram of a frontier surface element clustering viewpoint selection method according to an embodiment of the present invention;

[0081] Figure 9 Schematic diagram of planning of a joint planner in an embodiment of the present invention;

[0082] Figure 10 Schematic diagram of the host information integration steps in an embodiment of the present invention. DETAILED DESCRIPTION

[0083] The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to this embodiment.

[0084] This embodiment provides a multi-robot collaborative exploration method under communication-limited conditions. The method flow is as follows: Figure 2As shown in Figure 2, each robot has an environment information repository for storing environmental information and two planners: an independent planner and a joint planner. Accordingly, the multi-robot collaborative exploration process is divided into two phases: the independent exploration phase and the joint assembly phase, executed by the independent planner and the joint planner, respectively.

[0085] The multi-robot collaborative exploration process can be roughly described as follows: upon startup, the robots gather together and can communicate with each other, entering the joint assembly phase. During the joint assembly phase, the robots elect a moderator and upload all information from their environmental databases to the moderator. The moderator then integrates this information and assigns tasks, assigning each robot a target point to visit and a final agreed-upon location and time for the next joint assembly. At this point, the joint assembly phase ends, and each robot proactively disconnects, entering the independent exploration phase. During the independent exploration phase, each robot collects new environmental information through sensors, generates new obstacle-free spaces and frontier information, and updates the existing environmental information. Then, using the assigned target point and the final desired joint assembly location and time as constraints, the robots independently plan their journey, generating a new target sequence. At the agreed-upon joint assembly time, the robots arrive at the agreed-upon joint assembly location to share environmental data, transitioning from the independent exploration phase to the joint planning phase.

[0086] It should be noted that the joint planner is general and can flexibly handle situations where multiple robots encounter each other accidentally. Figure 3 This example shows an example of multi-robot collaborative exploration. There are three robots in this example, which are represented by blocks of different colors in the figure. Figure 3 As shown in (a), first, multiple robots start exploring and enter the first joint gathering stage. After joint planning, they get the time and place of the next joint gathering, that is, Figure 3 (a) is marked with a square with an 'M' in the center. Then, the robots disperse and explore independently. As the environment is explored and new environmental information is generated, each robot continuously uses an independent planner to replan, generating new target points while ensuring that it arrives at the designated joint meeting point on time. Figure 3 In (b), two robots meet by chance during their independent exploration and agree on a meeting of their own (the standard is another M square of a different color). At this time, the two robots have the same task. Then the joint planner of the two robots will adjust the task sequence and leave only one robot with the same unfinished task. This robot returns to Figure 3 The meeting place in (a) and the remaining robot meeting place ( Figure 3 (c)), and then go to the agreed block meeting place with another robot ( Figure 3 (b) The newly generated M blocks) complete data sharing (such as Figure 3 (d)).

[0087] The independent planner and the joint planner are described in detail below.

[0088] like Figure 4 As shown in the figure, the planning process of the independent planner is as follows: in the independent exploration phase, whenever the robot moves a sufficiently long distance from the operating position planned by the previous independent planner, it means that the robot has collected a lot of new environmental information, and a new independent planner planning will be triggered. All environmental data generated each time the independent planner is triggered is called a frame of data, and the environmental data includes obstacle-free space and frontier information. The obstacle-free space and frontier information generation process is as follows Figure 5 shown.

[0089] Step A1, sampling obstacle points: The purpose of this step is to generate a point set S for generating star convex by sampling obstacle points star . Here we first talk about the definition of star convex: star convex is a special polyhedron. If there is a point 0 in the polyhedron S, and the line from point 0 to any vertex of the polyhedron S is contained in S, then the polyhedron S is a star convex. When applied to robots to represent obstacle-free areas, point 0 is the position of the robot, and only recording the face elements that make up the star convex can describe the obstacle-free area contained in the entire star convex. When generating star convex, the hidden point removal (HPR: Hidden Point Removal) method proposed by Katz is used. Given an observation point, this method can generate the visible area of the observation point from the obstacle point cloud, and the visible area is represented by a star convex. Intuitively, if you want to generate an obstacle-free area observed at the current robot position, the point set S used to generate the star convex star It needs to include two parts: the currently observed obstacle point cloud S obs and the sampling point set S on the maximum observation radius of the sensor free .

[0090] Step A1.1: Generate obstacle point cloud S obs : In order to reduce the amount of computation and reduce sensor noise, the local occupancy grid map generated by fusing multiple frames of sensor data is sampled to generate the obstacle point cloud S that can be observed at the current robot position. obs .like Figure 5 As shown in (a), the current robot position P r is the origin of the cylindrical coordinate system, at multiple radii Perform sampling at different densities, where n represents the number of different radii used, R senor is the maximum sensing radius of the sensor. For each sampling point P s , from P r As a starting point, The direction of the light projection is to be set. If it collides with an obstacle during the projection process, the collision point is recorded as P hit , then P hit Join S obs The multi-radius sampling is used to sample obstacles closer to the robot at a higher density to generate more accurate star convex. It is worth noting that the sampling process is adaptive to the obstacle density. For two adjacent sampling points P i , P j , will be in P i , P j The intermediate angles are sampled recursively until one of the following conditions is met:

[0091]

[0092] ||P i -P j ||<D thr

[0093] The meaning of the above formula is: and The angle is less than the angle threshold θ thr , or click P i and P j The distance between them is less than the distance threshold D thr , the sampling recursion stops.

[0094] It should be noted that the local occupancy grid map is not used to generate S obs It is specially generated and its main function is to be used for trajectory planning. Therefore, the use of the local occupancy grid map here does not introduce additional computational complexity. In addition, the local occupancy grid map changes position dynamically and will not be stored for a long time and occupy memory.

[0095] Step A1.2: Generate the maximum observation area point cloud S free :When sampling the cylindrical coordinate system in step A1.1, when the sampling radius r=R senor When the light projection does not collide with the obstacle, the sampling point P s Join S free Point set.

[0096] Step A2: Star convex generation: Use the point set generated in step A1 to form the point set S for generating star convex star =S obs ∪S free , and use the HPR method to calculate the star convexity, such as Figure 5 (b) shows the star convex S of the current frame. k and the current robot position R k Record it in the environment information database. The star convex of the kth frame is S k, then the obstacle-free space currently explored by the robot can be expressed as S1∪S2∪...∪S N , where N is the number of spatial convexities recorded in the current robot's environment information.

[0097] Step A3, frontier surface element update: The frontier in the exploration neighborhood is defined as the boundary between the explored obstacle-free space and the unknown space. Since the star convex is used to represent the known obstacle-free area, the surface elements that make up the star convex are the boundaries of the known area. As long as the star convex surface elements generated entirely by obstacles are removed, the remaining surface elements are the boundary between the known obstacle-free space and the unknown space, which are called frontier surfaces. Figure 5 (c) shown.

[0098] Step A3.1. Generate the frontier surface element of the current frame: All the surface elements are judged. If the surface element meets any of the following conditions, the surface element is not a frontier surface element: a. All vertices of the surface element belong to the obstacle point cloud S obs ; b. The area of the bin is less than the minimum effective area threshold η area c. The absolute value of the z-axis component of the normal vector of the face element is greater than the z-axis component threshold z thr Condition a removes the surface elements generated by obstacles, condition b removes the tiny surface elements that cannot provide effective information, and condition c removes the invalid surface elements outside the sensor's field of view. After filtering all the surface elements of the current frame, the frontier surface element set F of the current frame will be obtained. k .

[0099] Step A3.2, inter-frame update of frontier facets: Assuming the current frame is the k-th frame, this step will complete two aspects of updates: (1) Delete the star convex {S1, S2, ..., S k-1} belongs to the frontier surface element of the current frame, update F k ; (2) Delete the star convex S of the current frame k The frontier pixels belonging to the previous frame, update {F1, F2, ..., F k-1 In order to efficiently perform the above two updates, this embodiment establishes a facet table for each star convex. The facet table can be used to quickly query whether a facet is in a star convex.

[0100] 1) The process of establishing the facet table is as follows:

[0101] 1.1) Create the object of the facet table: Convex S i All the face elements of are projected onto a gridded unit ball, and each grid on the unit ball is denoted as g i，j ; For a projected surface element f i, we can get its axial bounding box (AABB: axis-aligned bounding box) on the unit sphere and record it as B i , that is, Figure 6 The colored part of the grid on the unit ball shown in (a) will be in the B i All grids in are denoted as G B ={g i，j |g i，j ∈B i};

[0102] 1.2) G B The grid and f i The corresponding relationship is recorded in the facet table. Each grid on the final gridded unit sphere will correspond to one or more facets.

[0103] (2) The process of using the facet table for quick query is as follows:

[0104] For the face element f to be queried i Hexing Convex S j , first f i The center of the projection is projected onto the star convex S j The corresponding robot position R j On the grid unit sphere with the center as the sphere, the projection point falls on g i，j In the grid, in S j Query g in the facet table i，j The corresponding facet {f′ j |j∈{1,2,...,J}}, a total of J. Each facet f′ found j The three vertices of R j Connect to form a triangular pyramid, forming a total of J triangular pyramids, if f i The center of is in one of the triangular pyramids, it is judged to be f i In the Star Convex S j middle. Figure 6 (b) shows the query results of whether a point is inside the star convex. In the figure, the square point is judged to be inside the star convex after the query, and the circle point is judged to be outside the star convex.

[0105] Using a facet table allows for rapid inter-frame updates of both aspects of step A3.2. A facet table is created for a star convex the first time it is queried; it is not necessary for subsequent queries. The facet table is extremely small and has little impact on memory usage.

[0106] Step A4, Frontier Surfactant Clustering: To reduce the impact of the number of frontier surfels on decision-making efficiency, frontier surfels are clustered. To better enable the robot to observe the frontier (at a relatively vertical angle at an appropriate distance), a corresponding viewpoint is generated for each frontier cluster. Because the robot's sensor has a sufficiently large sensing range, closely spaced viewpoints are fused into a single superviewpoint.

[0107] Step A4.1. Frontier facet spectral clustering: General clustering methods usually quantitatively describe the attributes of the objects to be clustered, and then cluster them in the attribute space. However, for facets, it is difficult to give their positions in the high-dimensional attribute space, but the similarity between two facets can be simply described: Here, three distances are considered to measure similarity for facet clustering: a. Tangential distance: d t (F j , F j )=||(c i -c j )·n j ||2; b. Normal distance: d n (F i , F j )=||(c i -c j )×nj2; c. Normal angle difference: δnFi, Fj=ni-nj / 22. For clustering problems that can only describe the similarity of clustered objects in a high-dimensional attribute space but cannot describe their positions in the high-dimensional attribute space, spectral clustering methods are usually used. Spectral clustering requires the construction of two matrices: the degree matrix D and the similarity matrix S. For the degree matrix D, this embodiment connects all face elements with their k nearest neighbors in Euclidean distance to form a graph, and calculates the degree matrix of the graph as D. For the similarity matrix S, S is calculated according to the three similarity measurement standards defined above:

[0108] S=(exp(-s(f i , f j ) 2 / 2σ 2 )) i，j＝1...J

[0109] Among them, s(f i , f j ) is f i , f j The total distance between two surface elements, s(F i , F j )=||[d t , d n , δ n ]|| ω, where ω is the weight of the three similarity distances, and σ is the parameter of the preset Gaussian function. After calculating D and S, the frontier clustering can be obtained through spectral clustering. The clustering results are as follows Figure 7 , the numbers are the centers of each cluster, and the same cluster is connected by straight lines.

[0110] Step A4.2, viewpoint selection: In order to observe the frontier cluster at a suitable angle and distance, this embodiment generates a corresponding optimal viewpoint for each frontier cluster. When generating the viewpoint, in order to determine the optimal position of the viewpoint relative to the frontier cluster, the frontier cluster is first modeled as an elliptical surface. Its posture can be described by three mutually orthogonal unit vectors: like Figure 8 (a). Where N0 is equal to the vector of the normal vectors of all the facets in the cluster after normalization. Then, by performing PCA principal component analysis on the center points of all the facets in the cluster, a eigenvector matrix N is obtained. PCA and the corresponding eigenvalue vector λ PCA = [λ0, λ1, λ2], the eigenvalues in this vector are stored in descending order. PCA The first and second columns of the matrix are projected onto the plane determined by N0, respectively, to obtain N1 and N2. Within a certain distance range from the elliptical surface, a series of sampling points are generated at an angle relatively perpendicular to the surface:

[0111]

[0112] Among them, VP is the generated sampling point, C is the cluster center, R i is the observation distance of the sampling. ω=[1,ω1,ω2],where ω1∈[0,ω max ], By choosing different ω, we can get Figure 8 (b) shows different sampling points within the arc surface. ω1 and ω2 represent the weights of vectors N1 and N2 when generating the sampling points, respectively. A sampling viewpoint is first selected in a direction perpendicular to the ellipsoid. If the sampled viewpoint is within the known unobstructed space, it is selected as the viewpoint corresponding to the cluster. Otherwise, sampling continues until a viewpoint within the known unobstructed space is found. If no satisfactory viewpoint can be found within the sampling space, the cluster is considered unobservable and is directly deleted from the environment information.

[0113] Step A4.3, super viewpoint generation: Since the robot sensor has a sufficiently large sensing range, this embodiment fuses the viewpoints that are closer together into a super viewpoint. Figure 5 As shown in (c), the viewpoints contained in the same sphere with a given threshold radius generate a super viewpoint. The super viewpoint will be used as the target point that the robot needs to reach in the exploration decision.

[0114] Step A5: Vehicle Routing Problem (VRP) under mission constraints: This step requires the solution of the joint planner as an initial value and is further described below.

[0115] In this embodiment, Figure 9 As shown in Figure 2, the planning process of the joint planner is as follows:

[0116] Step B1: Moderator Selection: Once multiple robots meet and enter the joint assembly phase, they first need to determine a robot responsible for information integration and centralized decision-making and planning as the moderator. All participating robots will upload the environmental information they have collected from their environment database to the moderator. After the moderator completes information integration and centralized planning, it will transmit the integrated information and planning results to all robots, ending the joint assembly phase. This embodiment designs three rules for determining the moderator robot: a. Each robot obtains the timestamp of other robots entering the meeting through communication. Based on the timestamp, the first robot to enter the meeting is determined as the moderator. This will form a consensus on the moderator among all participating robots; b. When a robot enters a meeting led by a moderator, it will follow the other existing participants and remain the moderator; c. If the moderator leaves the meeting, the remaining participants will re-determine the moderator based on the timestamp.

[0117] Step B2: Information integration, such as Figure 10 As shown: the robots participating in the assembly upload all environmental information in the environmental information library to the host. Information integration is divided into two aspects: (1) integration of barrier-free space information; (2) integration of frontier information. For (1), the barrier-free space information of all robots is directly aggregated, and the union of all spatial convexities is all known barrier-free spaces. For (2), each robot's frontier face element needs to be checked to see if it is in the spatial convexities of other robots. If so, the frontier face element is deleted. In order to find the convexity that needs to be checked corresponding to any frontier face element, a KD-tree is established for the key pose points corresponding to all spatial convexities, and all key poses in the spherical range with the face element center as the origin and the maximum sensor radius as the radius are found. Then, check whether the face element is in the convexity corresponding to these key poses. The inspection method uses the face element table method in step A3.2 for rapid judgment. During the information integration process, if all the facets in a frontier cluster are deleted, the frontier cluster and its corresponding viewpoints will be deleted; if all the viewpoints in a superviewpoint are deleted, the superviewpoint will be deleted as well.

[0118] Step B3, Centralized Planning: This problem is formulated as follows: After integrating information, all superviewpoints are assigned to each robot as target points. A time and location for the next meeting is agreed upon. After passing through the assigned superviewpoints, each robot returns to the meeting location for a joint meeting. Each superviewpoint is assigned to only one robot. This problem is a constrained integer optimization problem. For optimization, the cost between two points is defined as:

[0119]

[0120] Among them, v max is the maximum speed of the robot.

[0121] The position of each robot is denoted as R c = {1, ..., n}, each super viewpoint position is recorded as S c ={n+1, ..., m}, define three 0-1 decision variables: If and only if robot k chooses to go from point i to point j, the value is 1; The value of node i is 1 if and only if it is passed by robot k; (3) t i :The value is 1 if and only if node i is selected as the next meeting place; then the centralized decision planning problem can be expressed as:

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129]

[0130]

[0131] Among them, N c =R c ∪S c , d ij =T m (p i , p j )

[0132] If the problem is solved as a whole, it will be very difficult. Therefore, a hierarchical solution method is proposed, that is, first select the next gathering location, that is, determine t i , and then solve the above problem. The process is:

[0133] (1) Determine the next rendezvous location: There are three options for the next rendezvous location: (1) Farthest rendezvous: Select the node with the greatest distance from all robots as the rendezvous location; (2) Closest rendezvous: Select the node with the smallest distance from all robots as the rendezvous location; (3) Optimal rendezvous: Exhaustively enumerate all possible nodes as rendezvous locations and select the rendezvous location with the smallest J. Method (3) uses an exhaustive enumeration method to enumerate all possible locations and is the optimal solution. This example evaluates the performance of these three methods, comprehensively considering the solution speed and the optimality of the solution, and selects the farthest rendezvous solution.

[0134] (2) Integer programming solution: After the rendezvous location is determined, the decision planning problem is transformed into a vehicle routing problem (VRP). First, the initial path search is performed using a heuristic function, and then the local path search is performed using a metaheuristic method. Starting from the current position of the robot, the path is expanded by iteratively adding arcs with the minimum cost. In this way, an initial solution is effectively obtained. Finally, an extension-guided local search algorithm is used to find an improved solution. At this point, the next rendezvous location P has been determined. c and the path of each robot Ψ = {1, ..., n}; then determine the assembly time and use the solution as the base time T b , set extra time for the robot to explore freely T e , the last unfinished meeting place and time of robot v is T cur is the current time, then the next meeting time is:

[0135]

[0136] On this basis, after the tasks and paths are assigned to the robots, they disperse and explore independently. At the same time, as the environment is explored and new superviewpoints are generated, each robot continuously replans its path, passes through some superviewpoints, and arrives at the next gathering point on time. This step is the replanning method after adding a new superviewpoint (i.e., step A5 above). At the same time, in order to reduce repeated exploration, this embodiment introduces a penalty for the area assigned to other robots for exploration. For robot r, its position is recorded as P r , and record the next meeting place as P m , the super viewpoint set is denoted as S d , then define the penalty a of node i i for

[0137]

[0138] Among them, v is the path assigned to robot v. Then the planning problem is:

[0139]

[0140]

[0141]

[0142]

[0143]

[0144]

[0145] Among them, N d =P r ∪P m ∪S d , T m It's time for the next gathering, T cur is the current moment. The VRP problem under this task constraint can be viewed as a variant of the Capacitated Vehicle Routing Problem (CVRP). When the robot replans, it performs a local search based on the previous solution to generate a new solution.

[0146] Those skilled in the art can make various corresponding changes and modifications based on the above technical solutions and concepts, and all of these changes and modifications should be included in the scope of protection of the claims of the present invention.

Claims

1. A multi-robot collaborative exploration method under communication-constrained conditions, characterized in that: Each robot has an environment information database for storing environment information and two planners, an independent planner and a joint planner. When the robots start, they gather together and can communicate with each other. This is the joint assembly stage. During the joint assembly stage, the joint planners of each robot are triggered to plan. The robots will select a host and upload all the information in their environmental information database to the host. The host will then integrate the information and assign tasks to each robot, assigning the target points to be visited and the final agreed location and time for the next assembly. At this point, the joint assembly stage ends, and each robot will actively disconnect and enter the independent exploration stage. During the independent exploration stage, each robot triggers an independent planner to plan, collect new environmental information through sensors, generate new barrier-free space and frontier information, and update the old environmental information. Then, independent planning is performed with the assigned target points and the final assembly location and time as constraints, generating a new target sequence. At the agreed assembly time, each robot arrives at the agreed joint assembly location to share environmental data, and then enters the joint planning stage again from the independent exploration stage. All the environmental data generated each time the independent planner is triggered is called a frame of data. The planning process of the independent planner is: Step A1, sampling obstacle points: Sample obstacle points to generate a point set S for generating star convexity star ; Point set S used to generate star convex star It needs to include two parts: the currently observed obstacle point S obs and the sampling point set S on the maximum observation radius of the sensor free ; Step A1.1: Generate obstacle point cloud S obs : Sample the local occupancy grid map generated by fusing multi-frame sensor data to generate an obstacle point cloud S that can be observed at the current robot position obs ; Take the current robot position P r is the origin of the cylindrical coordinate system, at multiple radii Perform sampling at different densities, where n represents the number of different radii used, R senor is the maximum sensing radius of the sensor; for each sampling point P s , from P r As a starting point, The direction of the light projection is to be set. If it collides with an obstacle during the projection process, the collision point is recorded as P hit , then P hit Join S obs gather; Step A1.2: Generate the maximum observation area point cloud S free :When sampling the cylindrical coordinate system in step A1.1, when the sampling radius r=R senor When the light projection does not collide with the obstacle, the sampling point P s Join S free point set; Step A2: Star convex generation: Use the point set generated in step A1 to form the point set S for generating star convex star =S obs ∪S free , and use the HPR method to calculate the star convexity; the star convexity S of the current frame k and the current robot position R k Record it in the environment information database; the star convex of the kth frame is recorded as S k , then the obstacle-free space that the current robot has explored is expressed as S1∪S2∪…∪S N , where N is the number of spatial convexities recorded in the current robot's environment information; the HPR method refers to the hidden point removal method proposed by Katz; Step A3: Update the frontier surface element: Since the known obstacle-free area is represented by a star convex, the surface elements that make up the star convex are the boundaries of the known obstacle-free area. As long as the star convex surface elements generated entirely by obstacles are removed, the remaining surface elements are the boundaries between the known obstacle-free space and the unknown space, which are called frontier surfaces. Step A3.

1. Generate the frontier surface element of the current frame: All the surfaces are judged. If the surface element meets any of the following conditions, the surface element is not a frontier surface element: a. All vertices of the surface element belong to S obs ; b. The area of the bin is less than the minimum effective area threshold η area c. The absolute value of the z-axis component of the normal vector of the face element is greater than the z-axis component threshold z thr ; After filtering all the facets of the current frame, the frontier facet set F of the current frame will be obtained k ; Step A3.2, inter-frame update of the frontier facets: Assuming that the current frame is the k-th frame, this step will complete two aspects of updates: (1) Delete the frontier facets {S1, S2, ..., S k-1 }, update F k ; (2) Delete the star convex S in the current frame k The frontier pixels of the previous frame in the update {F1,F2,…,F k-1 }; Step A4: Frontier surface element clustering: cluster the frontier surface elements, generate a corresponding viewpoint for each frontier cluster, and fuse the viewpoints with close distances into a super viewpoint.

2. The method according to claim 1, characterized in that The sampling process of step A1.1 is adaptive to the obstacle density. For two adjacent sampling points P i ,P j , will be in P i ,P j The intermediate angles are sampled recursively until one of the following conditions is met: ||P i -P j ||<D thr 。 3. The method according to claim 1, characterized in that In step A3.2, a facet table is created for each star convex, and the facet table is used to query whether a facet is in a star convex; 1) The process of establishing the facet table is as follows: 1.1) Create the object of the facet table: Convex S i All the face elements of are projected onto a gridded unit ball, and each grid on the unit ball is denoted as g i,j ; For a projected surface element f i , we can get its axial boundary rectangle on the unit sphere as B i , will be in the B i All grids in are denoted as G B ={g i,j |g i,j ∈B i }; 1.2) G B The grid and f i The corresponding relationship is recorded in the facet table; each grid on the final gridded unit sphere will correspond to one or more facets; (2) The process of using the facet table for quick query is as follows: For the face element f to be queried i Hexing Convex S j , first f i The center is projected onto the star convex S j The corresponding robot position R j On the grid unit sphere with the center as the sphere, the projection point falls on g i,j In the grid, in S j Query g in the facet table i,j The corresponding surface element {f j ′ |j∈{1,2,…,J}}, a total of J; each face element f j ′ The three vertices of R j Connect to form a triangular pyramid, forming a total of J triangular pyramids, if f i The center of is in one of the triangular pyramids, it is judged to be f i In the Star Convex S j middle.

4. The method according to claim 1, wherein The specific process of step A4 is as follows: Step A4.

1. Spectral clustering of frontier facets: To perform spectral clustering, two matrices must be constructed first: the degree matrix D and the similarity matrix S. For the degree matrix D, all facets are connected with their k nearest neighbors in Euclidean distance to form a graph, and the degree matrix of the graph is calculated as D; for the similarity matrix S, it is calculated as follows: s=(exp(-s(f i ,f j ) 2 / 2σ 2 )) i,j＝1…J Among them, s(f i ,f j ) is f i ,f j The total distance between two surface elements, s(F i ,F j )=||[d t ,d n ,δ n ]|| ω , σ is the parameter of the preset Gaussian function; d t ,d n ,δ n There are three distances to measure similarity, d t (F i ,F j )=||(c i -c j )·n j ||2 is the tangential distance, d n (F i ,F j )=||(c i -c j )×n j ||2 is the normal distance, δ n (F i ,F j )=||(n i -n j ) / 2||2 is the normal angle difference; After calculating D and S, the frontier clusters can be obtained through spectral clustering; Step A4.2, viewpoint selection: Generate a corresponding optimal viewpoint for each frontier cluster. When generating the viewpoint, in order to determine the optimal position of the viewpoint relative to the frontier cluster, the frontier cluster is first modeled as an elliptical surface; its position is described by three mutually orthogonal unit vectors Where N0 is equal to the vector of the normal vectors of all the facets in the cluster after normalization; then by performing PCA principal component analysis on the center points of all the facets in the cluster, an eigenvector matrix N is obtained. PCA and the corresponding eigenvalue vector λ PCA = [λ0,λ1,λ2], the eigenvalues in the vector are stored in descending order; N PCA The first and second columns of the matrix are projected onto the plane determined by N0, respectively, to obtain N1 and N2; within a certain distance range from the elliptical surface, a series of sampling points are generated at an angle relatively perpendicular to the surface: Among them, VP is the generated sampling point, C is the cluster center, R i is the observation distance of the sampling; ω=[1,ω1,ω2], where ω1∈[0,ω max ], Different sampling points can be obtained by selecting different ω, where ω1 and ω2 represent the weights of vectors N1 and N2 when generating sampling points, respectively. First, a sampling viewpoint is selected in the direction perpendicular to the elliptical surface. If the sampling viewpoint is within the known obstacle-free space, it is selected as the viewpoint corresponding to the cluster. Otherwise, sampling is continued until a viewpoint within the known obstacle-free space is sampled. If no viewpoint that meets the requirements can be found in the sampling space, the cluster is considered unobservable and is directly deleted from the environment information. Step A4.3, super viewpoint generation: A super viewpoint is generated from the viewpoints contained in the same sphere with a given threshold radius; the super viewpoint will serve as the target point that the robot needs to reach in the exploration decision.

5. The method according to claim 1, wherein The planning process of the joint planner is as follows: Step B1, Moderator Selection: The rules for determining the moderator robot are as follows: a. The robot determines the moderator of a new meeting by detecting the timestamps of other robots; b. When a robot enters a meeting led by a moderator, it will follow the other existing participants and remain the moderator; c. If the host leaves the meeting, the remaining participants will re-determine the host based on the timestamp; Step B2, information integration: The robots participating in the assembly upload all environmental information in the environmental information database to the host. Information integration is divided into two aspects: (1) integration of barrier-free space information; (2) integration of frontier information; for (1), all barrier-free space information of all robots are directly aggregated, and the union of all spatial convexities is all known barrier-free spaces; for (2), the frontier face element of each robot needs to be checked to see if it is in the spatial convexities of other robots. If so, the frontier face element is deleted; in order to find the convexity that needs to be checked corresponding to any frontier face element, a KD-tree is established for the key pose points corresponding to all spatial convexities, and all key poses of the sphere with the face element center as the origin and the maximum sensor radius as the radius are found, and then the face element is checked to see if it is in the convexity corresponding to these key poses; During the information integration process, if all the facets in a frontier cluster are deleted, the frontier cluster and its corresponding viewpoint are deleted; if all the viewpoints in a superviewpoint are deleted, the superviewpoint is deleted; Step B3, centralized planning: All superviewpoints after information integration are assigned to each robot as the target points to be reached. A time and place for the next meeting is agreed upon. After passing through the assigned superviewpoints, the robots return to the meeting place for a joint meeting. Each superviewpoint is assigned to only one robot. The cost between two points is defined as: Among them, v max is the maximum speed of the robot; The position of each robot is denoted as R c = {1,…,n}, each super viewpoint position is recorded as S c ={n+1,…,m}, define three 0-1 decision variables: (1) If and only if robot k chooses to go from point i to point j, the value is 1; (2) The value of node i is 1 if and only if it is passed by robot k; (3) t i :The value is 1 if and only if node i is selected as the next meeting place; then the centralized decision planning problem can be expressed as: Among them, N c =R c ∪S c , d ij =T m (p i ,p j ) Before solving the above problem, first select the next gathering location, that is, determine t i ; After the rendezvous location is determined, the decision-making planning problem is transformed into a vehicle routing problem. First, a heuristic function is used to search for an initial path, and then a meta-heuristic method is used to search for a local path. Starting from the robot's position, the path is extended by iteratively adding arcs with the minimum cost. This method obtains an initial solution. Finally, an extension-guided local search algorithm is used to find an improved solution. At this point, the next rendezvous location P is determined. c and the path of each robot Ψ={1,…,n}; then determine the assembly time and use the solution as the base time T b , set extra time for the robot to explore freely T e , the last unfinished meeting place and time of robot k is T cur is the current time, then the next meeting time is: On this basis, after tasks and paths are assigned to robots, they disperse to explore independently.

6. The method according to claim 5, characterized in that The node with the largest distance to all robots is selected as the meeting point.

7. The method according to claim 5, characterized in that As the environment is explored and new superviewpoints are generated, each robot continuously replans its path through an independent planner, passes through each superviewpoint, and arrives at the next collection point on time. In order to reduce repeated exploration, a penalty is introduced for the area allocated to other robots for exploration. For robot r, its position is recorded as P r , and record the next meeting place as P m , the super viewpoint set is denoted as S d , then define the penalty a of node i i for Among them, v is the path assigned to robot v; then the planning problem is: Among them, N d =P r ∪P m ∪S d , T m It's time for the next gathering, T cur is the current moment; when the robot replans, it performs a local search based on the previous solution to generate a new solution.

Images