Multi-robot collaborative brain-like SLAM method

Through the multi-robot collaborative brain-like SLAM method, the incremental binary search tree and dynamic island mechanism are used to perform environmental perception coding, and combined with the pose cell network and graph relaxation algorithm, the existing SLAM solution has a large amount of computation and low visual perception efficiency. The problem is to achieve efficient environmental map construction and map fusion between multiple robots.

CN119027940APending Publication Date: 2024-11-26SHENYANG INST OF AUTOMATION - CHINESE ACAD OF SCI
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310588128.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing SLAM solution has a large amount of computing and storage when navigating in unknown environments, which cannot meet the navigation needs of long-term and large environments. Moreover, the visual perception accuracy and low efficiency of RatSLAM are poor, so it is impossible to complete the construction of the environmental map of the large environment.

Method used

Using multi-robot collaborative brain SLAM method, local view cells are improved through incremental binary search trees and dynamic island mechanisms, pose cell network is constructed, environmental perception information is encoded and map fusion is integrated, and a centralized communication structure is used to perform multiple The overlapping area detection and pose estimation between robots are combined with the graph relaxation algorithm for map optimization and fusion.

Benefits of technology

It improves the mapping efficiency of SLAM, improves visual processing performance, realizes real-time communication and map fusion between multiple robots, and can complete accurate map construction and optimization in a large-scale environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027940A_ABST
    Figure CN119027940A_ABST
Patent Text Reader

Abstract

The invention relates to a multi-robot collaborative brain-like SLAM (Simultaneous Localization and Mapping) method, which comprises the following steps that: a single robot performs feature extraction on an acquired environment image to form an environment descriptor, incrementally constructs a binary search tree, retrieves a seen scene and stores new scene information, and completes loopback detection; coding a loopback detection result and robot speedometer information into description of an environment space through a brain-like pose cellular network, and forming an experience map of the environment; vision-based multi-robot overlapping region detection is carried out among multiple robots through a centralized communication structure by sharing vision information; performing relative pose estimation according to the overlapping region; and returning a pose estimation result to the slave robot for map fusion, and performing map fusion by using a new graph relaxation algorithm to obtain a globally consistent experience map. According to the method, the mapping efficiency based on brain-like SLAM can be remarkably improved, the practicability of a brain-like method is improved, and the method has important theoretical significance and use value.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of mobile robot navigation and positioning, and specifically is a multi-robot collaborative SLAM method based on the use of a brain-like navigation mechanism, and is a system for collaborative navigation and positioning of multiple robots in an unknown environment. Background Art

[0002] When a robot moves in an unknown environment to perform tasks, it is crucial to estimate its position in the environment. Simultaneous localization and mapping (SLAM) is one of the most promising and practical strategies to solve this problem. SLAM is a process in which a robot uses sensor information to model the environment, draw an environmental map, and estimate its position in the environmental map while moving in an unknown environment.

[0003] However, most of the current SLAM solutions are based on probability algorithms, such as Kalman filter algorithm, maximum expectation algorithm and particle filter algorithm. Although these methods have high mapping accuracy, they have defects such as large amount of calculation and large storage, and cannot meet the navigation needs in long-term and large environments. The RatSLAM method, which is based on the neural mechanism of navigation in the rodent brain, can solve the positioning and mapping problems with only a monocular camera, and has good application value. However, the original RatSLAM system uses the sum of the absolute difference of pixel values ​​after average pooling to measure the similarity between images, and needs to be compared with the saved visual template one by one to achieve position recognition or closed-loop detection. The visual perception accuracy is poor and the efficiency is low. Moreover, the robot needs to traverse the entire environment continuously at one time to build an environmental map. For robots with limited endurance, it is impossible to complete the construction of environmental maps of large environments, and the efficiency of environmental mapping is low. Summary of the invention

[0004] In view of the above problems, the present invention proposes a multi-robot collaborative brain-like SLAM method. A single robot collects environmental images and extracts features to form an environmental descriptor, incrementally constructs a binary search tree, retrieves scenes that have been seen, and stores new scene information to complete loop detection; the loop detection results and the robot odometer information are encoded into a description of the environmental space through a brain-like pose cell network, and an empirical map of the environment is formed; multiple robots share visual information and perform visual-based multi-robot overlapping area detection with a centralized communication structure; relative pose estimation is performed based on the overlapping area; the pose estimation results are returned to the slave robot for map fusion, and a new graph relaxation algorithm is used to perform map fusion to obtain a globally consistent empirical map. This improves the mapping efficiency of brain-like SLAM and expands the application scope of brain-like navigation.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0006] A multi-robot collaborative brain-like SLAM method comprises the following steps:

[0007] 1) Using incremental binary search tree and dynamic island mechanism to improve local view cells and obtain environmental perception information;

[0008] 2) Construct a pose cell network to convert the environmental perception information and the collected odometer information into the expression of the robot's position in the environment, and output the network state of the pose cells;

[0009] 3) Update the experience map according to the current posture cell network state;

[0010] 4) Based on the centralized multi-robot communication structure, detect the overlapping areas of multiple robots;

[0011] 5) When the local view cell of a single robot detects a closed position loop, the position of the robot in the map is optimized; when the slave robot receives the result of the relative position estimation from the master robot, map fusion is performed.

[0012] The step 1) comprises the following steps:

[0013] 1.1) Use the ORB algorithm to extract features from the scene image, extract N ORB feature points, calculate the BRIEF descriptor of each feature point, and generate a set of N m-dimensional binary feature descriptors of the image;

[0014] 1.2) Image I i All feature descriptors are input into the binary search tree for retrieval, and based on the similarity score, it is determined whether there is a feature that is similar to image I. i Similar images, if image I j and the query image I i If the similarity score between them is greater than the threshold, then image I j For image I i Similar images of image I j As an alternative image, otherwise, insert image I into the binary search tree i All features are used to update the binary search tree;

[0015] 1.3) Collection and Image I i All candidate images similar to I i Sort the similarity from high to low to get an ordered image set C i = {I i1 ,I i2 ,...,I in}, where I ij Indicates that iThe jth most similar image, j = 1, 2, ..., n, normalizes the similarity scores of the images in the set, removes images with scores less than the threshold, and obtains an ordered set of candidate images

[0016] 1.4) Using Alternative Image Sets All images in I ij , creating dynamically managed islands Among them, the islands Represents images with timestamps between m and n. Each island has a representative image, called the island center, which represents the image most similar to the query image in the timestamp range [m, n]. The island most similar to the query image is selected, and the image corresponding to its island center is used as the final closed-loop decision result.

[0017] The similarity score s(I i ,I j )for:

[0018]

[0019] Among them, n i,j I is the query image i With image I j The number of feature points that are successfully matched in the search tree, N is the total number of feature points in the query image, and τ1 is the number of feature points that are successfully matched in the search tree. i A dynamic threshold based on the mean similarity score of the previous n frames,

[0020]

[0021] The normalization of the similarity scores of the images in the set is specifically as follows:

[0022]

[0023] Among them, s′(I i ,I ij ) is the normalized similarity score.

[0024] The step 1.4) comprises the following steps:

[0025] 1.4.1) Yes All images in I ij , evaluates whether its timestamp is located on an existing island Within the time interval, if I ij Exists in the existing island, image I ij Set as The center of the island will be ijThe first k frames and the last k frames of are added to the island. In order to adapt to the size of the island, when the current island size exceeds the limit, the images far away from the center of the island are moved out of the island and updated. time interval; if I ij If it does not exist in the existing island, a new island is created. The new island consists of I ij and its previous k frames and the next k frames, the center of the island is I ij ;

[0026] 1.4.2) Check whether all islands intersect with other islands. If so, crop the borders of the islands that intersect to ensure that the same image only appears in one island;

[0027] 1.4.3) Perform a global evaluation on each island and calculate the global score

[0028]

[0029] 1.4.4) Select the island with the highest global score and use the image corresponding to the center of the island as the closed-loop decision.

[0030] The centralized multi-robot communication structure includes: a master robot and a slave robot. The slave robot sends a scene image and its own position information to the master robot. The local view cell of the master robot receives the image and detects the overlapping area of ​​the environment, calculates the relative positions between the robots, and returns the result to the slave robot.

[0031] The step 4) comprises the following steps:

[0032] 4.1) The slave robot sends the scene image of its current location to the master robot and its own position The main robot receives the image and performs feature extraction and calculation;

[0033] 4.2) The master robot puts the image descriptor of the slave robot into its own binary search tree for image retrieval, and calculates the similarity between the master robot's own image and the slave robot's image by setting a threshold to find the overlapping area;

[0034] 4.3) If you find the same scene According to the link relationship between the RatSLAM visual template and the experience node, the position of the main robot in the scene can be found

[0035] 4.4) Express the position coordinates of the master and slave robots using homogeneous coordinates:

[0036]

[0037] in, is the rotation matrix of robot A at the overlapping position, is the rotation matrix of robot B at the overlapping position, is the translation matrix of robot A at the overlapping position, is the rotation matrix of robot B at the overlapping position;

[0038] 4.5) Calculate the relative position of the master robot to the slave robot And the relative position Return to the slave robot:

[0039]

[0040] The step 5) comprises the following steps:

[0041] 5.1) When there is no overlapping position between multiple robots, the robot perceives its own environment and builds an experience map, which has no connection with the experience map of other robots; at this time, the graph relaxation algorithm is called at a certain frequency to correct the map, the update coefficient is set to α = 0.5, and the standard map optimization is performed;

[0042] 5.2) When the master robot detects that there is an overlapping position with the slave robot, the slave robot changes its current position and sets α=1 in the position update formula to optimize the map;

[0043] 5.3) After completing the map fusion, perform standard map relaxation optimization using the position update rule with α=0.5.

[0044] The graph relaxation algorithm is specifically:

[0045]

[0046] Among them, N f Representative node e i The number of nodes pointed to, N t Indicates that it points to node e i The number of nodes, α represents the confidence in other nodes pointed by the node, and p i , p j , p k Respectively represent the node e i , e j , e k Position in the empirical map, Δp ij , Δp ki Respectively represent the slave node e i To node e j and from node e k To node e iThe required position and orientation changes.

[0047] A multi-robot collaborative brain-like SLAM system, comprising:

[0048] The local view cell improvement module is used to improve the local view cells using the incremental binary search tree and dynamic island mechanism to obtain environmental perception information;

[0049] The pose cell network construction module is used to construct the pose cell network, convert the environmental perception information and odometer information into the expression of the robot's position in the environment, and output the network state of the pose cell;

[0050] The experience mapping module is used to update the experience map according to the current state of the pose cell network;

[0051] A multi-robot overlapping area detection module is used to detect the overlapping area of ​​multiple robots based on a centralized multi-robot communication structure;

[0052] The multi-robot map fusion optimization module is used to optimize the position of the robot in the map when the local view cell of a single robot detects a position closed loop; when the slave robot receives the result of the relative position estimation from the master robot, the map is fused.

[0053] The present invention has the following beneficial effects and advantages:

[0054] 1. The present invention proposes an improved visual processing algorithm applied to RatSLAM, which realizes fast and efficient image retrieval and storage, and greatly improves the performance of RatSLAM visual processing.

[0055] 2. The present invention proposes a multi-robot collaborative communication system based on a centralized structure, which can complete real-time communication between multiple robots, and based on this structure, complete real-time environmental overlapping area detection and position estimation between multiple robots. The communication system has high transmission efficiency and can meet the real-time requirements of the multi-robot system.

[0056] 3. The present invention proposes an improved map optimization and fusion algorithm. It can simultaneously optimize the local map of a single robot and fuse the local maps of multiple robots. The fusion has good real-time performance and high accuracy. It can also complete the map fusion of multiple robots on a large scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The framework diagram of the multi-robot collaborative RatSLAM system of the present invention;

[0058] Figure 2 Flow chart of the improved local view cell position recognition method of the present invention;

[0059] Figure 3 Schematic diagram of the incremental binary search tree structure of the present invention;

[0060] Figure 4 Schematic diagram of the dynamic island of the present invention;

[0061] Figure 5 Schematic diagram of the multi-robot communication structure of the present invention;

[0062] Figure 6 Schematic diagram of multi-robot overlapping area detection of the present invention;

[0063] Figure 7 Schematic diagram of multi-robot relative position estimation and map fusion of the present invention;

[0064] Figure 8 A graph showing the change in single-frame image recognition time versus the number of image frames of the present invention;

[0065] Fig. 9 Comparison chart of the single robot mapping position estimation error of the present invention with the pure odometer and the original RatSLAM method;

[0066] Fig.10 The experimental results of the collaborative mapping of multiple robots in the present invention under the public data set;

[0067] Fig.11 The experimental results of the collaborative mapping of multiple robots in a real indoor environment;

[0068] Fig.12 The experimental results comparing the mapping performance of a single robot and that of collaborative mapping of multiple robots in the present invention. DETAILED DESCRIPTION

[0069] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0070] like Figure 1As shown, the present invention proposes a multi-robot collaborative brain-like SLAM method. A single robot collects environmental images and extracts features to form an environmental descriptor, incrementally constructs a binary search tree, retrieves scenes that have been seen and stores new scene information, and completes loop detection; the loop detection results and the robot odometer information are encoded into a description of the environmental space through a brain-like pose cell network, and an empirical map of the environment is formed; multiple robots share visual information and perform visual-based multi-robot overlapping area detection with a centralized communication structure; relative pose estimation is performed based on the overlapping area; the pose estimation results are returned to the slave robot for map fusion, and a new graph relaxation algorithm is used to perform map fusion to obtain a globally consistent empirical map. The present invention includes six modules: local view cells, pose cell networks, empirical mapping, communication structures, relative position estimation, and map fusion, which are improved by incremental binary search trees and dynamic island mechanisms.

[0071] The improved local view cells use an incremental binary search tree and a dynamic island mechanism to improve the retrieval speed and recognition accuracy of the local view cells for the environmental location; if the current scene is identified as a new scene, a new local visual perception cell is established to record the ID of the scene, and an excitatory connection is established between the local visual perception cell and the most active posture cell unit in the posture cell network; conversely, if the current scene is identified as a scene that has been experienced before, the corresponding local visual perception cell is found, and energy is injected into the network through the excitatory connection between the local visual perception cell and the posture cell network, thereby changing the energy distribution of the current posture cell network.

[0072] like Figure 2 As shown, the local view cell improved by using the incremental binary search tree and the dynamic island mechanism includes the following steps:

[0073] Step 1: Use the ORB algorithm to extract features from the scene image collected by the robot, extract N ORB feature points, calculate the BRIEF descriptor of each feature point, and generate N m-dimensional binary feature descriptor sets of the image, where m is 256;

[0074] Step 2: Image I i All feature descriptors are input into the binary search tree for retrieval to determine whether there is one that matches image I. i Similar images. If image I does not exist in the binary tree j and the query image I i The similarity score between i ,I j )>τ1, then insert the image I into the binary search tree i All features update the binary search tree; otherwise, image I jAs candidate images, the similarity scores between images are calculated by the following formula:

[0075]

[0076] Here n i,j I is the query image i With image I j The number of feature points that are successfully matched in the search tree, N is the total number of feature points in the query image. τ1 is a dynamic threshold, which is I i The mean similarity score with the previous n frames depends on the frame rate of the image: the higher the frame rate, the larger n is. It is calculated as follows:

[0077]

[0078] This threshold screening is to ensure that I j It's a familiar scene.

[0079] Step 3: Collect and image I i All candidate images similar to I are screened again to remove the images with lower scores among these similar images. i Sort the similarity from high to low to get an ordered image set C i = {I i1 ,I i2 ,...,I in}, where I ij Indicates that i The jth most similar image. Their similarity scores are normalized and calculated as follows:

[0080]

[0081] If s′(I i ,I ij )<τ2, then discard it, otherwise keep it, and finally get an ordered set of candidate images For subsequent analysis, the threshold τ2 is generally selected as 0.4, which can ensure completeness and efficiency.

[0082] Step 4: Utilize Alternative Sets All images in I ij , creating dynamically managed islands The islands Represents images with timestamps between m and n to complete the final closed-loop decision. Each island has a representative image, called the island center, which represents the image most similar to the query image in the timestamp range [m, n]. Select the island most similar to the query image, and use the image corresponding to its island center as the final closed-loop decision result.

[0083] like Figure 3 As shown, a binary search tree has the following structure and operations:

[0084] Leaf node: stores the BRIEF descriptor set calculated from the feature points extracted from the image A subset of j is the jth BRIEF descriptor of the image, a dimensional binary vector, each descriptor d j The index of the image it represents will be stored for subsequent image retrieval. Different leaf nodes will store the feature set Divided into different areas, so that each descriptor belongs to a separate leaf node;

[0085] Root node and intermediate node N i : Stores bit index k i , k i ∈(0,1,…,dim(d)-1)), where dim(d) is the descriptor dimension. Each index value can only appear once in the tree to ensure the uniqueness of the index and limit the depth of the tree to below the descriptor dimension.

[0086] Insert operation: For any descriptor d j According to d j [k i ] is stored in the left or right subtree respectively, d j [k i ] is d j At index k i If d j [k i ] is 0, then enter the left subtree, otherwise enter the right subtree, and finally store it in the leaf node;

[0087] Retrieval operation: For the descriptor d to be queried q , starting from the root node, according to d q [k i ]Search the branch until the leaf node L * Until then, the descriptor to be queried is compared with all the descriptors stored in the leaf node one by one, and the most similar descriptor d is found according to a certain threshold. * .

[0088] like Figure 4 As shown, the dynamic island mechanism is to use the alternative set All images in I ij , creating dynamically managed islands The islands Represents images with timestamps between m and n to complete the final closed-loop decision. Each island has a representative image, called the island center, which represents the image most similar to the query image in the timestamp range [m, n]. Select the island most similar to the query image, and use the image corresponding to its island center as the final closed-loop decision result. The specific steps for establishing closed-loop verification using the dynamic island mechanism are as follows:

[0089] Step 1: Right All images in I ij , evaluates whether its timestamp is located on an existing island If yes, image I ij Set as The center of the island will be ij The first k frames and the last k frames of are added to the island. In order to adapt to the size of the island, when the current island size exceeds the limit, the images far away from the center of the island are moved out of the island and updated. If I ij If it does not exist in the existing island, a new island is created. The new island consists of I ij and its previous k frames and the next k frames, the center of the island is I ij Here the island size is limited to 9, and k=2.

[0090] Step 2: Check whether all islands intersect with other islands. If so, crop the border of the island to ensure that the same image only appears in one island.

[0091] Step 3: Perform a global evaluation for each island. The global score is calculated as follows:

[0092]

[0093] Step 4: Select the island with the highest global score and use the image corresponding to the center of the island as the closed-loop decision.

[0094] (2) The pose cell network is a theoretical model established based on the neural coding principles of position cells and head heading cells in the rat brain. It converts environmental perception information and odometry information into the expression of the robot's position in the environment. It is mainly driven by odometry information and estimates the robot's pose through path integration. It also accepts visual perception information from local view cells and corrects the error of pose estimation by injecting energy from local view cells into the pose cell network.

[0095] (3) Experience mapping: Based on the current state of the posture cell network, an environment cognitive map with topological relationships called the experience map is updated; this map consists of experience nodes e i and the connections between nodes ijComposition, experience node e i = {P i ,V i ,p i}, where P i represents the position of the activated pose cell (x′ i ,y′ i ,θ′ i ), p i Indicates the current position in the experience map (x i ,y i ,θ i ), V i Represents the corresponding local view cell, which stores the corresponding environment image information; node e i With node e j Connections between in, Indicates that from experience i To experience j The required position and orientation changes, From experience i To experience j The time required; calculate the current robot state {P j ,V j} and all experience nodes e in the experience map i The distance between the corresponding robot states:

[0096] S i 3μ p |P i -P j |+μ v |V i -V j |,i=1,…,M

[0097] where μ p and μ v They represent the weight coefficient of the degree of stimulation of the pose cells and the weight coefficient of the local visual perception cells, respectively, indicating the proportion of odometer information and visual information in the process of robot map building. Different weight coefficients can be set as needed; M is the number of experience nodes in the current experience map. Find the minimum S i S min If S min Greater than the threshold τ s , then establish a new experience node e j , and at the new experience node e j With the previous newly created experience node e i Establish a connection ij :

[0098]

[0099] in, Indicates that from experience i To experience j The required position and orientation changes, From experience i To experience j The time required.

[0100] 4) Based on centralized multi-robot communication structure, such as Figure 5 As shown in the figure, a centralized structure is used to realize communication between multiple robots. The system is divided into a master robot and a slave robot. The slave robot sends the environment image and its own position information to the master robot. The local view cell of the master robot receives the image and detects the overlapping area of ​​the environment, calculates the relative position between the robots, and returns the result to the slave robot.

[0101] 5) Multi-robot overlapping area detection, such as Figure 6 As shown, the multi-robot overlapping area detection is divided into the following steps:

[0102] Step 1: The slave robot sends an image of the environment where it is currently located to the master robot and its own position The main robot receives the image and performs feature extraction and calculation.

[0103] Step 2: The master robot puts the image descriptor from the robot into its own binary search tree for image retrieval, and calculates the similarity with a certain threshold to find the overlapping area of ​​the position.

[0104] Step 3: The master robot puts the image descriptor of the slave robot into the incremental binary search tree constructed by itself for image retrieval to find the overlapping area.

[0105] Step 4: If you find the same scene According to the link relationship between the RatSLAM visual template and the experience node, the position of the main robot in the scene can be found

[0106] Step 5: The position coordinates of the two robots can be expressed using homogeneous coordinates as follows:

[0107]

[0108] Step 6: Calculate the relative position and return the result to the slave robot. The calculation formula is as follows:

[0109]

[0110] 6) If Figure 7As shown, map fusion optimization, map fusion optimization adopts the following improved graph relaxation algorithm position update rule:

[0111]

[0112] Among them, N f Represents e i The number of nodes pointed to, N t Indicates that it points to node e i The number of nodes, α represents the confidence in other nodes pointed to by the node. The larger α is, the higher the confidence in the old experience is. After the map is updated, the new map nodes will be more close to the previous experience. On the contrary, the smaller α is, the more trust is placed in the new experience. After the map is updated, the old experience will be adjusted more to be close to the new experience.

[0113] (1) When there is no overlapping area among multiple robots, each robot perceives its own environment and builds a map. This experience map has no connection with the experience maps of other robots. At this time, the graph relaxation algorithm is called at a certain frequency to correct the map. The update coefficient in the following position update formula is set to α = 0.5, and standard map optimization is performed.

[0114] (2) When the master robot detects an overlapping position with the slave robot, that is, when it recognizes that the scene image of the slave robot is the same as a scene in its own environmental memory, it calculates the relative position of the two experiences and returns the result to the master robot. The slave robot changes its current position and sets α = 1 in the position update formula to optimize the map. In this case, the current position of the robot is considered to be completely correct, and the experience of the slave robot is converted to the coordinate system of the master robot to form a unified cognition. The number of iterations is set to the number of newly created experience nodes to ensure that all experience nodes of the slave robot are converted to the new coordinate space.

[0115] (3) After completing the map fusion, perform the standard map relaxation optimization using the position update rule with α = 0.5. In this case, the robot has a unified perception of the environment. Therefore, α = 0.5, thereby making a balanced position adjustment for the entire map.

[0116] like Figure 8 As shown, the single-frame image recognition time of the present invention is significantly faster than that of the original RatSLAM system. The original RatSLAM single-frame image recognition time theoretically increases linearly with the increase of the number of image frames, while the single-frame image recognition time of the present invention theoretically increases logarithmically with the increase of the number of image frames. The method of the present invention greatly reduces the time complexity of image processing.

[0117] like Fig. 9As shown in FIG. 1 , the position estimation error of the method of the present invention is significantly smaller than that of the pure odometer and the original RatSLAM method; Fig. 9 The cognitive map drawn by the method of the present invention is close to the real trajectory. The method of the present invention greatly improves the accuracy of environmental map drawing.

[0118] Fig.10 This is the collaborative mapping result of multiple robots under a public dataset. The two robots start at different starting positions at the same time and perceive their own environment. At t=19s, it can be seen that the two robots navigated to two different positions from the same starting point, indicating that there is no overlapping area between the two robots and the two local maps are unrelated. At t=38s, robot 2 has entered the position that robot 1 has previously visited, and has performed pose estimation and updated the map. From t=57s to the end, the two robots were able to perceive the environment incrementally, and the maps also converged and closed stably, and finally a globally stable empirical map was obtained.

[0119] Fig.11 This is the result of collaborative mapping of multiple robots in a real indoor environment. From the results, we can see that when t = 0s-115s, there is no global map fusion of the two robots, which means that the two robots did not detect the overlapping area of ​​the environment during this period, and the two maps are relatively independent. Then at t = 130s, robot 2 returned to the vicinity of the starting position of robot 1 and fused the maps. At this time, the maps of the two robots have been unified into one coordinate system, and finally the collaborative exploration ended at t = 140s. The two robots then continued the SLAM task, and the final empirical map is as follows Fig.11 f, which is basically consistent with the real environment. This shows that the multi-robot collaborative brain-like SLAM algorithm proposed in this paper can achieve collaborative mapping in a real multi-robot physical experimental platform, and can significantly improve the efficiency of RatSLAM mapping.

[0120] Fig.12 This is a comparison of the map constructed by a single robot and the map obtained by multi-robot collaboration in a real indoor environment. The results show that the two maps can basically remain overlapped, indicating that the accuracy of the experience map constructed by two robots is basically consistent with the complete exploration of the environment by a single robot. In addition, we also calculated the time it took for a single robot and multi-robot collaborative exploration of the experimental environment. The time spent on multi-robot collaborative exploration was 140 seconds, and each robot generated 2220 and 2194 experience nodes respectively. The time spent on the construction of a single robot was about 300 seconds, and 4060 experience nodes were generated. The efficiency of map construction has been significantly improved.

Claims

1. A multi-robot collaborative brain-like SLAM method, characterized in that: The following steps are involved: 1) Using incremental binary search tree and dynamic island mechanism to improve local view cells and obtain environmental perception information; 2) Construct a pose cell network to convert the environmental perception information and the collected odometer information into the expression of the robot's position in the environment, and output the network state of the pose cells; 3) Update the experience map according to the current posture cell network state; 4) Based on the centralized multi-robot communication structure, detect the overlapping areas of multiple robots; 5) When the local view cell of a single robot detects a closed position loop, the position of the robot in the map is optimized; When the slave robot receives the result of relative position estimation from the master robot, map fusion is performed.

2. A multi-robot collaborative brain-like SLAM method according to claim 1, characterized in that: The step 1) comprises the following steps: 1.1) Use the ORB algorithm to extract features from the scene image, extract N ORB feature points, calculate the BRIEF descriptor of each feature point, and generate a set of N m-dimensional binary feature descriptors of the image; 1.2) Image I i All feature descriptors are input into the binary search tree for retrieval, and based on the similarity score, it is determined whether there is a feature that is similar to image I. i Similar images, if image I j and the query image I i If the similarity score between them is greater than the threshold, then image I j For image I i Similar images of image I j As an alternative image, otherwise, insert image I into the binary search tree i All features are used to update the binary search tree; 1.3) Collection and Image I i All candidate images similar to I i Sort the similarity from high to low to get an ordered image set C i = {I i1 ,I i2 ,...,I in }, where I ij Indicates that i The jth most similar image, j = 1, 2, ..., n, normalizes the similarity scores of the images in the set, removes images with scores less than the threshold, and obtains an ordered set of candidate images 1.4) Using Alternative Image Sets All images in I ij , creating dynamically managed islands Among them, the islands Represents images with timestamps between m and n. Each island has a representative image, called the island center, which represents the image most similar to the query image in the timestamp range [m, n]. The island most similar to the query image is selected, and the image corresponding to its island center is used as the final closed-loop decision result.

3. A multi-robot collaborative brain-like SLAM method according to claim 2, characterized in that: The similarity score s(I i ,I j )for: Among them, n i,j I is the query image i With image I j The number of feature points that are successfully matched in the search tree, N is the total number of feature points in the query image, and τ1 is the number of feature points that are successfully matched in the search tree. i A dynamic threshold based on the mean similarity score of the previous n frames, 4. A multi-robot collaborative brain-like SLAM method according to claim 2, characterized in that: The normalization of the similarity scores of the images in the set is specifically as follows: Among them, s′(I i ,I ij ) is the normalized similarity score.

5. A multi-robot collaborative brain-like SLAM method according to claim 2, characterized in that: The step 1.4) comprises the following steps: 1.4.1) Yes All images in I ij , evaluates whether its timestamp is located on an existing island Within the time interval, if I ij Exists in the existing island, image I ij Set as The center of the island will be ij The first k frames and the last k frames of are added to the island. In order to adapt to the size of the island, when the current island size exceeds the limit, the images far away from the center of the island are moved out of the island and updated. time interval; if I ij If it does not exist in the existing island, a new island is created. The new island consists of I ij and its previous k frames and the next k frames, the center of the island is I ij ; 1.4.2) Check whether all islands intersect with other islands. If so, crop the borders of the islands that intersect to ensure that the same image only appears in one island; 1.4.3) Perform a global evaluation on each island and calculate the global score 1.4.4) Select the island with the highest global score and use the image corresponding to the center of the island as the closed-loop decision.

6. A multi-robot collaborative brain-like SLAM method according to claim 1, characterized in that: The centralized multi-robot communication structure includes: a master robot and a slave robot. The slave robot sends a scene image and its own position information to the master robot. The local view cell of the master robot receives the image and detects the overlapping area of ​​the environment, calculates the relative positions between the robots, and returns the result to the slave robot.

7. The multi-robot collaborative brain-like SLAM method according to claim 1, characterized in that: The step 4) comprises the following steps: 4.1) The slave robot sends the scene image of its current location to the master robot and its own position The main robot receives the image and performs feature extraction and calculation; 4.2) The master robot puts the image descriptor of the slave robot into its own binary search tree for image retrieval, and calculates the similarity between the master robot's own image and the slave robot's image by setting a threshold to find the overlapping area; 4.3) If you find the same scene According to the link relationship between the RatSLAM visual template and the experience node, the position of the main robot in the scene can be found 4.4) Express the position coordinates of the master and slave robots using homogeneous coordinates: in, is the rotation matrix of robot A at the overlapping position, is the rotation matrix of robot B at the overlapping position, is the translation matrix of robot A at the overlapping position, is the rotation matrix of robot B at the overlapping position; 4.5) Calculate the relative position of the master robot to the slave robot And the relative position Return to the slave robot:

8. The multi-robot collaborative brain-like SLAM method according to claim 1, characterized in that: The step 5) comprises the following steps: 5.1) When there is no overlapping position between multiple robots, the robot perceives its own environment and builds an experience map, which has no connection with the experience map of other robots; at this time, the graph relaxation algorithm is called at a certain frequency to correct the map, the update coefficient is set to α = 0.5, and the standard map optimization is performed; 5.2) When the master robot detects that there is an overlapping position with the slave robot, the slave robot changes its current position and sets α=1 in the position update formula to optimize the map; 5.3) After completing the map fusion, perform standard map relaxation optimization using the position update rule with α=0.

5.

9. A multi-robot collaborative brain-like SLAM method according to claim 8, characterized in that: The graph relaxation algorithm is specifically: Among them, N f Representative node e i The number of nodes pointed to, N t Indicates that it points to node e i The number of nodes, α represents the confidence in other nodes pointed by the node, and p i , p j , p k Respectively represent the node e i , e j , e k Position in the empirical map, Δp ij , Δp ki Respectively represent the slave node e i To node e j and from node e k To node e i The required position and orientation changes.

10. A multi-robot collaborative brain-like SLAM system, characterized in that: include: The local view cell improvement module is used to improve the local view cells using the incremental binary search tree and dynamic island mechanism to obtain environmental perception information; The pose cell network construction module is used to construct the pose cell network, convert the environmental perception information and odometer information into the expression of the robot's position in the environment, and output the network state of the pose cell; The experience mapping module is used to update the experience map according to the current state of the pose cell network; A multi-robot overlapping area detection module is used to detect the overlapping area of ​​multiple robots based on a centralized multi-robot communication structure; The multi-robot map fusion optimization module is used to optimize the position of the robot in the map when the local view cell of a single robot detects a position closed loop; when the slave robot receives the result of the relative position estimation from the master robot, the map is fused.

Citation Information

Patent Citations

  • A bionic navigation method based on mouse brain hippocampus grid cell reconstruction

    CN109886384A

  • Neuromorphic inspiring robot cognitive map building method

    CN111376273A

  • Multi-machine collaborative fusion positioning and mapping method for unknown space exploration

    CN114964212A

  • Visual sensor-based multi-mobile robot cooperative mapping method and system

    CN115131434A

  • Obstacle recognition method for autonomous robots

    US20200225673A1