Multi-type multi-robot cooperative large-range mapping method and device

Abstract

The invention discloses a multi-type multi-robot cooperative large-range mapping method and device, and the method comprises the steps: constructing a plurality of sub-pose maps of different types of robots, including spatial point cloud data of a public region associated with each sub-pose map, the public area is an overlapped area when different types of robots collect data in an application scene; based on the spatial point cloud data of the public area, performing data association on all the obtained sub-pose images, constructing closed-loop constraints, and obtaining a global pose image; and optimizing the global pose map to obtain a global consistent integrated and aligned map. According to the method, local sub-pose maps constructed by a plurality of different types of robots can be integrated, a large-range accurate and consistent global map is constructed, and the requirements of inspection tasks on construction of a scene large-range map in a scene in which a plurality of robots cooperate are met.

Description

Technical Field

[0001] The present invention relates to the technical field of robot positioning and mapping, and in particular to a large-scale mapping method and device for multi-type multi-robot collaboration. Background Art

[0002] SLAM mapping and positioning technology, which integrates data from multiple sensors such as lidar, visual cameras, wheel speed meters, and inertial navigation to build scene maps and position robots in them, has been widely used in various types of robots such as wheeled, footed, and unmanned aerial vehicles, and in many fields such as indoors, petrochemicals, electricity, and mines. Reliable mapping and positioning functions play a key role in the effective execution of navigation tasks by robots in the scene: only after the map is built can global path planning be effectively carried out over a large range, and accurate positioning in the map can effectively control motion. Therefore, the study of robust and reliable mapping and positioning technology is of great significance for the effective and long-term navigation algorithm function.

[0003] Large-scale construction Figure 1 Generally, they are based on a general mathematical framework for pose graph optimization. However, the general application scenarios are mainly for a single type of robot, either for wheeled mobile robots, or for legged robots, or for unmanned aerial vehicles, or just for the integration of subgraphs constructed by multiple robots of the same type. Moreover, existing algorithms often have the defect of insufficient robustness. If sensor data fails during the mapping process, it is difficult to work effectively. For example, the laser radar data source generates abnormal noise due to environmental water mist, dust, etc., the visual camera fails to image due to insufficient lighting conditions, the chassis wheels slip or the gyroscope data is stuck, resulting in abnormal wheel speed meter data for track calculation. These can easily cause system failures and cause major problems in the overall mapping and positioning function. Summary of the invention

[0004] Technical purpose:

[0005] In response to the above technical problems, the present invention proposes a large-scale mapping method and device for multi-type and multi-robot collaboration, which can integrate local pose subgraphs constructed by multiple different types of robots to construct a large-scale globally consistent global map, thereby meeting the needs of constructing large-scale scene maps in inspection tasks under multiple robot collaboration scenarios.

[0006] Technical solution:

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] A large-scale mapping method for multi-type multi-robot collaboration, comprising the steps of:

[0009] S1. Constructing multiple sub-pose graphs of different types of robots, each sub-pose graph is associated with spatial point cloud data of a common area, where the common area is an overlapping area when different types of robots collect data in an application scenario;

[0010] S2. Based on the spatial point cloud data of the common area, data association is performed on all constructed sub-pose graphs, closed-loop constraints are constructed, and a global pose graph is obtained;

[0011] S3. Integrate and optimize the entire pose graph to obtain a globally consistent map.

[0012] Preferably, the step S2 specifically includes the following steps:

[0013] S2.1, based on the spatial point cloud data of the common area, corresponding nodes on different sub-pose graphs are associated with each other to construct multiple closed-loop constraints between sub-pose graphs of different types of robots;

[0014] S2.2, screening and filtering the closed-loop constraints;

[0015] S2.3, prioritizing the closed-loop constraints after screening and filtering;

[0016] S2.4, construct the entire pose graph based on the sorted closed-loop constraints;

[0017] S2.5. Optimize the entire pose graph to obtain a globally consistent map.

[0018] Note: When building closed-loop constraints based on point cloud data in a public area, if a conflict of closed-loop constraints occurs, the closed-loop constraints with higher priorities will be used first, and the closed-loop constraints with lower priorities can be abandoned if necessary.

[0019] Preferably, in step 2.1, a marker is used to construct a closed-loop constraint between the sub-pose graphs, and the marker is an electromagnetic wave artificial landmark or a visual marker landmark set in the application scene.

[0020] Preferably, in step 2.2, point cloud features are used to screen and filter closed-loop constraints, and closed-loop constraints corresponding to point cloud features described as degraded scenes and closed-loop constraints corresponding to point cloud features whose observability values ​​are lower than a preset value are filtered out.

[0021] Preferably, in step 3, the entire pose graph is optimized using the least squares method, and an anti-interference mechanism is used during the optimization process.

[0022] Preferably, the types of robots in the application scenario include: wheeled robots, legged robots and aerial vehicles.

[0023] A multi-type multi-robot collaborative large-scale mapping device, comprising:

[0024] a) a sub-pose graph construction module, which is used to construct sub-pose graphs of multiple different types of robots, each of which has spatial point cloud data with certain common areas, where the common areas are overlapping areas when different types of robots collect data in an application scenario; and a whole pose graph construction module;

[0025] b) a pose graph closed-loop module, configured to perform data association on all acquired sub-pose graphs based on the spatial point cloud data of the common area, and use the point cloud of the common area to construct closed-loop constraints to obtain a global pose graph;

[0026] c) Pose graph optimization module, which is used to integrate and optimize the global pose graph to obtain a globally consistent map.

[0027] Preferably, the whole pose graph construction module specifically includes:

[0028] a) a data association module, which associates corresponding nodes on different sub-pose graphs with each other through point cloud matching based on the spatial point cloud data of the common area, and constructs multiple closed-loop constraints between sub-pose graphs of different types of robots;

[0029] b) a closed-loop constraint screening and filtering module, used to screen and filter the closed-loop constraints;

[0030] c) a closed-loop constraint priority sorting module, used to prioritize the closed-loop constraints after screening and filtering;

[0031] d) The pose graph construction module constructs the entire pose graph based on the sorted closed-loop constraints. At the same time, the closed-loop constraints are prioritized. If a conflict occurs, the closed-loop constraints with higher priority are retained, while the closed-loop constraints with lower priority are discarded.

[0032] A computer-readable storage medium stores a computer program or instruction, which implements the steps of the method when executed by a processor.

[0033] A computer program product comprises a computer program or instructions, which, when executed by a processor, implement the steps of the method.

[0034] Beneficial effects:

[0035] Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0036] The present invention can integrate local pose subgraphs constructed by multiple different types of robots, and has a stronger scope of application and expansion capabilities than traditional methods for mapping scenarios in which multiple types of robots collaborate in inspection scenarios. Even for the realization of extreme tasks such as search and rescue missions in unknown environments in inspection scenarios, the invention can provide a stable positioning and navigation function foundation, thereby greatly improving the applicability of mapping methods in extreme scenarios.

[0037] The present invention can construct a large-scale globally consistent map with strong anti-interference ability. It can overcome the problems of robot positioning and navigation that are difficult to effectively carry out due to the failure of some sensor data in application scenarios. It is suitable for various inspection scenarios such as chemical, oil fields, and electric power, especially scenarios where multiple types of robots such as land wheeled, legged, and aircraft cooperate. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the multi-type multi-robot collaborative mapping method proposed in the present invention;

[0039] Figure 2 Flowchart of a specific example of building a map for multi-robot collaboration;

[0040] Figure 3 Laser point cloud maps for collaborative mapping of multiple types of robots. DETAILED DESCRIPTION

[0041] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] The present invention proposes a large-scale mapping method and device for multi-type multi-robot collaboration, which is mainly based on laser radar and uses multi-source sensor data such as wheel speed meter, inertial navigation, camera vision, etc. as auxiliary, expanding the scope of application of the classic pose graph optimization framework of traditional mapping: first construct the sub-pose graph of a single robot, and then associate each sub-pose graph with each other through closed-loop constraints through data association to construct a global pose graph; then through the integration and optimization of the global pose graph, the accumulated errors in the sub-pose graphs explored by each robot are reduced, thereby reflecting strong scalability. At the same time, in the pose graph optimization process, the closed-loop constraints are evaluated for priority. When the closed-loop constraints conflict with each other, the closed-loop constraints with higher priority are used first, and the closed-loop constraints with lower priority are abandoned when necessary. At the same time, in the pose graph optimization calculation process, the anti-interference mechanism of the pose graph is adopted to improve the anti-interference ability of the pose graph optimization for erroneous motion constraints, thereby improving the effect of pose graph optimization and improving the mapping quality.

[0043] Embodiment 1

[0044] This embodiment proposes the basic logic of a multi-robot collaborative large-scale mapping method based on scalable pose graph optimization, and makes corresponding expansions on the basis of the traditional pose graph optimization framework.

[0045] (I) Traditional Mapping Method Based on Pose Graph Optimization

[0046] The basic idea is to construct inter-frame constraints of the pose graph at the front end, and the closed-loop constraints of the pose graph at the back end, and obtain a globally consistent pose graph through least squares pose graph optimization:

[0047] 1. The front end constructs the pose graph frame-to-frame motion constraint (consecutive motion constraint)

[0048] Use lidar-inertial-odometry (laser matching), wheel-inertial-odometry or visual inertial odometry to calculate the spatial relative geometric relationship between adjacent frame nodes.

[0049] 2. Backend builds intra-graph-loop-closure constraints

[0050] Find matching point cloud pairs based on spatial geometric distance or visual / laser feature data association, and construct intra-graph-loop-closure constraints between non-consecutive graph nodes in this pose graph through point cloud matching.

[0051] 3. After constructing the inter-frame constraints and closed-loop constraints of the pose graph, the pose graph can be optimized (Pose Graph Optimization) to solve the least squares problem with the robot pose nodes as optimization variables to reduce the cumulative error in the pose graph, that is, the cumulative error during the robot's motion process, so as to obtain a map that maintains consistency within the pose graph. In the pose graph optimization process, the robot pose nodes are used as optimization variables, and the motion constraints are used as the least squares constraints of the optimization variable objective function, and traditional optimization methods such as Gauss-Netwon or Levenburg-Marquardt are used to solve a series of robot poses.

[0052] (II) Multi-robot collaboration for large-scale mapping

[0053] like Figure 1As shown in the figure, the multi-type multi-robot collaborative mapping method based on pose graph optimization proposed in the present invention is specifically composed of the following steps:

[0054] a) Construct a single pose graph for each robot;

[0055] b) Closed-loop constraints are established between the pose graphs of different robots through data association. When establishing closed-loop constraints, priority sorting is used for screening and construction;

[0056] c) The entire pose graph is integrated and optimized to obtain a globally consistent map model.

[0057] The details of each step are described below:

[0058] 1. Construct a single robot pose graph:

[0059] The construction of a single robot pose graph is the basis for the subsequent integration and optimization of the global pose graph. The construction of the sub-pose graph is mainly achieved through real-time robot pose estimation motion tracking, so as to obtain the spatial geometric motion constraints between adjacent frames of the robot. It is generally achieved through multi-source fusion (fusion of multiple sensors such as wheel speed, inertial navigation and lidar data).

[0060] (1) Laser point cloud preprocessing

[0061] The significance of laser point cloud preprocessing is to filter out noise points in the original radar point cloud and other factors that are unfavorable to laser matching, provide a reliable data source for laser matching, and leave room for subsequent laser matching.

[0062] a) On the one hand, it eliminates the motion distortion of the laser radar observation value caused by its own motion, and on the other hand, it filters out abnormal noise points, reduces the negative impact of outliers on laser matching, and thins out the point cloud, which can reduce the number of point clouds that need to be matched and reduce computing power consumption;

[0063] b) Perform motion distortion correction, cloud merger and adaptive voxelization filtering on the original laser point cloud to convert the original point clouds from various sources into point clouds comparable in size and density, laying the foundation for subsequent laser matching;

[0064] (2) Laser matching to construct inter-frame motion constraints of pose graph

[0065] Laser matching is to match the laser point cloud of the current frame with the laser point cloud of the historical reference frame, and then estimate the pose of the current radar or robot relative to the reference frame coordinate system. Laser matching is manifested in two ways: laser matching laser and laser matching map. The goal is to build motion constraints between adjacent frames in the pose graph; in essence, a laser odometer is built to estimate the robot pose in real time, so as to estimate the relative rotation and relative translation between adjacent frames. In real-time pose estimation motion tracking, reliable pose estimation results can provide better pose priors for subsequent closed-loop detection based on spatial position geometric constraints, and more quickly and accurately build closed-loop constraints for data association.

[0066] Laser matching is mainly performed through GICP (General Iterative Closest Point) / NDT (Normal Distribution Transform). GICP uses the projection of the Euclidean distance between points in the normal vector direction as a metric to find matching pairs and construct the objective function. Generally, each iteration follows the method of finding matching pairs and then processing the matching pairs, thereby continuously approaching the optimal solution.

[0067] The method of laser matching map is similar to the above method. The difference is that the matching reference object is not the local point cloud map constructed in real time, but comes from the loaded point cloud data. In this way, it is necessary to first select the nearby local map point cloud in the loaded point cloud map through the initial pose prior, and then use the laser matching laser method to process it to obtain the spatial coordinate transformation value.

[0068] (3) Closed-loop matching to construct closed-loop motion constraints inside the pose graph

[0069] Find possible point cloud matching pairs through spatial Euclidean geometric distance or visual features or laser point cloud feature matching, run parallel multi-threaded laser point cloud matching based on spatial geometric distance (the matching prior uses the relative orientation from the laser odometer) to try to build closed-loop constraints, and use fitness score to filter the closed-loop constraints that are successfully matched;

[0070] 2. Sort and filter the closed-loop constraints between multiple robots:

[0071] After a single robot constructs a single pose graph, multi-robot collaborative large-scale mapping requires finding the parts where closed-loop constraints exist through data association, and constructing enough and effective closed-loop constraints, so as to unify the global pose graph by associating the local sub-pose graphs of multiple robots with each other, laying the foundation for subsequent optimization.

[0072] (1) Construct closed-loop constraints between sub-pose graphs through common areas

[0073] Data association mainly refers to finding corresponding matching pairs in point clouds. These matching pairs come from different point clouds and correspond to the same parts in the scene. After obtaining the data association matching pairs, laser point cloud matching is used to calculate the spatial relative geometric relationship between the point clouds, which is the closed-loop constraint. By constructing closed-loop constraints (inter-graph loop-closure constraints) between different robot pose graphs, multiple different sub-pose graphs can be associated to form a global pose graph, laying the foundation for subsequent optimization.

[0074] The specific method of data association is to first find two frames of point clouds that may have a closed-loop relationship based on spatial geometric distance or visual feature matching or laser point cloud feature matching; then perform iterative nearest neighbor matching on these two frames of point clouds. The specific point cloud matching method can use GICP general nearest neighbor matching, NDT normal distribution transformation, etc.

[0075] (2) Screening and evaluating closed-loop constraints

[0076] In the closed-loop constraints of the pose graph, errors or over-constraints will have a negative impact on the map generated by the pose graph optimization, so they need to be filtered. Prioritizing the closed-loop constraints, abandoning some closed-loop constraints with lower reliability, and selectively optimizing the pose graph can build a globally consistent map. At the same time, the use of anti-interference mechanisms in the pose graph optimization process can also eliminate the impact of erroneous closed-loop constraints on the pose graph optimization results. When the closed-loop constraints of the pose graph are prioritized, the following methods can be used to filter out closed-loop constraints with higher priorities to avoid conflicts between closed-loop constraints and reduce computational load.

[0077] a) Use markers to construct closed-loop constraint analysis:

[0078] Marker recognition has a higher priority in closed-loop constraint rating because the closed-loop constraints constructed by this method are more reliable. Markers can be electromagnetic wave artificial landmarks or visual marker landmarks.

[0079] Electromagnetic wave radio wave markers are placed in the environment, and when other robots pass by the markers, they will recognize the markers and indirectly use the radio wave markers as a medium to establish inter-graph loop-closure constraints between the pose graph subgraphs. The closed-loop constraints constructed in this way are generally more reliable and can be used first for pose graph optimization.

[0080] The visual fiducial marker recognition in the scene can be embodied in the form of a QR code. Its role is to detect and identify the target object to calculate the spatial relative displacement between the camera and the target object, and then obtain the spatial relative displacement between the robot and the target object. In this way, using the target object as an intermediary, a closed-loop constraint on the relative spatial coordinate transformation between different robots can be obtained. Visual target object recognition, such as QR code recognition, can refer to the following description for method details: After the camera image recognizes the pixel coordinates of the corresponding feature points on the target object, an associated matching pair is established with the spatial 3D coordinates. Then, the camera intrinsic parameters are used to solve the PNP problem to solve the camera pose. See Figure 3 , that is, the relative spatial coordinate transformation between the camera and the target image.

[0081] Specifically, in the actual use scenario of multi-robot collaborative mapping, visual markers are generally marked at the entrance of the scene for easy identification. After wheeled, legged, and flying robots detect visual markers, they will continue to estimate the relative pose of the camera relative to the visual markers, and then use the camera extrinsics to indirectly obtain the relative pose of the robot relative to the visual markers. In this way, the camera visual markers are used as an intermediate medium to indirectly calculate the relative spatial transformation between different robots, thereby building closed-loop constraints between the nodes of different pose graphs.

[0082] b) Calculate the observability of the laser point cloud for analysis:

[0083] Quantitative analysis through the statistical significance of point cloud data is also a method to evaluate the priority of closed loops. Point cloud features can be described by the observability value of the point cloud, including whether the point cloud features are rich or not, and whether it is a degraded scene. Specifically, the observability value of the point cloud is defined as the value corresponding to the minimum eigenvalue in the observable matrix, and the normalized observability score is defined as the ratio of the value corresponding to the minimum eigenvalue in the observable matrix to the value corresponding to the maximum eigenvalue. By filtering out the smaller part of the observability value, the closed-loop constraints corresponding to the scenes with feature degradation are filtered out.

[0084] c) Analysis based on pose graph neural network:

[0085] In order to reduce the computational burden of pose graph optimization, an approximate method is used for processing. A pose graph neural network is constructed to quickly approximate and infer the pose graph, evaluate the closed-loop options and select the best set, that is, the pose graph constraints that can reduce the target cost function in the pose graph the most. Specifically, a Gaussian Mixed Model pose graph convolutional network is constructed to construct a convolutional neural network for the pose graph. This neural network model can learn a series of integrated nodes in the pose graph, and use the messages integrated by the nodes and the side information of the pose graph to distinguish whether there are effective closed-loop constraints.

[0086] 3. Global pose graph optimization to obtain a globally consistent map model

[0087] On the basis of constructing the whole pose graph by multiple interrelated sub-pose graphs, the whole pose graph can be optimized to obtain a map with consistency in a large range. The global pose graph optimization can be processed in the background Base Station. On the one hand, the closed-loop constraints are constructed and screened, and on the other hand, the global pose graph can be optimized to obtain a globally consistent map. In addition, the advantages of parallel computing on remote servers can be fully utilized to improve the efficiency of multi-subgraph integration by optimizing and reintegrating the block pose graph.

[0088] For the entire pose graph that uses closed-loop constraints to build the connections between its sub-pose graphs, some basic operations of least squares pose graph optimization, such as Gauss-Newton and Levenburg-Marquardt, are used to optimize and obtain globally consistent results.

[0089] In order to reduce and eliminate the impact of erroneous closed-loop constraints on the optimization results, the pose graph optimization process can use anti-interference mechanisms, such as Dynamic Covariance Constraint, robust kernel anti-interference kernel function and other methods. You can also use Graduated-Non-Covexity GNC in conjunction with LM to perform anti-interference optimization to improve the anti-interference ability of misplaced motion constraints in the pose graph. Perform anti-interference optimization on the overall large-scale pose graph to improve the anti-interference ability of erroneous constraints.

[0090] The anti-interference scheme is based on the optimization of the classic pose graph. Here, it is only suitable for the collaborative mapping of multiple types of robots. It can be adjusted by setting the corresponding covariance matrix in the pose graph. It is mainly processed in the optimization process: the numerical value of the motion constraint is constructed by optimizing the variables, and then compared with the target value, and the covariance matrix is ​​defined as the corresponding optimization variable, so as to achieve anti-interference for the closed-loop constraints that may not be filtered out.

[0091] It can be seen from the above process that the large-scale collaborative mapping method proposed in the present invention is not limited by the type of robot, and can be different types of mobile robots, such as wheeled, legged, aerial vehicles, etc. Although the movement modes are different, as long as the front-end sub-pose graph constructed by its own sensor uses the same model structure (pose graph of associated spatial point cloud), and by constructing data associations between sub-pose graphs, it can be included in the multi-robot collaborative pose graph optimization mapping framework for collaborative mapping.

[0092] Cross-type robot collaborative mapping mainly reflects the sub-graphs constructed by different types of robots. They can be associated with each other through overlapping parts in the scene. For the non-overlapping map parts that belong to the robot alone, they can be retained when constructing the global map, so as to achieve the expansion of the overall sub-graphs. Figure 2 As shown in the figure: Each robot first constructs its own sub-pose graph and associates the spatial point cloud with the pose. Then, the corresponding nodes on different pose graphs are associated with each other through the spatial point cloud data in the common area, so that the extended pose graphs that build closed-loop constraints can be obtained.

[0093] Specifically, when building a map, it is necessary to ensure that there are common overlapping parts in the scene. These common parts are used for data association to construct closed-loop constraints of the pose graph to associate the sub-pose graphs constructed by different types of robots with each other. Specifically, since wheeled and legged robots are mainly modeled for ground objects, aerial robots also need to collect data for ground objects to ensure that the data range collected by the aerial robot's sensor perspective has a certain common part with the ground robot's perception range. Only with the foundation of public perception data can data association and global pose graph optimization be performed.

[0094] In addition, the map constructed by the present invention based on the data collected by various robots needs to use a unified map format. Among them, the spatial 3D point cloud map, the point cloud data source can come from 3D laser radar, or from the spatial coordinate points restored by visual images through multi-view geometry, the difference is only the data source is different.

[0095] See also Figure 3In the present invention, the laser point cloud map constructed by multiple types of robots in a collaborative manner, the unified format of the map model includes two aspects:

[0096] On the one hand, the pose graph optimization framework can unify and integrate the sub-pose graphs of different types of robots: the sub-pose graphs that perceive the common area are associated through closed-loop constraints, and then the sub-pose graphs are aligned and integrated through pose graph optimization;

[0097] On the other hand, the point cloud features that directly reflect the spatial geometric structure of the map model. The 3D point cloud mainly comes from LiDAR data. The 3D points restored by vision through multi-view geometry can also construct a map model. The global consistent map mainly refers to the global consistent map constructed by optimizing the sub-maps after they are associated through closed-loop constraints, so that the cumulative error is more evenly distributed. The characteristic of the global consistent map is that the overlapping laser point clouds can be aligned with each other, and there is rarely any misalignment or ghosting.

[0098] Point cloud maps describe the spatial structure of a scene. Furthermore, they can be further differentiated based on environmental characteristics and marked with more semantic information. After semantic partitioning, it is convenient to associate more detailed navigation tasks associated with semantic scenes, thereby achieving more complex navigation tasks.

[0099] Embodiment 2

[0100] This embodiment proposes a large-scale mapping device for multi-type multi-robot collaboration, which executes the mapping method described in the first embodiment. The device includes:

[0101] a) a sub-pose graph construction module, which is used to construct sub-pose graphs of multiple different types of robots, each sub-pose graph is associated with spatial point cloud data of a common area, and the common area is an overlapping area when different types of robots collect data in an application scenario;

[0102] b) Pose graph closed-loop constraint construction module, which is used to construct a global pose graph through closed-loop constraints. Based on the matching of spatial point cloud data in the common area, data association is performed on all acquired sub-pose graphs, and closed-loop constraints are constructed to obtain a complete pose graph;

[0103] c) a pose graph optimization module, used to optimize the entire pose graph to obtain a globally consistent map.

[0104] Among them, the closed-loop construction module is also the global pose graph construction module, which specifically includes:

[0105] a) an association module, which associates corresponding nodes on different sub-pose graphs with each other based on the spatial point cloud data of the common area, and constructs multiple closed-loop constraints between sub-pose graphs of different types of robots;

[0106] b) a screening and filtering module, used to screen and filter the closed-loop constraints;

[0107] c) a priority sorting module, used to prioritize the closed-loop constraints after screening and filtering;

[0108] d) Pose graph construction module, which constructs the entire pose graph based on the sorted closed-loop constraints.

[0109] To sum up, from a macroscopic perspective, each robot constructs its own sub-pose graph through its own front-end (responsible for constructing consecutive constraints between nodes in adjacent frames of the pose graph) and back-end (responsible for constructing loop-closure constraints between nodes in non-adjacent frames within the pose graph). At the same time, the background Base Station closed-loop construction module constructs data-related motion constraints between each robot's sub-pose graph, so that closed-loop constraints (inter-graph loop-closure constraints) can be established between different nodes in different pose graphs of multiple different robots, thereby integrating and optimizing to obtain a globally consistent map through pose graph optimization.

[0110] The actual economic benefit of the multi-robot collaborative mapping proposed in the present invention is that it can fully integrate the sub-pose graph data from multiple robots, realize parallel and synchronous mapping, improve mapping efficiency, improve mapping accuracy and application scope, improve robustness and anti-interference ability, and is particularly suitable for large-scale mapping covering larger areas such as oil fields, electric power, and chemical industries.

[0111] The method of the present invention is applied in inspection scenarios such as petrochemicals and electrical engineering. The large-scale mapping algorithm of multiple robots across different robot types can associate the sub-pose graphs constructed by different robots through closed-loop constraints constructed in the common area. After obtaining the global map, it is optimized and integrated to obtain a globally consistent map, thereby realizing cross-type multi-robot collaborative mapping to cover a larger range of scenes, and can overcome the problem that the large scene area will cause the mapping process to take a long time. If there is a problem with the map quality, re-construction of the map will often bring a large maintenance economic burden.

[0112] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A large-scale mapping method for multi-type multi-robot collaboration, characterized in that: Includes steps: S1. Constructing multiple sub-pose graphs of different types of robots, each sub-pose graph is associated with spatial point cloud data of a common area, where the common area is an overlapping area when different types of robots collect data in an application scenario; S2. Based on the spatial point cloud data of the common area, data association is performed on all constructed sub-pose graphs, closed-loop constraints are constructed, and a global pose graph is obtained; S3. Integrate and optimize the global pose graph to obtain a globally consistent map.

2. A large-scale mapping method for multi-type multi-robot collaboration according to claim 1, characterized in that: The step S2 specifically includes the following steps: S2.1, based on the spatial point cloud data of the common area, corresponding nodes on different sub-pose graphs are associated with each other to construct multiple closed-loop constraints between sub-pose graphs of different types of robots; S2.2, screening and filtering the closed-loop constraints; S2.3, prioritizing the closed-loop constraints after screening and filtering; S2.4, construct the entire pose graph based on the sorted closed-loop constraints; S2.

5. Optimize the entire pose graph to obtain a globally consistent map.

3. The method for large-scale mapping by multi-type multi-robot collaboration according to claim 1, characterized in that: In step 2.1, a marker is used to construct a closed-loop constraint between sub-pose graphs, and the marker is an electromagnetic wave artificial landmark or a visual marker landmark set in the application scene.

4. The method for large-scale mapping by multi-type multi-robot collaboration according to claim 1, characterized in that: In step 2.2, point cloud features are used to screen and filter closed-loop constraints, and closed-loop constraints corresponding to point cloud features described as degraded scenes and closed-loop constraints corresponding to point cloud features whose observability values ​​are lower than preset values ​​are filtered out.

5. The method for large-scale mapping by multi-type multi-robot collaboration according to claim 1, characterized in that: In step 3, the entire pose graph is optimized using the least squares method, and an anti-interference mechanism is used during the optimization process.

6. The method for large-scale mapping by multi-type multi-robot collaboration according to claim 1, characterized in that: The types of robots in the application scenario include: wheeled robots, legged robots and aerial vehicles.

7. A multi-type multi-robot collaborative large-scale mapping device, characterized in that: include: A sub-pose graph construction module is used to construct sub-pose graphs of multiple different types of robots, each of which has spatial point cloud data with certain common areas. The common area is the area where different types of robots overlap when collecting data in an application scenario; a whole pose graph construction module; A pose graph closed-loop module, used to perform data association on all acquired sub-pose graphs based on the spatial point cloud data of the common area, use the point cloud of the common area to construct closed-loop constraints, and obtain a global pose graph; The pose graph optimization module is used to integrate and optimize the global pose graph to obtain a globally consistent map.

8. The multi-type multi-robot collaborative large-scale mapping device according to claim 7, characterized in that: The entire pose graph construction module specifically includes: A data association module, based on the spatial point cloud data of the common area, associates corresponding nodes on different sub-pose graphs with each other through point cloud matching, and constructs multiple closed-loop constraints between sub-pose graphs of different types of robots; A closed-loop constraint screening and filtering module, used for screening and filtering the closed-loop constraints; A closed-loop constraint priority sorting module is used to prioritize closed-loop constraints after filtering; The pose graph construction module constructs the entire pose graph based on the sorted closed-loop constraints; at the same time, the closed-loop constraints are prioritized. If a conflict occurs, the closed-loop constraints with higher priority are retained, and the closed-loop constraints with lower priority are discarded.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of any method described in claims 1-6 are implemented.

Images