A heterogeneous multi-robot cooperative SLAM method and system based on a cloud platform

By employing a heterogeneous multi-robot collaborative SLAM method and utilizing a cloud platform to process data, the problems of high hardware requirements and unstable positioning accuracy under lighting conditions in large-scale single-robot SLAM were solved, achieving high-precision map construction and improved robustness.

CN117146801BActive Publication Date: 2026-11-17NANJING INST OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311129571.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-01
Publication Date
2026-11-17
Estimated Expiration
2043-09-01

AI Technical Summary

Technical Problem

When single-robot SLAM builds maps in a large space, it has high hardware requirements, large computational load, and the positioning accuracy of the sensors is unstable under different lighting conditions, which affects the accuracy of map building.

Method used

A heterogeneous multi-robot collaborative SLAM method is adopted, and data processing is performed using a cloud platform. Robots equipped with different sensors work collaboratively under navigation mode switching to generate a global map.

Benefits of technology

It reduces the hardware requirements for robots, improves the accuracy and robustness of map building, is suitable for large-scale spaces, and avoids the impact of sensor data load on endurance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117146801B_ABST
    Figure CN117146801B_ABST
Patent Text Reader

Abstract

This invention discloses a heterogeneous multi-robot collaborative SLAM method and system based on a cloud platform, relating to the technical field of localization and mapping. The method includes: selecting a navigation mode based on the sensor types installed on a first and second mobile robot, and controlling the two mobile robots to execute corresponding commands according to the navigation mode; determining whether the conditions for switching navigation modes are met based on the ambient light intensity detected during movement; if not, maintaining the original navigation mode and executing the corresponding commands; if so, switching the navigation modes and movement order of the two mobile robots and executing the corresponding commands; and fusing all local maps generated by the first and second mobile robots executing SLAM commands to generate a global map. This invention enables collaborative localization and mapping in a large space, overcoming the limitations of single-robot SLAM and improving the accuracy and robustness of map construction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the technical field of localization and mapping, specifically relating to a heterogeneous multi-robot collaborative SLAM method and system based on a cloud platform. Background Technology

[0002] With the development of mobile robot technology, traditional single-robot SLAM performs well in small spaces ranging from tens to two hundred square meters, with high mapping accuracy. However, when facing larger spaces of nearly a thousand square meters, the hardware requirements for single robots are high, and the accuracy of single-robot map building is affected. In some cases, the excessive computation may even lead to mapping failure. Cloud platforms provide large-scale data storage and processing capabilities, allowing sensor data obtained by mobile robots to be uploaded to the cloud platform, thereby reducing the computational burden on the mobile robots themselves.

[0003] Mobile robots equipped with LiDAR are unaffected by lighting conditions during SLAM and can operate in darkness, strong light, and inclement weather. However, their positioning accuracy is poor in environments with relatively simple structures. Mobile robots equipped with vision sensors can capture depth information in the 3D environment for object shape recognition and tracking during SLAM. However, the depth information may be unstable in environments with varying lighting conditions, affecting the accuracy of localization and map building. In homogeneous multi-robot systems, the mobile robots have identical chassis structures and sensors. When performing LiDAR SLAM, the LiDAR has poor localization accuracy in environments with relatively simple structures. When performing visual SLAM, the accuracy of localization and map building is affected by varying lighting conditions.

[0004] It is now possible to mount two types of sensors on the same robot, but this operation places high demands on the mobile robot's hardware. The large amount of data generated by the sensors may increase the mobile robot's load and affect its working time and battery life. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a heterogeneous multi-robot collaborative SLAM method and system based on a cloud platform, which can perform collaborative localization and mapping in a large space, overcome the limitations of single-robot SLAM, and improve the accuracy and robustness of map construction.

[0006] This invention provides the following technical solution:

[0007] Firstly, a heterogeneous multi-robot collaborative SLAM method based on a cloud platform is provided, including:

[0008] Based on the sensor types installed on the first and second mobile robots, the navigation modes of the first and second mobile robots are selected, and the first and second mobile robots are controlled to execute corresponding instructions according to the navigation modes. The execution of the corresponding instructions includes: if the navigation mode is an active navigator, executing SLAM instructions to generate a local map; if the navigation mode is a follower navigator, executing actions to follow the active navigator and detecting ambient light intensity.

[0009] Based on the ambient light intensity detected during movement, determine whether the conditions for switching navigation modes are met:

[0010] If the conditions are not met, the original navigation modes of the first and second mobile robots will be maintained, and the first and second mobile robots will be controlled to execute corresponding instructions according to the original navigation modes.

[0011] If the conditions are met, the navigation mode and movement order of the first and second mobile robots will be switched, and the first and second mobile robots will be controlled to execute corresponding instructions according to the switched navigation mode.

[0012] Based on the SLAM instructions executed by the first mobile robot and the second mobile robot, all local maps generated by the SLAM instructions executed by the first and second mobile robots are merged to generate a global map.

[0013] Preferably, the first mobile robot is a four-wheeled differential mobile robot, and the sensor type installed on the first mobile robot includes a depth camera; the second mobile robot is a Mecanum wheeled mobile robot, and the sensor type installed on the second mobile robot includes a lidar; the condition for determining whether the navigation mode is switched is to determine whether the light intensity is less than a first threshold, the first threshold being based on the parameter setting of the depth camera.

[0014] Preferably, both the first and second mobile robots are equipped with a monocular camera and a target; the following navigator's movement to follow the active navigator includes: calculating the relative pose matrix of the first and second mobile robots based on the robot coordinates corresponding to the active navigator and the following navigator, the coordinates of the monocular camera and the target; and obtaining the pose between the first and second mobile robots based on the calculated relative pose matrix to achieve the following of the following navigator.

[0015] Preferably, calculating the relative pose matrix of the first mobile robot and the second mobile robot includes:

[0016] Based on the map coordinates of the mobile robot corresponding to the navigator and the map coordinates of the monocular camera, the first transformation matrix of the monocular camera corresponding to the navigator to the mobile robot coordinate system is calculated.

[0017] Based on the map coordinates of the mobile robot and the target corresponding to the active navigator, the second transformation matrix of the target corresponding to the active navigator relative to the coordinate system of the mobile robot is calculated.

[0018] Based on the target relative coordinates of the active navigator and the monocular camera relative to the follower navigator, the third transformation matrix of the monocular camera relative to the target is calculated.

[0019] The relative pose matrix is ​​calculated based on the first transformation matrix, the second transformation matrix, and the third transformation matrix.

[0020] Preferably, the first transformation matrix for:

[0021]

[0022] Where D represents the translation of the monocular camera corresponding to the navigator relative to its corresponding mobile robot on the Y-axis; preferably, the second transformation matrix... for:

[0023]

[0024] Where -C represents the translation of the active navigator target relative to its corresponding mobile robot on the Y-axis.

[0025] Preferably, the third transformation matrix for:

[0026]

[0027] Where T is the translation matrix of the monocular camera corresponding to the follower navigator, and R is the rotation matrix of the monocular camera corresponding to the follower navigator; taking the target B1 corresponding to the active navigator as the reference, the unit of translation of the follower navigator monocular camera C2 along the X-axis is a, the unit of translation along the Y-axis is b, the rotation angle of the follower navigator monocular camera around the Z-axis is θ, the center point of the surface of the target B1 corresponding to the active navigator is the origin O1, the X-axis is set with the target ball pointing to the left front of the origin O1, the Z-axis is set with the vertical upper surface upward, and the Y-axis is defined by the right-hand coordinate system.

[0028] Preferably, the translation matrix T of the monocular camera corresponding to the follower is:

[0029]

[0030] The rotation matrix R of the monocular camera corresponding to the follower is:

[0031]

[0032] In this system, with the target B1 corresponding to the active navigator as the reference, the unit of translation of the monocular camera C2 of the navigator along the X-axis is a, the unit of translation along the Y-axis is b, the angle of rotation of the monocular camera of the navigator around the Z-axis is θ, the center point of the surface of the target B1 corresponding to the active navigator is the origin O1, the X-axis is set with the target ball pointing to the left front of the origin O1, the Z-axis is set with the vertical upward of the upper surface, and the Y-axis is defined by the right-hand coordinate system.

[0033] Preferably, the relative pose matrix for:

[0034]

[0035] in, This is the first transformation matrix. This is the second transformation matrix. The third transformation matrix is ​​defined as follows: Taking the target B1 corresponding to the active navigator as the reference, the unit translation of the navigator's monocular camera C2 along the X-axis is 'a', the unit translation along the Y-axis is 'b', the rotation angle of the navigator's monocular camera around the Z-axis is 'θ', the center point of the surface of the target B1 corresponding to the active navigator is the origin O1, the X-axis is defined as the target ball pointing to the left front from the origin O1, the Z-axis is defined as the vertical upward direction from the upper surface, and the Y-axis is defined by the right-hand coordinate system; D is the translation of the monocular camera corresponding to the navigator relative to the mobile robot on the Y-axis; -C is the translation of the active navigator's target relative to the mobile robot on the Y-axis.

[0036] Secondly, a heterogeneous multi-robot collaborative SLAM system based on a cloud platform is provided, including:

[0037] The navigation mode acquisition module is used to acquire the navigation modes of the first mobile robot and the second mobile robot.

[0038] The control module is used to control the first mobile robot and the second mobile robot to execute corresponding instructions according to the navigation mode obtained by the navigation mode acquisition module. The execution of the corresponding instructions includes: if the navigation mode is an active navigator, executing SLAM instructions to generate a local map; if the navigation mode is a follower navigator, executing the movement of following the active navigator and detecting the ambient light intensity.

[0039] The judgment module is used to determine whether the conditions for switching navigation modes are met based on the ambient light intensity detected during the movement. If not, the original navigation modes of the first and second mobile robots are maintained, and the first and second mobile robots are controlled to execute corresponding instructions according to the original navigation modes. If the conditions are met, the navigation modes and movement order of the first and second mobile robots are switched, and the first and second mobile robots are controlled to execute corresponding instructions according to the switched navigation modes.

[0040] The fusion generation module is used to fuse all the local maps generated by the first mobile robot executing SLAM instructions and the second mobile robot executing SLAM instructions to generate a global map.

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] (1) The lidar and depth camera are installed on the first mobile robot and the second mobile robot respectively, thereby reducing the hardware requirements of the first mobile robot and the second mobile robot and avoiding the large amount of data generated after the sensor superposition may increase the load of the mobile robot and affect its working time and endurance; at the same time, by using the first mobile robot and the second mobile robot to cooperate in SLAM, the limitations of single robot SLAM can be overcome, the accuracy and robustness of map building can be improved, and it is suitable for spaces of nearly 10 million square meters.

[0043] (2) Based on the calculated relative pose matrix, the relative pose between the first mobile robot and the second mobile robot is obtained, so that the follower can follow according to the relative pose and adjust its speed and direction to ensure the distance and attitude with the active navigator. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the heterogeneous multi-robot collaborative SLAM method based on a cloud platform according to the present invention.

[0045] Figure 2 This is a schematic diagram of the structure of the first mobile robot of the present invention;

[0046] Figure 3 This is a schematic diagram of the structure of the second mobile robot of the present invention;

[0047] Figure 4 This is a schematic diagram of the target structure of the present invention;

[0048] Figure 5 This is a flowchart of the heterogeneous multi-robot collaborative SLAM process of the present invention;

[0049] Figure 6This is a flowchart of the sensor operation of each robot in the heterogeneous multi-robot collaborative SLAM of the present invention;

[0050] Figure 7 This is a flowchart of the heterogeneous multi-robot navigation mode switching based on environmental changes according to the present invention;

[0051] Figure 8 This is a flowchart of the heterogeneous multi-robot data fusion and uploading process of the present invention.

[0052] The diagram is labeled as follows: 1 is the first mobile robot, 2 is a monocular camera, 3 is a depth camera, 4 is a Raspberry Pi, 5 is a target, 6 is the second mobile robot, 7 is a lidar, 501 is the target ball in the front left, 502 is the target ball in the front right, 503 is the target ball in the rear left, and 504 is the target ball in the rear right. Detailed Implementation

[0053] The invention will now be described in further detail with reference to the accompanying drawings.

[0054] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0055] Example 1

[0056] like Figure 1 and 5 As shown, a heterogeneous multi-robot collaborative SLAM method based on a cloud platform is provided, including:

[0057] S100: Based on the sensor types installed on the first mobile robot 1 and the second mobile robot 6, select the navigation mode of the first mobile robot 1 and the second mobile robot 6, and control the first mobile robot 1 and the second mobile robot 6 to execute corresponding instructions according to the navigation mode.

[0058] The navigation modes are divided into active navigator and follower navigator. There are two navigation modes and two mobile robots. That is, after the navigation mode of the first mobile robot 1 is selected, the navigation mode of the second mobile robot 6 is another and fixed. The execution of the corresponding instructions includes: if the navigation mode is active navigator, executing SLAM instructions to generate a local map. The execution of SLAM instructions is based on a cloud platform, and the specific execution process can refer to existing technologies. If the navigation mode is follower navigator, executing the movement of following the active navigator and detecting the ambient light intensity. The detection of light intensity is based on a light sensor. The first mobile robot 1 is equipped with a light sensor one, and the second mobile robot 6 is equipped with a light sensor two.

[0059] Optionally, such as Figure 2-4As shown in Figure 6, the first mobile robot 1 is a four-wheeled differential mobile robot, and the sensors installed on the first mobile robot 1 include a depth camera 3; the second mobile robot 6 is a Mecanum wheeled mobile robot, and the sensors installed on the second mobile robot 6 include a lidar 7; both the first mobile robot 1 and the second mobile robot 6 are equipped with a monocular camera 2, a target 5, and a Raspberry Pi 4. The target 5 has four target balls: a target ball 501 at the front left, a target ball 502 at the front right, a target ball 503 at the rear left, and a target ball 504 at the rear right; the four-wheeled differential mobile robot achieves steering and movement by controlling the speed of the left and right drive wheels separately. The first mobile robot 1 makes the speed of one drive wheel greater than that of the other side to enable rotation or turning, which is suitable for environments with a relatively large turning radius; each wheel of the Mecanum wheeled mobile robot is equipped with a set of inclined rollers, which allows the second mobile robot 6 to move smoothly in any direction, and the rollers on each wheel can rotate independently, achieving omnidirectional movement, which is suitable for environments requiring a small turning radius and high mobility.

[0060] S200: such as Figure 7 As shown, based on the ambient light intensity detected during movement, it is determined whether the conditions for switching navigation modes are met. The condition for determining whether the conditions for switching navigation modes are met is whether the light intensity is less than a first threshold, which is manually set based on the parameters of the depth camera 3. In some other embodiments, the navigation mode can also be switched based on the distance in the environment, i.e., the distance is set as the first threshold. Of course, it can also be determined whether to switch based on both the light intensity and the distance in the environment. In this case, the conditions for switching are the first threshold of light intensity and the second threshold of distance.

[0061] S201: If not satisfied, maintain the original navigation modes of the first mobile robot 1 and the second mobile robot 6, and control the first mobile robot 1 and the second mobile robot 6 to execute corresponding instructions according to the original navigation modes; that is, if the navigation mode of the first mobile robot 1 or the second mobile robot 6 is an active navigator, execute SLAM instructions to generate a local map; if the navigation mode of the second mobile robot 6 or the first mobile robot 1 is a follower navigator, execute the movement of following the active navigator and detect the ambient light intensity.

[0062] S202: If satisfied, the navigation mode and movement order of the first mobile robot 1 and the second mobile robot 6 are switched, and the first mobile robot 1 and the second mobile robot 6 are controlled to execute corresponding instructions according to the switched navigation mode; that is, before switching the navigation mode, if the first mobile robot 1 is the active navigator and the second mobile robot 6 is the follower navigator, the first mobile robot 1 moves in front of the follower navigator. After switching, the navigation mode of the first mobile robot 1 becomes the follower navigator, and the navigation mode of the second mobile robot 6 becomes the active navigator. The first mobile robot 1 decelerates, and the second mobile robot 6 accelerates until the second mobile robot 6 moves in front of the first mobile robot 1. The first mobile robot 1 and the second mobile robot 6 execute corresponding instructions according to the switched navigation mode. The mobile robot corresponding to the active navigator executes SLAM instructions to generate a local map; the mobile robot corresponding to the follower navigator executes the action of following the active navigator and detecting the ambient light intensity.

[0063] S300: such as Figure 8 As shown, based on the SLAM instructions executed by the first mobile robot 1 and the second mobile robot 6, all local maps generated by the SLAM instructions executed by the first mobile robot 1 and the second mobile robot 6 are merged to generate a global map.

[0064] Two mobile robots operate simultaneously. The robots perceive their surroundings through installed sensors. One robot is selected as the active navigator, while the other automatically becomes the follower. The follower robot identifies the target 5 using its monocular camera 2 and enters follow mode. In well-lit and close-range environments, the four-wheeled differential mobile robot, acting as the active navigator, uses its depth camera 3 to perform visual SLAM, generating a map and perceiving the environment. Meanwhile, the Mecanum wheeled mobile robot, equipped with a LiDAR 7, acts as the follower robot. Its monocular camera 2 captures the target 5 of the active navigator, acquiring visual information to calculate the relative pose between the two robots. The mobile robot corresponding to the follower robot then follows based on this relative pose, adjusting its position accordingly. The system determines the robot's own speed and direction to maintain distance and posture from the active navigator. Simultaneously, a light sensor detects ambient light intensity. If the light shifts from a bright to a darker environment, a navigation mode switching condition is triggered. The robot in front decelerates, causing the robot behind to switch from follower mode to active navigator mode, continuing SLAM in the new environment. The two robots switch movement modes back and forth depending on the environment until mapping and navigation are complete. Finally, the maps built by the two sensors are fused to obtain a complete global map, which is then uploaded to a cloud platform for storage and processing, enabling map data sharing and utilization.

[0065] In some other embodiments, the following navigator performing the following movement of the active navigator includes:

[0066] D100: Calculate the relative pose matrix of the first mobile robot 1 and the second mobile robot 6 based on the robot coordinates corresponding to the active navigator and the follower navigator, the coordinates of the monocular camera 2 and the target 5.

[0067] Calculating the relative pose matrix of the first mobile robot 1 and the second mobile robot 6 includes:

[0068] D101: Based on the map coordinates of the mobile robot corresponding to the navigator and the map coordinates of the monocular camera 2, calculate the first transformation matrix of the monocular camera 2 corresponding to the navigator to the mobile robot coordinate system.

[0069] The first transformation matrix for:

[0070]

[0071] Where D is the translation of the monocular camera 2 corresponding to the navigator relative to the mobile robot on the Y-axis.

[0072] A monocular camera 2 following the navigator is installed at the front end of its corresponding mobile robot. The position of the mobile robot in the map coordinate system is (x, y, z), and the position of the monocular camera 2 in the map coordinate system is (x, y+D, z). The translation of the monocular camera 2 relative to the mobile robot it is on the Y-axis is D.

[0073] D102: Based on the map coordinates of the mobile robot corresponding to the active navigator and the map coordinates of target 5, calculate the second transformation matrix of target 5 relative to the mobile robot coordinate system.

[0074] The second transformation matrix for:

[0075]

[0076] Where -C represents the translation of the active navigator target 5 relative to the mobile robot on the Y-axis.

[0077] Target 5 is installed at the end of a mobile robot. The position of the mobile robot corresponding to the active navigator in the map coordinate system is (x, y, z), and the position of target 5 in the map coordinate system is (x, yC, z). The translation of target 5 relative to the mobile robot on the Y-axis is -C.

[0078] D103: Based on the relative coordinates of the target 5 corresponding to the active navigator and the relative coordinates of the monocular camera 2 of the follower navigator, the third transformation matrix of the monocular camera 2 relative to the target 5 is calculated.

[0079] The third transformation matrix for:

[0080]

[0081] Where T is the translation matrix of the monocular camera 2 corresponding to the navigator, and R is the rotation matrix of the monocular camera 2 corresponding to the navigator.

[0082] The translation matrix T of the monocular camera 2 corresponding to the follower navigator is:

[0083]

[0084] The rotation matrix R of the monocular camera 2 corresponding to the follower is:

[0085]

[0086] Taking the target B1 corresponding to the active navigator as the reference, the unit of translation of the monocular camera C2 of the follower along the X-axis is a, the unit of translation along the Y-axis is b, and the unit of translation along the Z-axis is 0. The angle of rotation of the monocular camera 2 of the follower around the Z-axis is θ. The center point of the surface of the target B1 corresponding to the active navigator is the origin O1. The X-axis is set with the target ball 501 pointing to the left front of the origin O1, and the Z-axis is set with the vertical upper surface upward. The Y-axis is defined by the right-hand coordinate system.

[0087] D104: The relative pose matrix is ​​calculated based on the first transformation matrix, the second transformation matrix, and the third transformation matrix.

[0088] The relative pose matrix for:

[0089]

[0090] in, This is the first transformation matrix. This is the second transformation matrix. The third transformation matrix is ​​defined as follows: Taking the target B1 corresponding to the active navigator as the reference, the unit of translation of the navigator's monocular camera C2 along the X-axis is 'a', the unit of translation along the Y-axis is 'b', and the unit of translation along the Z-axis is '0'. The angle of rotation of the navigator's monocular camera 2 around the Z-axis is 'θ'. The center point of the surface of the target B1 corresponding to the active navigator is the origin O1. The X-axis is defined with the target ball 501 pointing to the left front from the origin O1, and the Z-axis is defined with the vertical upper surface upwards. The Y-axis is defined using a right-handed coordinate system. D represents the translation of the navigator's monocular camera 2 relative to the mobile robot on the Y-axis. -C represents the translation of the active navigator's target 5 relative to the mobile robot on the Y-axis.

[0091] D200: Based on the calculated relative pose matrix, the pose between the first mobile robot 1 and the second mobile robot 6 is obtained, enabling the navigator to follow.

[0092] When acquiring the pose between the two, the monocular camera 2 and the target 5 on the chassis of the first mobile robot 1 and the second mobile robot 6 can be adjusted to ensure that the target ball on the target 5 of the mobile robot corresponding to the active navigator is within the field of view of the monocular camera 2 of the mobile robot corresponding to the follower navigator.

[0093] Example 2

[0094] A heterogeneous multi-robot collaborative SLAM system based on a cloud platform is provided, comprising:

[0095] The navigation mode acquisition module is used to acquire the navigation modes of the first mobile robot 1 and the second mobile robot 6.

[0096] The control module is used to control the first mobile robot 1 and the second mobile robot 6 to execute corresponding instructions according to the navigation mode obtained by the navigation mode acquisition module. The execution of the corresponding instructions includes: if the navigation mode is an active navigator, executing SLAM instructions to generate a local map; if the navigation mode is a follower navigator, executing the active navigator's follow movement and detecting the ambient light intensity.

[0097] The judgment module is used to determine whether the conditions for switching navigation modes are met based on the ambient light intensity detected during the movement. If not, the original navigation modes of the first mobile robot 1 and the second mobile robot 6 are maintained, and the first mobile robot 1 and the second mobile robot 6 are controlled to execute corresponding instructions according to the original navigation modes. If the conditions are met, the navigation modes of the first mobile robot 1 and the second mobile robot 6 are switched, and the first mobile robot 1 and the second mobile robot 6 are controlled to execute corresponding instructions according to the switched navigation modes.

[0098] The fusion generation module is used to merge all the local maps generated by the first mobile robot 1 executing SLAM instructions and the second mobile robot 6 executing SLAM instructions to generate a global map.

[0099] Those skilled in the art will understand that the embodiments of this application can be provided as methods, systems, or, of course, computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application; it should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0101] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0102] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A heterogeneous multi-robot collaborative SLAM method based on a cloud platform, characterized in that, include: Based on the sensor types installed on the first and second mobile robots, the navigation modes of the first and second mobile robots are selected, and the first and second mobile robots are controlled to execute corresponding instructions according to the navigation modes. The execution of the corresponding instructions includes: if the navigation mode is an active navigator, executing SLAM instructions to generate a local map; if the navigation mode is a follower navigator, executing actions to follow the active navigator and detecting ambient light intensity. Based on the ambient light intensity detected during movement, determine whether the conditions for switching navigation modes are met: If the conditions are not met, the original navigation modes of the first and second mobile robots will be maintained, and the first and second mobile robots will be controlled to execute corresponding instructions according to the original navigation modes. If the conditions are met, the navigation mode and movement order of the first and second mobile robots will be switched, and the first and second mobile robots will be controlled to execute corresponding instructions according to the switched navigation mode. Based on the SLAM instructions executed by the first mobile robot and the SLAM instructions executed by the second mobile robot, all local maps generated by the first and second mobile robots are merged to generate a global map. The first mobile robot is a four-wheeled differential mobile robot, and the sensor type installed on the first mobile robot includes a depth camera; the second mobile robot is a Mecanum wheeled mobile robot, and the sensor type installed on the second mobile robot includes a lidar; the condition for determining whether the navigation mode is switched is to determine whether the light intensity is less than a first threshold, and the first threshold is based on the parameter setting of the depth camera. Both the first and second mobile robots are equipped with monocular cameras and targets; the following navigator's actions of following the active navigator include: Based on the robot coordinates corresponding to the active navigator and the follower navigator, the coordinates of the monocular camera and the target, the relative pose matrix of the first mobile robot and the second mobile robot is calculated. Based on the calculated relative pose matrix, the pose between the first and second mobile robots is obtained, enabling the navigator to follow.

2. The heterogeneous multi-robot collaborative SLAM method based on a cloud platform according to claim 1, characterized in that, The calculation of the relative pose matrix of the first mobile robot and the second mobile robot includes: Based on the map coordinates of the mobile robot corresponding to the navigator and the map coordinates of the monocular camera, the first transformation matrix of the monocular camera corresponding to the navigator to the mobile robot coordinate system is calculated. Based on the map coordinates of the mobile robot and the target corresponding to the active navigator, the second transformation matrix of the target corresponding to the active navigator relative to the coordinate system of the mobile robot is calculated. Based on the target relative coordinates of the active navigator and the monocular camera relative to the follower navigator, the third transformation matrix of the monocular camera relative to the target is calculated. The relative pose matrix is ​​calculated based on the first transformation matrix, the second transformation matrix, and the third transformation matrix.

3. The heterogeneous multi-robot collaborative SLAM method based on a cloud platform according to claim 2, characterized in that, The first transformation matrix for: in, D The translation of the monocular camera corresponding to the navigator on the Y-axis relative to its corresponding mobile robot.

4. The heterogeneous multi-robot collaborative SLAM method based on a cloud platform according to claim 2, characterized in that, The second transformation matrix for: in, This represents the translation of the active navigator target relative to its corresponding mobile robot on the Y-axis.

5. A heterogeneous multi-robot collaborative SLAM method based on a cloud platform according to claim 2, characterized in that, The third transformation matrix for: in, To follow the translation matrix of the monocular camera corresponding to the navigator, R To follow the rotation matrix of the monocular camera corresponding to the navigator; with the target corresponding to the active navigator. B1 Based on the baseline, follow the navigator's monocular camera C2 The unit of translation along the X-axis is a The unit of translation along the Y-axis is b The angle of rotation around the Z-axis of the monocular camera following the navigator is [value missing]. The target of the active navigator B1 The center point of the surface is the origin O1. The X-axis is set with the target ball pointing to the left front from the origin O1, and the Z-axis is set with the vertical upward direction from the upper surface. The Y-axis is obtained by defining the right-hand coordinate system.

6. The heterogeneous multi-robot collaborative SLAM method based on a cloud platform according to claim 5, characterized in that, The translation matrix of the monocular camera corresponding to the follower navigator for: The rotation matrix of the monocular camera corresponding to the follower navigator R for: Among them, the target corresponding to the active navigator B1 Based on the baseline, follow the navigator's monocular camera C2 The unit of translation along the X-axis is a The unit of translation along the Y-axis is b The angle of rotation around the Z-axis of the monocular camera following the navigator is [value missing]. The target of the active navigator B1 The center point of the surface is the origin O1. The X-axis is set with the target ball pointing to the left front from the origin O1, and the Z-axis is set with the vertical upward direction from the upper surface. The Y-axis is obtained by defining the right-hand coordinate system.

7. The heterogeneous multi-robot collaborative SLAM method based on a cloud platform according to claim 2, characterized in that, The relative pose matrix for: in, This is the first transformation matrix. This is the second transformation matrix. The third transformation matrix; with the target corresponding to the active navigator. B1 Based on the baseline, follow the navigator's monocular camera C2 The unit of translation along the X-axis is a The unit of translation along the Y-axis is b The angle of rotation around the Z-axis of the monocular camera following the navigator is [value missing]. The target of the active navigator B1 The center point of the surface is the origin O1. The X-axis is the target ball pointing to the left front of the origin O1. The Z-axis is the vertical upward direction of the upper surface. The Y-axis is defined by the right-hand coordinate system. D The translation of the monocular camera corresponding to the navigator relative to the mobile robot on the Y-axis; This represents the translation of the active navigator target relative to the mobile robot along the Y-axis.

8. A heterogeneous multi-robot collaborative SLAM system based on a cloud platform, characterized in that, include: The navigation mode acquisition module is used to acquire the navigation modes of the first mobile robot and the second mobile robot. The control module is used to control the first mobile robot and the second mobile robot to execute corresponding instructions based on the navigation mode obtained by the navigation mode acquisition module. The execution of the corresponding instructions includes: if the navigation mode is active navigator, executing SLAM instructions to generate a local map; if the navigation mode is follower navigator, executing actions to follow the active navigator and detecting ambient light intensity. The judgment module is used to determine whether the conditions for switching navigation modes are met based on the ambient light intensity detected during the movement. If not, the original navigation modes of the first and second mobile robots are maintained, and the first and second mobile robots are controlled to execute corresponding instructions according to the original navigation modes. If the conditions are met, the navigation modes and movement order of the first and second mobile robots are switched, and the first and second mobile robots are controlled to execute corresponding instructions according to the switched navigation modes. The fusion generation module is used to fuse all the local maps generated by the first mobile robot executing SLAM instructions and the second mobile robot executing SLAM instructions to generate a global map. The first mobile robot is a four-wheeled differential mobile robot, and the sensor type installed on the first mobile robot includes a depth camera; the second mobile robot is a Mecanum wheeled mobile robot, and the sensor type installed on the second mobile robot includes a lidar; the condition for determining whether the navigation mode is switched is to determine whether the light intensity is less than a first threshold, and the first threshold is based on the parameter setting of the depth camera. Both the first and second mobile robots are equipped with monocular cameras and targets; the following navigator performs the following movement with the active navigator, which includes: calculating the relative pose matrix of the first and second mobile robots based on the robot coordinates corresponding to the active navigator and the following navigator, the coordinates of the monocular camera and the target; and obtaining the pose between the first and second mobile robots based on the calculated relative pose matrix to achieve the following of the following navigator.

Citation Information

Patent Citations

  • Robot inter-detection method and robot inter-detection device

    CN106643701A

  • Mobile robot hybrid navigation method, equipment and storage device

    CN111780744A