A map updating method based on slam construction, a storage medium and an electronic device thereof

Abstract

The application discloses a map updating method based on SLAM construction, a storage medium and an electronic device, and comprises the following steps: acquiring current perception data of the surrounding environment of a current scene and a current positioning pose estimation through a sensor of a mobile robot, and creating a submap at any time; acquiring current perception point cloud data from the current perception data; based on the current positioning pose estimation, registering the current perception point cloud data, a map of an original scene and the submap respectively, and recording scores obtained by the registration as mscore and subscore respectively; if subscore>mscore and subscore is greater than a threshold value, it is considered that a change occurs within the range of the submap submap in the original scene, the submap is associated with a corresponding region in the map of the original scene, and the mobile robot continues to execute a current task by using the submap, so that the problem that the map cannot be updated in time and positioning is abnormal when a scene changes in the execution process is solved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision technology and robotics, and in particular to a map update method based on SLAM and its storage medium and electronic device. Background Technology

[0002] Simultaneous localization and mapping (SLAM) technology can capture environmental information using image data, perform real-time localization and pose estimation, and simultaneously construct a 3D environmental map. It boasts advantages such as low cost, accurate localization, and rich perceptual information, and is widely used in robot navigation and localization. This significantly improves the performance of mobile robots in dynamic environments, greatly enhancing their ability to recognize moving objects in the environment, and making the entire process more intelligent, safe, and reliable.

[0003] In indoor scenes containing moving targets, such as people, pets, toy cars, and other mobile robots, traditional visual SLAM combined with semantic information can eliminate the dynamic features of dynamic scenes, effectively improving the robot's intelligence. To achieve this goal, a common approach is to use semantic segmentation to extract pixel-level semantic information from images and label the semantic categories on the corresponding map point cloud, forming a semantic information mapping from two-dimensional image pixels to three-dimensional landmark points. Therefore, the completeness and accuracy of semantic information in a semantic map heavily depends on the effectiveness of image semantic segmentation. This improves positioning accuracy, system robustness, and the map's intelligence. Path planning is a fundamental aspect of mobile robot navigation, enabling the robot to safely and collision-free navigate around obstacles during movement. It reflects the robot's comprehensive judgment of the dynamic environment and allows for better intelligent decision-making.

[0004] Before a mobile robot begins its work, it uses SLAM technology to build a map of its current work environment. The completed map is stored in a specific format and can be used for localization and control applications during robot operation. Traditional SLAM mapping saves instantaneous scene information, which cannot meet the need for real-time updates based on scene changes. When scene changes necessitate map updates, these updates typically occur only after the robot has completed its current task. If the scene remains unchanged, localization matching is optimal. However, if significant changes occur in a part of the scene during the robot's task, and the updated map cannot be used for navigation, localization anomalies can easily occur. This can lead to the robot losing its ability to recognize scene changes and getting stuck in a loop of optimal localization matching during task execution. Summary of the Invention

[0005] Therefore, a map update method based on SLAM is needed to solve the problem that when the scene changes during execution, the map cannot be updated in time, resulting in positioning anomalies and causing the mobile robot to get stuck in a loop of finding the best positioning match while performing the current task.

[0006] To achieve the above objectives, this invention provides a map update method based on SLAM, which includes the following steps:

[0007] The mobile robot creates and stores a map of the original scene and a semantic map of the original scene (vecormap), and then performs the current task.

[0008] The mobile robot acquires current perception data of the surrounding environment and current positioning and pose estimation through its sensors, and creates a submap at any time.

[0009] Obtain the current sensed point cloud data spoints from the current sensed data;

[0010] Based on the current positioning pose estimation, the current perceived point cloud data spoints and the original scene map and submap are registered respectively, and the scores obtained from the registration are denoted as mscore and subscore respectively.

[0011] If subscore > mscore and subscore is greater than the set threshold, it means that the scene has been identified as having changed within the submap. The submap and the current localization pose estimate are retained. The submap is associated with the corresponding region in the original scene map. The mobile robot continues to perform the current task with the submap. Scene features are captured from the current perception data and matched with the scene feature model set models of the semantic map vecormap of the original scene. The current localization pose estimate of the mobile robot is labeled with the semantic map vecormap of the original scene, and the localization of the mobile robot is updated.

[0012] Otherwise, it means that no changes were detected in the scene within the submap. When the submap is full, delete the first half of the submap and keep the second half.

[0013] Furthermore, it also includes updating the original scene map based on the submap during idle time after the current task is completed, replacing the submap with the corresponding area in the original scene map.

[0014] Furthermore, the creation of the original scene map includes the following steps:

[0015] Following the planned trajectory path, walk around the scene once, and record perception data during the walking process;

[0016] A map is constructed using a laser SLAM mapping algorithm based on the perception data, denoted as lmap;

[0017] Based on the perception data and constrained by the laser SLAM trajectory path, a visual SLAM mapping algorithm is used to construct a map, denoted as vmap;

[0018] The lmap and vmap are overlaid to generate a map of the original scene.

[0019] Furthermore, the semantic map vecormap of the original scene includes the following steps:

[0020] By using perception data, deep learning is trained on obvious non-dynamic features of the scene to obtain a set of scene feature models.

[0021] Based on the scene feature model set (models), semantic annotation is performed on the original scene map (map) to generate a semantic map (vecormap) of the original scene.

[0022] Furthermore, the sensing data includes laser point cloud data, visual point cloud data, and camera RBG-D data.

[0023] Furthermore, the acquisition of the current perceived point cloud data spoints involves fusing the current laser point cloud data and the current visual point cloud data from the current perceived data.

[0024] Furthermore, when a change is detected in the scene within the submap, the system also optimizes the current localization pose estimation, retains the submap and the optimized current localization pose estimation, associates the submap with the corresponding region in the original scene's map, and the mobile robot continues to perform the current task using the submap. It then captures scene features from the current perception data and matches them with the scene feature model set models of the original scene's semantic map vecormap. The optimized current localization pose estimation is labeled with the original scene's semantic map vecormap, and the current localization is updated. At the same time, the corresponding optimized current localization pose in the submap is also updated, and error correction is performed on adjacent poses.

[0025] Furthermore, optimizing the current positioning pose estimation includes using the map-matched pose of the original scene as a benchmark, initiating nonlinear optimization, and optimizing the current positioning pose estimation.

[0026] A storage medium storing a computer program, which, when executed by a processor, performs the steps of the SLAM-based map update method described above.

[0027] An electronic device includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the steps of the SLAM-based map update method described above.

[0028] Unlike existing technologies, the above technical solution acquires current perception data and current positioning pose estimation of the surrounding environment from the mobile robot's sensors during the execution of the current task, and continuously creates a submap. Based on the current positioning pose estimation, the current perception point cloud data (spoints) is registered with the original scene map and the submap respectively. It identifies whether the scene has changed within the submap range. If it has changed, the submap is associated with the corresponding area in the original scene map. The mobile robot continues to execute the current task with the submap. The current positioning pose estimation of the mobile robot is labeled with the semantic map (vecormap) of the original scene to update the mobile robot's localization. That is, the features of the current frame of the submap are matched with the features of the original scene map to determine the robot's movement trajectory. This solves the problem that when the scene changes during execution, the map cannot be updated in time, resulting in localization anomalies and causing the mobile robot to get stuck in a loop of finding the best localization match during the execution of the current task. This ensures the mobile robot's ability to recognize the environment during task execution, increases safety and reliability, realizes effective pose estimation and route planning for the mobile robot in dynamic environments, and also meets the real-time obstacle avoidance requirements of the mobile robot. Attached Figure Description

[0029] Figure 1 This is a flowchart of a map update method based on SLAM according to the present invention;

[0030] Figure 2 The flowchart describes how to create and store the original scene map and the original scene semantic map vecormap for this invention. Detailed Implementation

[0031] To explain in detail the technical content, structural features, objectives, and effects of the technical solution, the following description is provided in conjunction with specific embodiments and accompanying drawings.

[0032] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0033] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0034] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0035] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0036] Unless otherwise specified, the use of terms such as “comprising,” “including,” “having,” or other similar expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a list of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.

[0037] In this application, expressions such as "greater than", "less than", and "exceeding" are understood to exclude the stated number; expressions such as "above", "below", and "within" are understood to include the stated number. Furthermore, in the description of the embodiments of this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups" and "multiple times", unless otherwise explicitly specified.

[0038] In the description of the embodiments of this application, the space-related expressions used, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "vertical," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," indicate the orientation or positional relationship based on the orientation or positional relationship shown in the specific embodiments or drawings. They are only for the purpose of describing the specific embodiments of this application or for the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0039] Unless otherwise expressly specified or limited, the terms "installation," "connection," "linking," "fixing," and "setting," as used in the description of the embodiments of this application, should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two components or the interaction between two components. For those skilled in the art to which this application pertains, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0040] Please see Figures 1 to 2 This invention provides a map update method based on SLAM, which includes the following steps:

[0041] The mobile robot creates and stores a map of the original scene and a semantic map of the original scene (vecormap), and then performs the current task.

[0042] The mobile robot acquires current perception data of the surrounding environment and current positioning and pose estimation through its sensors, and creates a submap at any time.

[0043] Obtain the current sensed point cloud data spoints from the current sensed data;

[0044] Based on the current positioning pose estimation, the current perceived point cloud data spoints and the original scene map and submap are registered respectively, and the scores obtained from the registration are denoted as mscore and subscore respectively.

[0045] If subscore > mscore and subscore is greater than the set threshold, it means that the scene has been identified as having changed within the submap. The submap and the current localization pose estimate are retained. The submap is associated with the corresponding region in the original scene map. The mobile robot continues to perform the current task with the submap. Scene features are captured from the current perception data and matched with the scene feature model set models of the semantic map vecormap of the original scene. The current localization pose estimate of the mobile robot is labeled with the semantic map vecormap of the original scene, and the localization of the mobile robot is updated.

[0046] Otherwise, it means that no changes were detected in the scene within the submap. When the submap is full, delete the first half of the submap and keep the second half.

[0047] The above-described technical solution, during the execution of the current task by the mobile robot, acquires current perception data and current positioning and pose estimation of the surrounding environment from the mobile robot's sensors, and continuously creates a submap. Based on the current positioning and pose estimation, the current perception point cloud data (spoints) is registered with the original scene map and the submap respectively. It identifies whether the scene has changed within the submap range. If it has changed, the submap is associated with the corresponding area in the original scene map. The mobile robot continues to execute the current task with the submap. The current positioning and pose estimation of the mobile robot is labeled with the semantic map (vecormap) of the original scene, and the mobile robot's positioning is updated. That is, the features of the current frame of the submap are matched with the features of the original scene map to determine the robot's movement trajectory. This solves the problem that when the scene changes during execution, the map cannot be updated in time, resulting in positioning anomalies and causing the mobile robot to get stuck in a loop of finding the best positioning match during the execution of the current task. This ensures the mobile robot's ability to recognize the environment during task execution, increases safety and reliability, realizes effective pose estimation and route planning for the mobile robot in dynamic environments, and also meets the real-time obstacle avoidance requirements of the mobile robot. The aforementioned current positioning pose estimation is composed of the previous positioning pose, plus the trajectory deviation from the previous moment to the current moment (the deviation inferred from multi-sensor data fusion, such as odometry + IMU pre-integration, etc.); the aforementioned mobile robot creates a submap during task execution, which is composed of multiple frames of current sensing data (laser point cloud + visual point cloud) and the current positioning pose estimation at a certain size; the aforementioned sensors acquire current sensing data of the surrounding environment of the current scene based on laser point cloud data, visual point cloud data, and visual RGBD data acquired by the mobile robot's LiDAR and camera; the current sensing point cloud data points are obtained by fusing the current laser point cloud data and the current visual point cloud data in the current sensing data; in the above, when no changes are detected in the scene within the submap, when the submap is full, the first half of the submap is deleted, and the second half is retained, in order to maintain scene continuity. If scene changes are detected, the submap and map are matched and corrected and stored after the scene is identified as unchanged and the features are obvious, in preparation for subsequent map updates.

[0048] During the execution of the current task, the original scene map is not updated. Only a submap is associated with the corresponding region in the original scene map. The mobile robot continues to execute the current task based on the submap to prevent the task from being interrupted due to potential anomalies caused by map updates during execution. Therefore, after the current task is completed, during idle time, the original scene map needs to be updated based on the submap, replacing the submap in the corresponding region of the original scene map.

[0049] The creation of the original scene map includes the following steps:

[0050] Following the planned trajectory path, walk around the scene once, and record perception data during the walking process;

[0051] A map is constructed using a laser SLAM mapping algorithm based on the perception data, denoted as lmap;

[0052] Based on the perception data and constrained by the laser SLAM trajectory path, a visual SLAM mapping algorithm is used to construct a map, denoted as vmap;

[0053] The lmap and vmap are overlaid to generate a map of the original scene.

[0054] By overlaying maps built from different sensory data, the accuracy and stability of the original scene map can be improved.

[0055] The semantic map vecormap of the original scene includes the following steps:

[0056] By using perception data, deep learning is trained on obvious non-dynamic features of the scene to obtain a set of scene feature models.

[0057] Based on the scene feature model set (models), semantic annotation is performed on the original scene map (map) to generate a semantic map (vecormap) of the original scene.

[0058] Feature points, such as key points and feature descriptions, are extracted from the perception data to create a map of the original scene. Semantic annotation is then performed on the map of the original scene to generate a semantic map (vecormap) of the original scene. This effectively improves the intelligence level of the robot, thereby enhancing positioning accuracy, system robustness, and map intelligence.

[0059] The aforementioned perception data includes laser point cloud data, visual point cloud data, and visual RGBD data acquired by LiDAR and cameras. Laser point cloud data is used for map construction using a laser SLAM mapping algorithm; visual point cloud data is used for map construction using a visual SLAM mapping algorithm with the laser SLAM trajectory path as constraints; and visual RGBD data is used for deep learning training on obvious non-dynamic features of the scene to obtain a scene feature model set (models).

[0060] A further improvement to the above embodiments includes optimizing the current positioning pose estimation when a change is detected within the submap area. The submap and the optimized current positioning pose estimation are retained, and the submap is associated with the corresponding region in the original scene's map. The mobile robot continues its current task using the submap, captures scene features from the current perception data, and matches them with the scene feature model set (models) of the original scene's semantic map (vecormap). The optimized current positioning pose estimation is labeled with the original scene's semantic map (vecormap) to update the current positioning. Simultaneously, the corresponding optimized current positioning pose in the submap is updated, and adjacent poses are also corrected for errors. The optimization of the current positioning pose estimation includes initiating nonlinear optimization based on the pose matched with the original scene's map map. This prevents potential accumulated errors within the submap by correcting them using the original scene's map map, ensuring that subsequent updates to the original scene's map map do not result in edge ghosting or other anomalies due to the fusion of old and new maps. The aforementioned nonlinear optimization can be based on the submap matching pose and map matching pose error. Through nonlinear optimization, such as the least squares method, the robot's pose and map can be optimized to make the position and map of the mobile robot more accurate.

[0061] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the SLAM-based map update method described above. In this embodiment, the storage medium can be a storage medium installed in an electronic device, allowing the electronic device to read the contents of the storage medium and achieve the effects of the present invention. Alternatively, the storage medium can be a separate storage medium connected to an electronic device, enabling the electronic device to read the contents of the storage medium and implement the steps of the SLAM-based map update method.

[0062] It should be noted that although the above embodiments have been described herein, this does not limit the scope of patent protection of the present invention. Therefore, any changes and modifications made to the embodiments described herein based on the innovative concept of the present invention, or equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included within the scope of patent protection of the present invention.

Claims

1. A map update method based on SLAM construction, characterized in that, Includes the following steps: The mobile robot creates and stores a map of the original scene and a semantic map of the original scene (vecormap), and then performs the current task. The mobile robot acquires current perception data of the surrounding environment and current positioning and pose estimation through its sensors, and creates a submap at any time. Obtain the current sensed point cloud data spoints from the current sensed data; Based on the current positioning pose estimation, the current perceived point cloud data spoints and the original scene map and submap are registered respectively, and the scores obtained from the registration are denoted as mscore and subscore respectively. If subscore > mscore and subscore is greater than the set threshold, it means that the scene has been identified as having changed within the submap. The submap and the current localization pose estimate are retained. The submap is associated with the corresponding region in the original scene map. The mobile robot continues to perform the current task with the submap. Scene features are captured from the current perception data and matched with the scene feature model set models of the semantic map vecormap of the original scene. The current localization pose estimate of the mobile robot is labeled with the semantic map vecormap of the original scene, and the localization of the mobile robot is updated. When a change is detected in the scene within the submap, the current localization pose estimation is optimized. The submap and the optimized current localization pose estimation are retained. The submap is associated with the corresponding region in the original scene map. The mobile robot continues to perform the current task with the submap. Scene features are captured from the current perception data and matched with the scene feature model set models of the semantic map vecormap of the original scene. The optimized current localization pose estimation is labeled with the semantic map vecormap of the original scene to update the current localization. At the same time, the corresponding optimized current localization pose in the submap is also updated, and error correction is performed on adjacent poses. Otherwise, it means that no changes were detected in the scene within the submap. When the submap is full, delete the first half of the submap and keep the second half. It also includes updating the original scene map based on the submap during idle time after the current task is completed, replacing the submap with the corresponding area in the original scene map.

2. The map update method based on SLAM construction according to claim 1, characterized in that, The creation of the original scene map includes the following steps: Following the planned trajectory path, walk around the scene once, and record perception data during the walking process; A map is constructed using a laser SLAM mapping algorithm based on the perception data, denoted as lmap; Based on the perception data and constrained by the laser SLAM trajectory path, a visual SLAM mapping algorithm is used to construct a map, denoted as vmap; The lmap and vmap are overlaid to generate a map of the original scene.

3. The map update method based on SLAM construction according to claim 2, characterized in that, The semantic map vecormap of the original scene includes the following steps: By using perception data, deep learning is trained on obvious non-dynamic features of the scene to obtain a set of scene feature models. Based on the scene feature model set (models), semantic annotation is performed on the original scene map (map) to generate a semantic map (vecormap) of the original scene.

4. The map update method based on SLAM construction according to claim 2, characterized in that, The sensing data includes laser point cloud data, visual point cloud data, and camera RBG-D data.

5. The map update method based on SLAM construction according to claim 1, characterized in that, The acquisition of the current perceived point cloud data spoints involves fusing the current laser point cloud data and the current visual point cloud data from the current perceived data.

6. The map update method based on SLAM construction according to claim 5, characterized in that, Optimizing the current positioning pose estimation includes using the map-matched pose of the original scene as a reference, initiating nonlinear optimization, and optimizing the current positioning pose estimation.

7. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the SLAM-based map update method according to any one of claims 1-6.

8. An electronic device, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the map update method based on SLAM construction as described in any one of claims 1-6.

Images