Robot mapping method and device, robot, medium and product

Abstract

The invention provides a robot mapping method and device, a robot, a medium and a product, and relates to the field of intelligent robotics.The method comprises the steps that in the process that the robot constructs a global raster map based on a laser SLAM module, in response to the situation that laser sensor data of a current frame does not meet a preset condition, a visual SLAM module in an initial state is started, and the visual SLAM module is started; in the initial state, the visual SLAM module responds to the received starting command and can be quickly started for positioning and mapping, so that the time required for mapping is shortened, and the mapping efficiency is improved. And the visual SLAM module is started to work only when the laser SLAM module cannot meet the conditions, so that the high calculation burden caused by simultaneous operation of the two SLAM modules is avoided, the calculation power demand is reduced, and the robot is suitable for being operated on an embedded platform.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent robots, and in particular, to a robot mapping method, device, robot, medium, and product. Background Art

[0002] The SLAM (Simultaneous Localization and Mapping) technology is an important field in modern robot technology and has been widely applied, especially in cleaning robots. When the SLAM technology is applied in a cleaning robot, for example, in a laser sensor or a vision sensor, the robot can navigate autonomously in an unknown environment and simultaneously achieve localization and mapping.

[0003] In related technologies, a laser sensor can be used to obtain the current laser frame of the target environment. When the matching between the current laser frame and the target local map fails, that is, when the laser SLAM module fails to localize, visual information obtained by a vision sensor can be used for assisted localization, that is, the current video frame of the target environment is obtained by the vision sensor for localization, so as to complete the mapping of the robot.

[0004] However, the above-mentioned mapping method of the robot has a large demand for computing power and a long mapping time. Summary of the Invention

[0005] The present disclosure provides a robot mapping method, device, robot, medium, and product, which can perform accurate mapping and localization using a laser SLAM module and a vision SLAM module with low computing power overhead. By designing to run only one SLAM module during the mapping process and starting the vision SLAM module only when the laser SLAM module cannot meet the conditions, the overall computing power requirement is reduced, and when the vision SLAM module is started, the vision SLAM module has been initialized in advance, greatly reducing the mapping time required.

[0006] In a first aspect, the present disclosure provides a robot mapping method. The robot includes a laser simultaneous localization and mapping (SLAM) module and a vision SLAM module. The laser SLAM module is used to construct a global grid map based on laser sensor data and determine the position of the robot in the global grid map. The vision SLAM module is used to construct a global grid map based on vision sensor data and determine the position of the robot in the global grid map. The method includes:

[0007] During the process of the robot constructing a global grid map based on the laser SLAM module, in response to the laser sensor data of the current frame not meeting the preset conditions, the visual SLAM module in the initialization state is started, so that the visual SLAM module uses the visual sensor data of the current frame for positioning to achieve the construction of the global grid map. In the initialization state, the visual SLAM module starts to work in response to receiving the start command.

[0008] In the present disclosure, the robot only runs the laser SLAM module under normal circumstances, saving computing power and power resources. Only when the laser sensor data is unreliable, the visual SLAM module is started. This on-demand startup strategy can effectively reduce the overall computational burden and ensure that the robot continuously performs effective positioning and mapping. This design improves the reliability of the robot under various environmental conditions. Since the visual SLAM module is in the initialization state, it can be quickly started for positioning and mapping after receiving the start command, reducing the time required for mapping and avoiding long interruptions.

[0009] Therefore, by the design of starting the visual SLAM module only when the laser SLAM module cannot meet the conditions, the high computational burden of running two SLAM modules simultaneously is avoided. This on-demand startup strategy significantly reduces the computing power requirements, making the robot more suitable for running on an embedded platform. Moreover, since it is not necessary to run two high-load SLAM modules simultaneously, it can be implemented on lower-performance hardware, reducing the hardware cost. Also, since it is not necessary to run two modules simultaneously, the power consumption of the robot is correspondingly reduced, which is particularly important for battery-powered robots.

[0010] It should also be understood that in the present disclosure, by dynamically adjusting the use of the SLAM module according to the real-time data quality, the visual SLAM module is only started when necessary, and unnecessary initialization and processing time are reduced when starting, ensuring that the robot can respond quickly and reliably complete positioning and map construction under different environmental conditions, reducing the waiting time and improving the overall efficiency.

[0011] Optionally, the method further includes:

[0012] In response to the end of the positioning of the visual SLAM module, the laser SLAM module determines whether to re-enable the laser SLAM module for positioning based on the laser sensor data of the next frame obtained, for constructing the global grid map.

[0013] Therefore, by switching back to the laser SLAM module when the preset conditions are appropriate, the high-precision characteristics of the laser SLAM module can be utilized to improve the accuracy and efficiency of positioning and mapping. This dynamic switching mechanism can ensure that the robot uses the appropriate sensor module under different environmental conditions, thereby optimizing the use of computing resources and power. Moreover, by flexibly switching between the visual SLAM module and the laser SLAM module, the robot can better adapt to environmental changes, enhancing the overall robustness and reliability.

[0014] Optionally, the laser sensor data includes point cloud data, and determining whether to re-enable the laser SLAM module for positioning based on the acquired laser sensor data of the next frame includes:

[0015] Determining the first pose information of the robot in the global grid map based on the point cloud data of the next frame;

[0016] Determining the first reliability of the orientation of the robot in the global grid map based on the first feature in the first pose information;

[0017] Determining whether to re-enable the laser SLAM module for positioning based on the first reliability.

[0018] Therefore, by evaluating the first reliability of the point cloud data, the laser SLAM module can be re-enabled when the first reliability is high, so that the pose information of the robot can be determined more accurately by using high-quality point cloud data, thereby improving the positioning accuracy. Especially when the environmental conditions improve, re-enabling the laser SLAM module can make full use of its advantages in specific scenarios to ensure the adaptability of the robot in a changing environment.

[0019] Optionally, the method further includes:

[0020] In response to the first reliability being greater than the first threshold, re-enabling the positioning function in the laser SLAM module for positioning and turning off the positioning function in the visual SLAM module;

[0021] In response to the first reliability not being greater than the first threshold, continue to use the positioning function in the visual SLAM module to perform positioning using the visual sensor data of the next frame and turn off the positioning function in the laser SLAM module.

[0022] In this way, by selecting to use an appropriate positioning function based on the first reliability corresponding to the laser sensor data, not only can computing resources be effectively managed, unnecessary computing burdens be avoided, and unnecessary positioning functions be turned off, reducing the overall computing power requirement while also saving power consumption, thereby extending the battery life of the robot. Moreover, in the present disclosure, when it is determined that the first reliability is high, the positioning function of the laser SLAM module is used for high-precision positioning to ensure that the robot can achieve an ideal positioning accuracy. This mechanism of flexibly selecting the positioning function according to the first reliability can ensure that the robot provides reliable services in a changing environment.

[0023] Optionally, it is determined that the laser sensor data of the current frame does not meet the preset conditions in the following manner:

[0024] Based on the acquired laser sensor data of the previous frame and the laser sensor data of the current frame, determine the target pose;

[0025] Determine the second reliability of the laser sensor data of the current frame;

[0026] Based on the second reliability and the target pose, determine the third reliability of the target pose;

[0027] In response to the third reliability being less than the second threshold, determine that the laser sensor data of the current frame does not meet the preset conditions.

[0028] In this way, through the multi-level reliability evaluation of the laser sensor data, the quality of the data can be judged more accurately, ensuring that high-quality sensor data is used for positioning and map construction. When the sensor data is unreliable, the robot can promptly identify and take measures such as switching to the visual SLAM module for positioning, which can reduce positioning errors and task interruptions, improve the overall operation efficiency and stability, thereby enhancing the robot's robustness under different environmental conditions. And by avoiding using unreliable sensor data, computing resources and power can be saved, and unnecessary processing and error correction can be avoided.

[0029] Optionally, based on the acquired laser sensor data of the previous frame and the laser sensor data of the current frame, determining the target pose includes:

[0030] Based on the laser sensor data of the previous frame and the laser sensor data of the current frame, determine at least one predicted pose;

[0031] For each predicted pose among the at least one predicted pose, project the laser sensor data of the current frame onto the global grid map to determine the matching degree result of each predicted pose in the global grid map;

[0032] Based on the matching degree result, determine the target pose.

[0033] In this way, by evaluating and comparing multiple predicted poses, a more realistic predicted pose can be selected, thereby improving the accuracy and reliability of positioning. Moreover, by using the matching of laser sensor data and the global grid map to intelligently select the target pose, positioning errors and task interruptions can be reduced, and operation efficiency and stability can be improved.

[0034] Optionally, determining the second reliability of the laser sensor data of the current frame includes:

[0035] Determining the second pose information of the robot in the global grid map based on the point cloud data of the current frame;

[0036] Extracting the second feature in the second pose information and determining the number of second features;

[0037] Determining the second reliability of the laser sensor data of the current frame based on the second feature and the number of second features.

[0038] In this way, by analyzing the second feature and its number in the second pose information to determine the second reliability of the laser sensor data of the current frame, the richness and quality of the laser sensor data can be more accurately evaluated, ensuring that the robot uses high-quality and feature-rich data for positioning and map construction. And through the second reliability evaluation, it is ensured that when the point cloud data is unreliable, it can be recognized and measures can be taken in time, such as switching to the visual SLAM module for positioning, improving the robustness of the robot under different environmental conditions.

[0039] Optionally, the method further includes:

[0040] In response to the end of the positioning of the visual SLAM module, sending the positioning result of the visual SLAM module to the laser SLAM module;

[0041] After receiving the positioning result, the laser SLAM module turns off the positioning function in the laser SLAM module.

[0042] In the present disclosure, by turning off the positioning function in the laser SLAM module after the positioning of the current frame by the visual SLAM module is completed, computing resources and power consumption can be saved. This resource optimization helps to extend the battery life of the robot, and it can also avoid repeated positioning calculations of the laser SLAM module, reduce the dependence on the laser SLAM module, and improve the adaptability of the robot under different environmental conditions. Therefore, by coordinating the operations of different SLAM modules, data redundancy and processing conflicts can be reduced, ensuring the consistency and accuracy of positioning information, and thus the switching and resource allocation of the SLAM module can be intelligently managed, reducing interruptions and errors.

[0043] Optionally, the visual sensor data includes image data. The visual SLAM module includes an image matching function and a pose solution function. The image matching function is used to perform feature matching on the image data of two adjacent frames, and the pose solution function is used to determine the pose information based on the result of the feature matching to ensure that the visual SLAM module performs positioning based on the pose information. And the method further includes:

[0044] When using the laser SLAM module for positioning, the image matching function and the pose solution function are in the off state.

[0045] In this way, by turning off the relevant functions in the visual SLAM module when performing laser SLAM positioning, not only can computing resources and power be saved, and the battery life of the robot can be extended, but also the computing burden of the robot can be reduced, and the complexity and potential conflicts brought by simultaneously processing multiple sensor data can be avoided.

[0046] Optionally, the construction of the global grid map includes:

[0047] In response to the laser sensor data of the current frame meeting the preset conditions, using the positioning result of the laser SLAM module to construct the global grid map;

[0048] In response to the laser sensor data of the current frame not meeting the preset conditions, using the positioning result of the visual SLAM module to construct the global grid map.

[0049] In this way, by selecting the positioning result of the most reliable SLAM module to construct the global grid map, the accuracy and reliability of the global grid map can be improved, and by intelligently selecting the map construction strategy, the computing resources and power consumption can be optimized, the map construction error and task interruption can be reduced, and the operation efficiency and stability can be improved.

[0050] Optionally, the laser SLAM module is deployed in the laser sensor, and the visual SLAM module is deployed in the visual sensor. The laser sensor is used to transmit the laser sensor data to the laser SLAM module, and the visual sensor is used to transmit the visual sensor data to the visual SLAM module. The laser sensor and the visual sensor work alternately during the map construction process.

[0051] Therefore, by alternately using the laser sensor and the visual sensor, the robot can flexibly select the appropriate sensor for map construction under different environmental conditions, improving the map construction efficiency. And this alternating working mechanism not only reduces the computing burden of simultaneously processing multiple sensor data, optimizes the use of computing resources and power, but also reduces the continuous working time of the sensor, helps to reduce the wear and energy consumption of the sensor, and thus extends the service life of the robot.

[0052] In a second aspect, the present disclosure provides a robot mapping device. The robot includes a laser simultaneous localization and mapping (SLAM) module and a visual SLAM module. The laser SLAM module is configured to construct a global grid map based on laser sensor data and determine the position of the robot in the global grid map. The visual SLAM module is configured to construct a global grid map based on visual sensor data and determine the position of the robot in the global grid map. The device includes:

[0053] A start-up module, which is configured to, during the process of the robot constructing a global grid map based on the laser SLAM module, in response to the laser sensor data of the current frame not meeting a preset condition, start the visual SLAM module in an initialization state, so that the visual SLAM module uses the visual sensor data of the current frame for positioning to achieve the construction of the global grid map. In the initialization state, the visual SLAM module starts to work in response to receiving a start command.

[0054] In a third aspect, the present disclosure provides a robot, including: a laser SLAM module, a visual SLAM module, a processor, and a memory communicatively connected to the processor;

[0055] The laser SLAM module is configured to construct a global grid map based on laser sensor data and determine the position of the robot in the global grid map. The visual SLAM module is configured to construct a global grid map based on visual sensor data and determine the position of the robot in the global grid map;

[0056] The memory stores computer-executable instructions;

[0057] The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of the first aspect.

[0058] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method according to any one of the first aspect when executed by a processor.

[0059] In a fifth aspect, the present disclosure provides a computer program product including a computer program, which implements the method according to any one of the first aspect when executed by a processor.

[0060] It should be noted that the second to fifth aspects of the present disclosure correspond to the technical solutions of the first aspect of the present disclosure. The beneficial effects obtained by each aspect and the corresponding feasible implementation manners are similar and will not be elaborated herein.

[0061] In summary, the present disclosure provides a method, apparatus, robot, medium, and product for robot mapping. The robot integrates a laser SLAM module and a visual SLAM module. Under normal circumstances, the robot mainly relies on the laser SLAM module to construct and localize the global grid map because the mapping accuracy of the laser SLAM module is higher. However, in some cases, such as when the laser sensor data does not meet the preset conditions, the laser SLAM module may not be able to work effectively. Therefore, during the process of constructing the global grid map based on the laser SLAM module, when the robot determines that the laser sensor data of the current frame detected by the laser SLAM module does not meet the preset conditions, the visual SLAM module in the initialization state will be started, so that the visual SLAM module can use the visual sensor data of the current frame for localization and map construction. This mechanism ensures that the robot can still continue to perform effective localization and mapping even when the laser SLAM module cannot work properly.

[0062] Since running the laser SLAM module and the visual SLAM module simultaneously will significantly increase the computational burden, in the present disclosure, by designing a mechanism that only runs one laser SLAM module during the mapping process and starts another visual SLAM module in the initialization state when the laser sensor data of the current frame does not meet the preset conditions, the computing power consumption can be effectively reduced, making it suitable for embedded platforms with limited computing power. And because the visual SLAM module is in the initialization state where it can start localization and mapping at any time, that is, the visual SLAM module has completed the initialization steps and can start working immediately after receiving the start command, and the initialization step is a necessary prerequisite for starting the state of localization and mapping. In this way, by completing the initialization in advance, when the laser SLAM module fails, it can quickly switch to the visual SLAM module for localization and mapping, reducing the response time and the possibility of mapping interruption, and thus reducing the time required for mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0064] Figure 1 It is a partial structural schematic diagram of a robot provided by an embodiment of the present disclosure;

[0065] Figure 2 It is a partial structural schematic diagram of another robot provided by an embodiment of the present disclosure;

[0066] Figure 3 It is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0067] Figure 4Flow diagram of a method for a robot to build a map provided by an embodiment of the present disclosure;

[0068] Figure 5 Structural diagram of a device for a robot to build a map provided by an embodiment of the present disclosure;

[0069] Figure 6 Structural diagram of a robot provided by an embodiment of the present disclosure.

[0070] Through the above-mentioned drawings, specific embodiments of the present disclosure have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present disclosure in any way, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0071] In order to facilitate a clear description of the technical solutions of the embodiments of the present disclosure, in the embodiments of the present disclosure, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and roles. For example, the first device and the second device are only used to distinguish different devices, and do not limit their sequence. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and terms such as "first" and "second" do not necessarily mean different.

[0072] It should be noted that in the present disclosure, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present disclosure should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0073] In the present disclosure, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B may be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or its similar expression hereinafter refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c may represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, c may be single or multiple.

[0074] There are mainly two implementation schemes for SLAM technology: laser SLAM and visual SLAM. Laser SLAM usually uses a laser sensor to obtain distance information of the surrounding environment for positioning and mapping. It has the advantages of high precision and reliability, and performs excellently especially in low-light or no-light environments. However, the cost of laser sensors is relatively high, and the data processing requires a large amount of computing resources.

[0075] On the other hand, visual SLAM relies on visual sensors such as cameras, and extracts environmental features through image processing technology for positioning and mapping. The hardware cost of the visual SLAM solution is relatively low, but it may face challenges in environments with large light changes, and also requires high computing power to process complex image data.

[0076] In related technologies, a laser sensor can be used to obtain the current laser frame of the target environment. When the matching between the current laser frame and the target local map fails, that is, when the laser SLAM module fails to locate, visual information obtained by a visual sensor can be used for assisted positioning, that is, the current video frame of the target environment is obtained by the visual sensor for positioning, so as to complete the mapping of the robot.

[0077] However, during the robot mapping process, the computing power requirements for running both the laser SLAM module and the visual SLAM module are relatively large, especially it is difficult to be applicable to an embedded platform with limited computing power. Moreover, the startup and initialization of the visual sensor may take a certain amount of time, resulting in a relatively long mapping time.

[0078] It should be noted that on an embedded platform, due to the limitations of computing resources and power supply, running both the laser SLAM module and the visual SLAM module solution may cause overload, affecting the real-time performance and battery life of the robot. Therefore, in practical applications, it is usually necessary to choose between the laser SLAM module and the visual SLAM module in order to achieve the best performance under limited computing power conditions.

[0079] In view of the above problems, the present disclosure provides a robot mapping method, which can use the laser SLAM module and the visual SLAM module for accurate mapping and positioning with low computing power overhead. The robot integrates the laser SLAM module and the visual SLAM module. Under normal circumstances, the robot mainly relies on the laser SLAM module to construct and position the global grid map because the mapping accuracy of the laser SLAM module is higher.

[0080] However, in some cases, for example, when the laser sensor data does not meet the preset conditions, the laser SLAM module may not work effectively. Therefore, during the process of constructing a global grid map based on the laser SLAM module, when the robot determines that the laser sensor data of the current frame detected by the laser SLAM module does not meet the preset conditions, the visual SLAM module in the initialization state will be started, so that the visual SLAM module can use the visual sensor data of the current frame for positioning and map construction. This mechanism ensures that the robot can still continue to perform effective positioning and mapping even when the laser SLAM module fails to work properly.

[0081] Since running the laser SLAM module and the visual SLAM module simultaneously will significantly increase the computational burden, in the present disclosure, by designing a mechanism that only runs one laser SLAM module during the mapping process and starts another visual SLAM module in the initialization state when the laser sensor data of the current frame does not meet the preset conditions, the computational power consumption can be effectively reduced, making it suitable for embedded platforms with limited computational power. And because the visual SLAM module is in the initialization state where it can start positioning and mapping at any time, that is, the visual SLAM module has completed the initialization steps and can start working immediately after receiving the start command, and the initialization step is a necessary prerequisite for starting the positioning and mapping state. In this way, by completing the initialization in advance, when the laser SLAM module fails, it can quickly switch to the visual SLAM module for positioning and mapping, reducing the response time and the possibility of map construction interruption, and thus reducing the time required for map construction.

[0082] Optionally, the robot mapping method provided in the present disclosure is applied to a robot, which can be a cleaning robot such as a floor cleaning robot, an industrial mobile robot such as a material handling robot, a service robot such as a navigation robot, a detection and rescue robot, etc. The embodiments of the present disclosure do not specifically limit the type of the robot, and as long as it is a robot with a mapping function, the method of the present disclosure can be applied.

[0083] Exemplarily, Figure 1 is a partial structural schematic diagram of a robot provided by an embodiment of the present disclosure. As Figure 1 shown, the robot 100 includes a laser SLAM module 101 and a visual SLAM module 102. The laser SLAM module 101 is used to construct a global grid map based on the laser sensor data and determine the position of the robot 100 in the global grid map. The visual SLAM module 102 is used to construct a global grid map based on the visual sensor data and determine the position of the robot 100 in the global grid map.

[0084] Among them, the laser SLAM module 101 determines the position of the robot 100 in the map by matching real-time laser sensor data such as point cloud data with the global grid map, and the visual SLAM module 102 determines the position of the robot 100 in the map by matching real-time visual sensor data such as image data with the global grid map.

[0085] Optionally, the laser SLAM module 101 includes a positioning function. The positioning function in the laser SLAM module 101 is used for positioning based on laser sensor data. The visual SLAM module 102 includes a positioning function. The positioning function in the visual SLAM module 102 is used for positioning based on visual sensor data.

[0086] Optionally, the visual SLAM module 102 may further include an image matching function and a pose solving function. The image matching function is used for feature matching of image data of two adjacent frames, and the pose solving function is used for determining pose information according to the result of feature matching to ensure that the visual SLAM module 102 performs positioning based on the pose information.

[0087] Among them, the image matching function extracts image features from image data of two adjacent frames, such as corner points, edges or other significant image features. The embodiments of the present disclosure do not make specific limitations thereto. Further, the image matching function matches the image features of the current frame with the image features of the previous frame. The matching process may be to calculate the similarity between image features to determine the matching result, etc. The embodiments of the present disclosure do not make specific limitations to the matching process. The above is only an example for illustration.

[0088] The pose solving function performs pose estimation according to the feature matching result provided by the image matching function. By analyzing the feature matching result, the position and pose (pose information) of the robot 100 in the current frame are calculated. Furthermore, the visual SLAM module 102 uses the pose information provided by the pose solving function for positioning, and this pose information is used to update the position of the robot 100 in the global grid map.

[0089] Optionally, the pose solving function may also output the pose information to the mapping function in the laser SLAM module to assist in completing the construction of the global grid map.

[0090] In this way, through the image matching function and the pose solving function for accurate feature matching and pose estimation, the visual SLAM module 102 performs high-precision positioning.

[0091] Optionally, Figure 2 This is a partial structural schematic diagram of another robot provided by the embodiments of the present disclosure, such as Figure 2As shown in the figure, the robot 100 includes a laser SLAM module 101 and a visual SLAM module 102. The laser SLAM module 101 is deployed in the laser sensor 201, and the visual SLAM module 102 is deployed in the visual sensor 202. The laser sensor 201 is used to transmit laser sensor data to the laser SLAM module 101, and the visual sensor 202 is used to transmit visual sensor data to the visual SLAM module 102. The laser sensor 201 and the visual sensor 202 work alternately during the mapping process.

[0092] Among them, the laser sensor 201 and the visual sensor 202 are arranged on the side of the robot 100, and the robot 100 can be provided with one or more laser sensors 201 and one or more visual sensors 202. The present disclosure embodiment does not specifically limit the specific positions and numbers of the laser sensor 201 and the visual sensor 202.

[0093] It should be noted that the laser sensor 201 can collect point cloud data in the environment, and the visual sensor 202 can collect image data in the environment. The present disclosure embodiment does not specifically limit the order of data collection by the laser sensor 201 and the visual sensor 202. During the mapping process of the robot 100, the laser sensor 201 and the visual sensor 202 work alternately, that is, the laser sensor 201 and the visual sensor 202 are alternately triggered, and thus point cloud data and image data are alternately obtained.

[0094] Since the laser sensor 201 and the visual sensor 202 can work alternately, this means that when the data of one sensor is used to construct the global grid map, the other sensor can be in a standby or low-power state. The low-power state refers to turning off at least one function based on the resource consumption situation. For example, when mapping using the laser SLAM module 101 in the laser sensor 201, the positioning function in the visual SLAM module 102 can be turned off.

[0095] Therefore, by alternately using the laser sensor 201 and the visual sensor 202, the robot 100 can flexibly select a suitable sensor for map construction under different environmental conditions, improving the mapping efficiency. And this alternate working mechanism not only reduces the computational burden of simultaneously processing multiple sensor data, optimizes the use of computational resources and power, but also reduces the continuous working time of the sensors, helping to reduce the wear and energy consumption of the sensors, thereby extending the service life of the robot 100.

[0096] Exemplarily, Figure 3 is a schematic diagram of an application scenario provided by the present disclosure embodiment, such as Figure 3As shown, the application scenario includes a floor cleaning robot 301 and a user's terminal device 302. The floor cleaning robot 301 and the terminal device 302 establish a communication connection for real-time communication. The floor cleaning robot 301 includes a laser sensor and a vision sensor. A laser SLAM module is set in the laser sensor, and a vision SLAM module is set in the vision sensor.

[0097] As Figure 1 shown, taking the floor cleaning robot 301 mapping the area within the living room as an example. During the mapping process, the floor cleaning robot 301 alternately triggers the laser sensor and the vision sensor to obtain point cloud data and image data in the living room environment. Then, the laser sensor transmits the point cloud data to the laser SLAM module, and the vision sensor transmits the image data to the vision SLAM module. By default, the floor cleaning robot 301 uses the laser SLAM module to construct a global grid map using the point cloud data.

[0098] However, in response to the point cloud data of the current frame not meeting the preset conditions, the vision SLAM module in the initialization state can be activated so that the vision SLAM module uses the image data of the current frame for positioning and then constructs a global grid map. Since the vision SLAM module is in the initialization state, that is, it can start the positioning and mapping work immediately after receiving the start command without performing the initialization steps of the vision SLAM module. The initialization steps include the initialization of the state variables that the vision SLAM module needs to maintain, such as initializing the scale information and other steps. Therefore, it can be ensured that the vision SLAM module can always take over from the laser SLAM module for positioning when the point cloud data of the current frame does not meet the preset conditions. In this way, the time required for mapping can be optimized and the mapping efficiency can be improved.

[0099] Optionally, after the vision SLAM module finishes positioning using the image data of the current frame, the laser SLAM module can determine whether to reactivate the laser SLAM module for positioning based on the obtained point cloud data of the next frame. If it is determined to reactivate, the laser SLAM module is used for the next frame of positioning. If it is determined not to activate the laser SLAM module, the vision SLAM module continues to be used for the next frame of positioning. The above process is repeated continuously until the global grid map of the living room area is constructed.

[0100] Optionally, after completing the mapping of the living room area, the floor cleaning robot 301 can send the constructed global grid map of the living room area to the user's terminal device 302 for visual display and storage, so that the user can view the mapping result of the floor cleaning robot 301.

[0101] Among them, the terminal device can also be called User Equipment (UE), Mobile Station (MS), Mobile Terminal, Terminal, intelligent terminal, etc. In practical applications, the terminal device is, for example: desktop computer, notebook, Personal Digital Assistant (PDA), smart phone, tablet computer, vehicle-mounted device, wearable device (such as smart watch, smart bracelet), smart home device (such as smart display device), etc. The embodiments of the present disclosure do not specifically limit the type of the terminal device.

[0102] It should be noted that the embodiments of the present disclosure do not specifically limit the application scenarios of the robot mapping method. In different application scenarios, the types of robots may be different, and the specific areas or specific modes of robot mapping are different. The embodiments of the present disclosure do not specifically limit this, and the above are only illustrative examples.

[0103] The following uses specific embodiments to describe in detail the technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of the present disclosure with reference to the drawings.

[0104] Figure 4 It is a schematic flowchart of a robot mapping method provided by an embodiment of the present disclosure. As Figure 4 shown, the robot mapping method can be applied to Figure 1 - Figure 2 the robot shown below. The robot mapping method includes the following steps:

[0105] S401. During the process of the robot constructing a global grid map based on the laser SLAM module, in response to the laser sensor data of the current frame not meeting the preset conditions, start the visual SLAM module in the initialization state. In the initialization state, the visual SLAM module starts to work in response to receiving the start command.

[0106] In the embodiments of the present disclosure, when the robot is running normally, it mainly relies on the laser SLAM module to construct a global grid map, that is, the laser SLAM module uses the laser sensor data provided by the laser sensor to achieve high-precision environment mapping and positioning. The embodiments of the present disclosure do not specifically limit the type of the laser sensor. For example, the laser sensor can be a laser ranging sensor, a lidar, etc.

[0107] In some cases, the laser sensor data may not meet the preset conditions, which may be data loss, excessive noise, signal interference, or the inability to accurately determine the position and posture of the data. The embodiments of the present disclosure do not specifically limit the preset conditions.

[0108] Therefore, when it is detected that the laser sensor data of the current frame does not meet the preset conditions, the robot will start the visual SLAM module. The visual SLAM module is on standby in the initialization state and has completed the initialization steps. After receiving the start command, it can start positioning and mapping immediately without the need for the initialization step. The initialization step is a necessary prerequisite for the visual SLAM module to start positioning and mapping.

[0109] It can be understood that the visual SLAM module can use the visual sensor information captured by the visual sensor to provide additional environmental perception capabilities when the laser sensor fails or the data is unreliable. This design allows the robot to flexibly adapt to different environmental conditions. For example, in an environment with large light changes, the visual SLAM module can supplement the deficiencies of the laser SLAM module.

[0110] S402, the visual SLAM module uses the visual sensor data of the current frame for positioning to achieve the construction of a global grid map.

[0111] In this step, the construction of the global grid map refers to updating the global grid map using the positioning results of the visual SLAM module. When the robot starts to build the map, the global grid map is the first frame of point cloud data collected by the laser sensor that the robot will receive. It is at the zero point position of the world coordinate system by default. The global grid map is updated by converting the first frame of point cloud data to the coordinate system of the global grid map.

[0112] In the present disclosure, the robot only runs the laser SLAM module under normal circumstances, saving computing power and power resources. The visual SLAM module is only started when the laser sensor data is unreliable. This on-demand startup strategy can effectively reduce the overall computing burden and ensure that the robot continues to perform effective positioning and mapping. This design improves the reliability of the robot under various environmental conditions. Since the visual SLAM module is in an initialized state, it can quickly start positioning and mapping after receiving the start command, reducing the time required for mapping and avoiding long interruptions.

[0113] Therefore, by designing to start the visual SLAM module only when the laser SLAM module fails to meet the conditions, the high computational burden of running two SLAM modules simultaneously is avoided. This on-demand startup strategy significantly reduces the computing power requirement, making the robot more suitable for running on an embedded platform. Moreover, since there is no need to run two high-load SLAM modules simultaneously, it can be implemented on hardware with lower performance, which can also reduce the hardware cost. And since there is no need to run two modules simultaneously, the power consumption of the robot is also correspondingly reduced, which is particularly important for battery-powered robots.

[0114] It should also be understood that in the present disclosure, by dynamically adjusting the use of the SLAM module according to real-time data quality, the visual SLAM module is only started when necessary, and unnecessary initialization and processing time are reduced when starting, ensuring that the robot can respond quickly and reliably complete positioning and map construction under different environmental conditions, reducing waiting time and improving the overall efficiency.

[0115] Optionally, the method further includes:

[0116] In response to the end of the positioning of the visual SLAM module, the laser SLAM module determines whether to re-enable the laser SLAM module for positioning based on the laser sensor data of the next frame obtained, for constructing a global grid map.

[0117] In this step, after the visual SLAM module completes its positioning task, that is, after successfully updating the global grid map, the robot evaluates whether it can switch back to the laser SLAM module. That is, the laser SLAM module determines whether it meets the preset requirements based on the laser sensor data of the next frame obtained. If the conditions are met, the laser SLAM module will be re-enabled to continue high-precision positioning and construction of the global grid map.

[0118] Among them, re-enabling the laser SLAM module may refer to restarting at least one function in the laser SLAM module. The at least one function includes a positioning function, a mapping function, a reliability evaluation function, etc. The present disclosure embodiment does not specifically limit the at least one re-enabled function, and it can be set based on actual situations.

[0119] It should be noted that when the environmental conditions improve, re-enabling the laser SLAM module can make full use of its advantages in specific scenarios. For example, in low-light or no-light environments, the laser SLAM module is usually more reliable for positioning and mapping than the visual SLAM module.

[0120] Therefore, by switching back to the laser SLAM module when the preset conditions are appropriate, the high-precision characteristics of the laser SLAM module can be utilized to improve the accuracy and efficiency of positioning and mapping. This dynamic switching mechanism can ensure that the robot uses the appropriate sensor module under different environmental conditions, thereby optimizing the use of computing resources and power. Moreover, by flexibly switching between the visual SLAM module and the laser SLAM module, the robot can better adapt to environmental changes, enhancing the overall robustness and reliability.

[0121] Optionally, the laser sensor data includes point cloud data, and determining whether to re-enable the laser SLAM module for positioning based on the acquired next-frame laser sensor data includes:

[0122] Determining the first pose information of the robot in the global grid map based on the next-frame point cloud data;

[0123] Determining the first reliability of the orientation of the robot in the global grid map based on the first feature in the first pose information;

[0124] Determining whether to re-enable the laser SLAM module for positioning based on the first reliability.

[0125] In the embodiments of the present disclosure, the point cloud data generated by the laser sensor can contain rich environmental information and can be used to describe the three-dimensional structure of surrounding objects. Therefore, the laser sensor data can be point cloud data. In this way, after the visual SLAM module finishes positioning, the next-frame point cloud data can continue to be acquired.

[0126] In this step, the next-frame point cloud data is used to calculate the first pose information of the robot. The first pose information includes the position and orientation of the robot and can be determined by analyzing features in the point cloud data such as the plane where it is located, corner points, etc. The embodiments of the present disclosure do not specifically limit the method for determining the first pose information;

[0127] Further, based on the features in the first pose information, the first reliability of the orientation of the robot in the global grid map is calculated. The evaluation of the first reliability may involve multiple factors, including the density of the point cloud data, the matching degree of feature matching, the richness of features, etc. It can be understood that the higher the density of the point cloud data, the higher the matching degree of feature matching, and the higher the richness of features, the higher the first reliability;

[0128] Further, according to the first reliability, it is judged whether to re-enable the laser SLAM module. If the first reliability is high, it indicates that the laser sensor data is stable and accurate enough, and then the laser SLAM module can be switched back for positioning and map updating.

[0129] Exemplarily, the richness of features means that the structural information represented by the features can sufficiently reflect the specific position and orientation of the robot. If the features of the point cloud data scanned by the robot only represent an unbounded wall (similar to a straight line), the richness of the features is insufficient and it is difficult to reflect the specific position and orientation of the robot. If the features of the point cloud data scanned by the robot represent the corner of a wall, the specific position and orientation of the robot can be reflected, indicating that the features are relatively rich.

[0130] It should be noted that the embodiments of the present disclosure do not specifically limit the method for evaluating the first reliability, and the above is only an example for illustration.

[0131] Therefore, by evaluating the first reliability of the point cloud data, when the first reliability is high, the laser SLAM module can be restarted, so that the pose information of the robot can be more accurately determined by using high-quality point cloud data, thereby improving the positioning accuracy. Especially when the environmental conditions improve, restarting the laser SLAM module can make full use of its advantages in specific scenarios and ensure the adaptability of the robot in a changing environment.

[0132] Optionally, the method further includes:

[0133] In response to the first reliability being greater than the first threshold, restart the positioning function in the laser SLAM module for positioning and turn off the positioning function in the visual SLAM module;

[0134] In response to the first reliability not being greater than the first threshold, continue to use the positioning function in the visual SLAM module to perform positioning using the visual sensor data of the next frame, and turn off the positioning function in the laser SLAM module.

[0135] In this step, when the first reliability is greater than the first threshold, it indicates that the features of the laser sensor data are sufficiently reliable and rich. Therefore, when the features of the point cloud data are relatively rich, an attempt can be made to restore the positioning function of the laser SLAM. If the restoration is successful, the positioning function in the laser SLAM module will be used for positioning to utilize the high-precision characteristics of the laser sensor. At the same time, the positioning function in the visual SLAM module will be turned off to reduce the overall computing power requirement and save computing resources. If the restoration is not successful, such as a failure of the laser SLAM module, positioning will be performed by using the positioning function in the visual SLAM module.

[0136] When the first reliability is not greater than the first threshold, it indicates that the features of the laser sensor data are not reliable or rich enough. Therefore, the robot will continue to use the positioning function in the visual SLAM module to perform positioning using the visual sensor data of the next frame. At the same time, the positioning function in the laser SLAM module will be turned off to avoid positioning using unreliable data.

[0137] It should be noted that the size of the first threshold in the embodiments of the present disclosure is not specifically limited. The setting of the first threshold can ensure that the features of the laser sensor data are reliable and rich enough. It can be set based on the requirements of the actual application scenario or determined in advance based on empirical values.

[0138] In this way, by selecting and using an appropriate positioning function based on the first reliability corresponding to the laser sensor data for positioning, it is not only possible to effectively manage computing resources, avoid unnecessary computing burdens, and by turning off unnecessary positioning functions, while reducing the overall computing power requirements, it can also save power consumption, thereby extending the battery life of the robot. Moreover, in the present disclosure, when the first reliability is determined to be relatively high, the positioning function of the laser SLAM module is used for high-precision positioning to ensure that the robot can achieve an ideal positioning accuracy. This mechanism of flexibly selecting the positioning function according to the first reliability can ensure that the robot provides reliable services in a changing environment.

[0139] Optionally, it is determined that the laser sensor data of the current frame does not meet the preset conditions in the following manner:

[0140] Based on the acquired laser sensor data of the previous frame and the laser sensor data of the current frame, determine the target pose;

[0141] Determine the second reliability of the laser sensor data of the current frame;

[0142] Based on the second reliability and the target pose, determine the third reliability of the target pose;

[0143] In response to the third reliability being less than the second threshold, determine that the laser sensor data of the current frame does not meet the preset conditions.

[0144] In the embodiments of the present disclosure, the target pose is the expected position and orientation of the robot in the environment, which can be determined by matching and comparing the features in two frames of point cloud data. For example, by matching the laser sensor data of the current frame with the laser sensor data of the previous frame, a prediction of the pose information of the current frame is obtained. Then, based on the predicted pose information, a small search box is set, and for each pose in the search box, the laser sensor data of the current frame is projected onto the global grid map to determine the matching degree of the pose with the current global grid map, and scored based on a scoring mechanism, and the pose with the highest score is selected as the target pose of the current frame;

[0145] Among them, the search box is a set of all possible poses, including the offsets of position and orientation.

[0146] Exemplarily, after the laser SLAM module calculates the target pose based on the laser sensor data of the previous frame and the current frame, and determines the second reliability of the laser sensor data of the current frame, the third reliability of the target pose can be calculated based on the second reliability and the target pose. If the third reliability is less than a preset second threshold, it is determined that the laser sensor data of the current frame does not meet the preset conditions, which means that the laser sensor data of the current frame may not be reliable enough for precise positioning and map updating. This third reliability reflects the credibility of the target pose and takes into account the quality of the laser sensor data and the accuracy of pose calculation.

[0147] Among them, the evaluation method of the second reliability can be similar to the evaluation method of the first reliability, or a new evaluation method can be redefined. The embodiments of the present disclosure do not make specific limitations on this.

[0148] Optionally, the evaluation method of the third reliability can be evaluated based on the scoring of the second reliability and the target pose. For example, a weighted algorithm can be used to evaluate the reliability of the target pose by weighted summing the scoring of the second reliability and the target pose.

[0149] Optionally, the third reliability of the target pose can also be characterized only by the magnitude of the second reliability, or the third reliability can be characterized only by the scoring of the target pose. The embodiments of the present disclosure do not make specific limitations on the evaluation method of the third reliability.

[0150] It should be noted that the setting of the second threshold is similar to that of the first threshold. The embodiments of the present disclosure do not make specific limitations on the magnitude of the second threshold, and it can be set based on the requirements of the actual application scenario.

[0151] Optionally, after the laser SLAM module determines the third reliability of the target pose, it can output the third reliability to the visual SLAM module so that the visual SLAM module performs corresponding operations based on the third reliability. When the third reliability is greater than or equal to the second threshold, the visual SLAM module is in the initialization state, but the positioning and mapping functions are not enabled, that is, the time-consuming image matching function and pose solving function are in the off state. When the third reliability is less than the second threshold, the visual SLAM module in the initialization state is started to construct a global grid map. In this way, it can be ensured that the visual SLAM module takes over the laser SLAM module for positioning at any time when the third reliability is lower than the second threshold.

[0152] In this way, through the multi-level reliability evaluation of the laser sensor data, the quality of the data can be judged more accurately, ensuring that high-quality sensor data is used for positioning and map construction. When the sensor data is unreliable, the robot can identify it in time and take measures such as switching to the visual SLAM module for positioning, which can reduce the positioning error and task interruption, improve the overall operation efficiency and stability, thereby improving the robustness of the robot under different environmental conditions. Moreover, by avoiding using unreliable sensor data, computing resources and power can be saved, and unnecessary processing and error correction can be avoided.

[0153] Optionally, based on the laser sensor data of the previous frame and the current frame obtained, determining the target pose includes:

[0154] Based on the laser sensor data of the previous frame and the current frame, determining at least one predicted pose;

[0155] For each predicted pose among the at least one predicted pose, projecting the laser sensor data of the current frame onto the global grid map to determine the matching degree result of each predicted pose in the global grid map;

[0156] Determining the target pose based on the matching degree result.

[0157] Optionally, scoring the matching degree result based on a preset scoring mechanism to obtain a scoring result, and determining the predicted pose with the highest score in the scoring result as the target pose. The preset scoring mechanism can be user-defined or a pre-configured scoring mechanism. The embodiments of the present disclosure do not specifically limit the content of the preset scoring mechanism.

[0158] Exemplarily, after receiving the laser sensor data, the laser SLAM module can calculate at least one predicted pose by using the laser sensor data of the previous frame and the current frame. Further, for each predicted pose, project the point cloud data of the current frame onto the global grid map, that is, linearly transform the coordinates of each point cloud data by using the transformation matrix of the predicted pose to obtain the point cloud data obtained after the transformation of the predicted pose, and compare the transformed point cloud data with the existing global grid map to calculate the matching degree result of each predicted pose. Then, based on the matching degree results of all predicted poses, select the pose with the highest matching degree as the target pose, and this target pose is the output pose of the laser SLAM module.

[0159] Among them, the predicted pose is an estimation of the current position and attitude of the robot, which can be generated by analyzing the feature changes in laser sensor data such as point cloud data. The matching degree result can reflect the similarity or consistency between the predicted pose and the global grid map, and can be determined by methods such as calculating the overlapping area and minimizing errors. The embodiments of the present disclosure do not specifically limit the methods for determining the predicted pose and calculating the matching degree result.

[0160] In this way, by evaluating and comparing multiple predicted poses, a predicted pose that is more in line with the actual situation can be selected, thereby improving the accuracy and reliability of positioning. And by utilizing the matching of laser sensor data and the global grid map, the target pose can be intelligently selected, which can reduce positioning errors and task interruptions, and improve operation efficiency and stability.

[0161] Optionally, determining the second reliability of the laser sensor data of the current frame includes:

[0162] Determining the second pose information of the robot in the global grid map based on the point cloud data of the current frame;

[0163] Extracting the second feature in the second pose information and determining the number of the second features;

[0164] Determining the second reliability of the laser sensor data of the current frame based on the second feature and the number of the second features.

[0165] In the embodiments of the present disclosure, the second pose information is also used to indicate the position and direction of the robot. The determination process of the second pose information is similar to that of the first pose information, except that the data frames of the corresponding point cloud data are different. Correspondingly, the definitions of the second feature and the first feature are also similar. For details, reference can be made to the description of the above embodiments and will not be elaborated here.

[0166] Exemplarily, based on the point cloud data of the current frame, the second pose information of the robot in the global grid map is calculated, the second feature is extracted from the second pose information, and the number of the second features is determined. This feature number is an index to measure the data richness. Therefore, based on the extracted second feature and its number, the second reliability of the laser sensor data of the current frame can be evaluated. This second reliability evaluation method is similar to the first reliability evaluation method in the above embodiments and will not be elaborated here.

[0167] In this way, by analyzing the second feature and its number in the second pose information, the second reliability of the laser sensor data of the current frame is determined, which can more accurately evaluate the richness and quality of the laser sensor data, ensure that the robot uses high-quality and feature-rich data for positioning and map construction, and through the second reliability evaluation, ensure that when the point cloud data is unreliable, it can be recognized in time and measures can be taken, such as switching to the visual SLAM module for positioning, to improve the robustness of the robot under different environmental conditions.

[0168] Optionally, the method further includes:

[0169] In response to the end of the positioning of the visual SLAM module, sending the positioning result of the visual SLAM module to the laser SLAM module;

[0170] After receiving the positioning result, the laser SLAM module turns off the positioning function in the laser SLAM module.

[0171] In this step, after the visual SLAM module completes the positioning task of the current frame, the position information and pose of the current robot are generated, and these position information and pose are the positioning results of the visual SLAM module. Further, the positioning results of the visual SLAM module are sent to the laser SLAM module. After the laser SLAM module receives the positioning results of the visual SLAM module, the laser SLAM module turns off its positioning function.

[0172] Wherein, a communication connection is established between the visual SLAM module and the laser SLAM module to transfer the positioning results.

[0173] In the present disclosure, by turning off the positioning function of the laser SLAM module after the visual SLAM module finishes positioning the current frame, computing resources and power consumption can be saved. This resource optimization helps to extend the battery life of the robot, and it can also avoid repeated positioning calculations of the laser SLAM module, reduce the dependence on the laser SLAM module, and improve the adaptability of the robot under different environmental conditions. Therefore, by coordinating the operations of different SLAM modules, data redundancy and processing conflicts can be reduced, the consistency and accuracy of the positioning information can be ensured, and further, the switching and resource allocation of the SLAM modules can be intelligently managed, reducing interruptions and errors.

[0174] Optionally, the visual sensor data includes image data, and the method further includes:

[0175] When using the laser SLAM module for positioning, the image matching function and the pose solution function are in the off state.

[0176] In the implementation of the present disclosure, when the robot decides to use the laser SLAM module for positioning, the positioning function of the laser SLAM module is responsible for processing the laser sensor data for precise positioning and map construction. At this time, in addition to turning off the positioning function in the visual SLAM module, the image matching function and the pose solution function in the visual SLAM module can also be turned off, which means that the visual SLAM module neither performs positioning nor processes image data, thus saving computing resources.

[0177] Therefore, when using the laser SLAM module for positioning, at least one function in the visual SLAM module is turned off to reduce the overall computing power requirement. The at least one function may include an image matching function, a pose solving function, a second positioning function, and other functions that require computing resources.

[0178] In this way, by turning off the relevant functions in the visual SLAM module when the laser SLAM is used for positioning, not only can computing resources and power be saved, and the battery life of the robot be extended, but also the computing burden on the robot can be reduced, and the complexity and potential conflicts caused by simultaneously processing multiple sensor data can be avoided.

[0179] Optionally, the construction of the global grid map includes:

[0180] In response to the laser sensor data of the current frame meeting a preset condition, the positioning result of the laser SLAM module is used to construct the global grid map;

[0181] In response to the laser sensor data of the current frame not meeting the preset condition, the positioning result of the visual SLAM module is used to construct the global grid map.

[0182] Optionally, the robot may further include a mapping module, or the laser SLAM module includes a mapping function, and the visual SLAM module includes a mapping function. The functions of the mapping module and the mapping function are similar, both for constructing the global grid map. The present disclosure embodiment does not limit the deployment position corresponding to the mapping function. Generally, both the laser SLAM module and the visual SLAM module have a mapping function.

[0183] Exemplarily, when the reliability of the third determined by the current laser SLAM module is relatively high, the mapping function in the laser SLAM module will use the positioning result of the laser SLAM module, such as the predicted pose with the highest score, to update the global grid map. Conversely, the mapping function in the visual SLAM module will use the positioning result of the visual SLAM module, such as the pose information calculated by the pose solving function, to update the global grid map.

[0184] In this way, by selecting the positioning result of the most reliable SLAM module to construct the global grid map, the accuracy and reliability of the global grid map can be improved, and by intelligently selecting the map construction strategy, the computing resources and power consumption can be optimized, the map construction error and task interruption can be reduced, and the operation efficiency and stability can be improved.

[0185] In the foregoing embodiments, the method for a robot to build a map provided by the present disclosure has been introduced. To implement the various functions in the method provided by the present disclosure, the robot as the execution subject may include a hardware structure and / or software modules, and implement the above-mentioned various functions in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module. Whether a certain function among the above-mentioned various functions is executed in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module depends on the specific application and design constraints of the technical solution.

[0186] For example, Figure 5 FIG. 5 is a schematic structural diagram of a robot mapping device provided by an embodiment of the present disclosure. The robot includes a laser simultaneous localization and mapping (SLAM) module and a visual SLAM module. The laser SLAM module is used to construct a global grid map based on laser sensor data and determine the position of the robot in the global grid map. The visual SLAM module is used to construct a global grid map based on visual sensor data and determine the position of the robot in the global grid map. As Figure 5 shown, the robot mapping device 500 includes:

[0187] A start module 501, configured to, during the process of the robot constructing a global grid map based on the laser SLAM module, in response to the laser sensor data of the current frame not meeting a preset condition, start the visual SLAM module in an initialization state, so that the visual SLAM module uses the visual sensor data of the current frame for positioning to implement the construction of the global grid map. In the initialization state, the visual SLAM module starts to work in response to receiving a start command.

[0188] Optionally, the robot mapping device 500 further includes an acquisition module, and the acquisition module is configured to:

[0189] In response to the end of the positioning of the visual SLAM module, the laser SLAM module determines whether to re-enable the laser SLAM module for positioning based on the acquired laser sensor data of the next frame, for constructing a global grid map.

[0190] Optionally, the laser sensor data includes point cloud data, and the acquisition module is specifically configured to:

[0191] Determine the first pose information of the robot in the global grid map based on the point cloud data of the next frame;

[0192] Determine the first reliability of the orientation of the robot in the global grid map based on the first feature in the first pose information;

[0193] Determine whether to re-enable the laser SLAM module for positioning based on the first reliability.

[0194] Optionally, the robot mapping device 500 further includes an enabling and disabling module, which is configured to:

[0195] In response to the first reliability being greater than the first threshold, re-enable the positioning function in the laser SLAM module for positioning and disable the positioning function in the visual SLAM module;

[0196] In response to the first reliability not being greater than the first threshold, continue to use the positioning function in the visual SLAM module to perform positioning using the visual sensor data of the next frame, and disable the positioning function in the laser SLAM module.

[0197] Optionally, the startup module 501 includes a determination unit, which is configured to:

[0198] Based on the laser sensor data of the previous frame and the laser sensor data of the current frame, determine the target pose;

[0199] Determine the second reliability of the laser sensor data of the current frame;

[0200] Based on the second reliability and the target pose, determine the third reliability of the target pose;

[0201] In response to the third reliability being less than the second threshold, determine that the laser sensor data of the current frame does not meet the preset conditions.

[0202] Optionally, the determination unit includes a first determination subunit, which is configured to:

[0203] Based on the laser sensor data of the previous frame and the laser sensor data of the current frame, determine at least one predicted pose;

[0204] For each predicted pose in the at least one predicted pose, project the laser sensor data of the current frame into the global grid map to determine the matching degree result of each predicted pose in the global grid map;

[0205] Based on the matching degree result, determine the target pose.

[0206] Optionally, the determination unit further includes a second determination subunit, which is configured to:

[0207] Based on the point cloud data of the current frame, determine the second pose information of the robot in the global grid map;

[0208] Extract the second feature in the second pose information and determine the number of second features;

[0209] Based on the second feature and the number of second features, determine the second reliability of the laser sensor data of the current frame.

[0210] Optionally, the robot mapping device 500 further includes a sending module, which is configured to:

[0211] In response to the completion of the positioning by the visual SLAM module, send the positioning result of the visual SLAM module to the laser SLAM module;

[0212] After receiving the positioning result, the laser SLAM module turns off the positioning function in the laser SLAM module.

[0213] Optionally, the visual sensor data includes image data. The visual SLAM module includes an image matching function and a pose solving function. The image matching function is used to perform feature matching on the image data of two adjacent frames, and the pose solving function is used to determine the pose information based on the result of the feature matching to ensure that the visual SLAM module performs positioning based on the pose information. The robot mapping device 500 further includes a positioning module, which is configured to:

[0214] When using the laser SLAM module for positioning, the image matching function and the pose solving function are in the off state.

[0215] Optionally, the robot mapping device 500 further includes a mapping module, which is configured to:

[0216] In response to the current frame of laser sensor data meeting the preset conditions, use the positioning result of the laser SLAM module to construct a global grid map;

[0217] In response to the current frame of laser sensor data not meeting the preset conditions, use the positioning result of the visual SLAM module to construct a global grid map.

[0218] Optionally, the laser SLAM module is deployed in the laser sensor, and the visual SLAM module is deployed in the visual sensor. The laser sensor is used to transmit the laser sensor data to the laser SLAM module, and the visual sensor is used to transmit the visual sensor data to the visual SLAM module. The laser sensor and the visual sensor work alternately during the mapping process.

[0219] It should be noted that the specific implementation principles and effects of the above robot mapping device can refer to the relevant descriptions and effects corresponding to the above embodiments, and will not be elaborated here.

[0220] The embodiment of the present disclosure also provides a schematic structural diagram of a robot. Figure 6 As shown in the schematic structural diagram of a robot provided by the embodiment of the present disclosure, Figure 6 As shown, the robot 100 may include: a laser SLAM module 101, a visual SLAM module 102, a processor 103, and a memory 104 communicatively connected to the processor 103;

[0221] The laser SLAM module 101 is used to construct a global grid map based on laser sensor data and determine the position of the robot 100 in the global grid map. The visual SLAM module 102 is used to construct a global grid map based on visual sensor data and determine the position of the robot 100 in the global grid map.

[0222] The memory 104 stores computer-executable instructions. The processor 103 executes the computer-executable instructions stored in the memory 104, so that the processor 103 executes the method described in any of the foregoing embodiments.

[0223] Among them, the memory 104 and the processor 103 can be connected through a bus.

[0224] The embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in any of the foregoing embodiments of the present disclosure.

[0225] The embodiments of the present disclosure also provide a chip for running instructions. The chip is used to execute the method described in any of the foregoing embodiments executed by the robot in any of the foregoing embodiments.

[0226] The embodiments of the present disclosure also provide a computer program product. The program product includes a computer program, and when the computer program is executed by a processor, it can implement the method described in any of the foregoing embodiments executed by the robot in any of the foregoing embodiments.

[0227] In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or modules can be in electrical, mechanical or other forms.

[0228] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to implement the solution of this embodiment.

[0229] In addition, in each embodiment of the present disclosure, each functional module may be integrated in a processing unit, may exist physically alone for each module, or two or more modules may be integrated in one unit. The unit formed by the above modules may be implemented in the form of hardware, or in the form of a hardware plus software functional unit.

[0230] The integrated module implemented in the form of a software functional module as described above may be stored in a computer-readable storage medium. The above software functional module is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods described in each embodiment of the present disclosure.

[0231] It should be understood that the above processor may be a central processing unit (CPU for short), and may also be other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the application may be directly embodied as being executed and completed by a hardware processor, or may be executed and completed by a combination of hardware and software modules in the processor.

[0232] The memory may include high-speed random access memory (RAM for short), and may also include non-volatile memory (NVM for short), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disc, etc.

[0233] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the buses in the drawings of the present disclosure are not limited to only one bus or one type of bus.

[0234] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, a magnetic disk or an optical disc. The storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0235] An exemplary storage medium is coupled to a processor, enabling the processor to read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC for short). Of course, the processor and the storage medium can also exist as discrete components in a cleaning device or a main control device.

[0236] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present disclosure is not limited by the described action sequence, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present disclosure.

[0237] Furthermore, it should be noted that although the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart can include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0238] In the above embodiments, the descriptions of the various embodiments each have their own emphasis. For parts not elaborated in a certain embodiment, reference may be made to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as falling within the scope described in this specification.

[0239] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0240] As mentioned above, the above is only the specific implementation manner of the embodiments of the present disclosure, but the protection scope of the embodiments of the present disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of the present disclosure should be covered within the protection scope of the embodiments of the present disclosure. Therefore, the protection scope of the embodiments of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A robot mapping method, characterized in that: The robot comprises a laser simultaneous positioning and mapping SLAM module and a visual SLAM module, wherein the laser SLAM module is used to construct a global grid map based on laser sensor data and determine the position of the robot in the global grid map, and the visual SLAM module is used to construct a global grid map based on visual sensor data and determine the position of the robot in the global grid map. The method comprises: In the process of the robot constructing a global grid map based on the laser SLAM module, in response to the laser sensor data of the current frame not satisfying the preset conditions, the visual SLAM module in the initialization state is started so that the visual SLAM module uses the visual sensor data of the current frame for positioning to realize the construction of the global grid map. In the initialization state, the visual SLAM module starts working in response to receiving a start command.

2. The method according to claim 1, characterized in that The method further comprises: In response to the completion of positioning by the visual SLAM module, the laser SLAM module determines whether to re-enable the laser SLAM module for positioning based on the acquired laser sensor data of the next frame, so as to construct the global grid map.

3. The method according to claim 2, characterized in that The laser sensor data includes point cloud data, and determining whether to re-enable the laser SLAM module for positioning based on the acquired laser sensor data of the next frame includes: Determining first posture information of the robot in the global grid map based on the point cloud data of the next frame; Determining a first reliability of the robot's position in the global grid map based on a first feature in the first posture information; Determine whether to re-enable the laser SLAM module for positioning based on the first reliability.

4. The method according to claim 3, characterized in that The method further comprises: In response to the first reliability being greater than a first threshold, re-enabling the positioning function in the laser SLAM module for positioning, and disabling the positioning function in the visual SLAM module; In response to the first reliability being not greater than the first threshold, continuing to use the positioning function in the visual SLAM module to perform positioning using the visual sensor data of the next frame, and turning off the positioning function in the laser SLAM module.

5. The method according to claim 1, characterized in that The laser sensor data of the current frame does not meet the preset condition by: Determine the target pose based on the acquired laser sensor data of the previous frame and the laser sensor data of the current frame; Determining a second reliability of the laser sensor data of the current frame; Determining a third reliability of the target posture based on the second reliability and the target posture; In response to the third reliability being less than the second threshold, it is determined that the laser sensor data of the current frame does not meet a preset condition.

6. The method according to claim 5, characterized in that Determining the target pose based on the acquired laser sensor data of the previous frame and the laser sensor data of the current frame includes: Determine at least one predicted pose based on the laser sensor data of the previous frame and the laser sensor data of the current frame; For each predicted pose of the at least one predicted pose, projecting the laser sensor data of the current frame into the global grid map to determine a matching result of each predicted pose in the global grid map; The target pose is determined based on the matching result.

7. The method according to claim 5, characterized in that Determining a second reliability of the laser sensor data of the current frame includes: Determining second posture information of the robot in the global grid map based on the point cloud data of the current frame; extracting second features from the second posture information, and determining the number of the second features; A second reliability of the laser sensor data of the current frame is determined based on the second feature and the number of the second features.

8. The method according to claim 1, characterized in that The method further comprises: In response to the completion of positioning by the visual SLAM module, sending the positioning result of the visual SLAM module to the laser SLAM module; After receiving the positioning result, the laser SLAM module turns off the positioning function in the laser SLAM module.

9. The method according to claim 1, characterized in that: The visual sensor data includes image data, the visual SLAM module includes an image matching function and a posture solving function, the image matching function is used to perform feature matching on image data of two adjacent frames, the posture solving function is used to determine posture information according to the result of feature matching, so as to ensure that the visual SLAM module performs positioning based on the posture information, and the method further includes: In response to positioning using the laser SLAM module, the image matching function and the posture solving function are in a closed state.

10. The method according to claim 1, characterized in that The construction of the global grid map includes: In response to the laser sensor data of the current frame satisfying a preset condition, constructing the global grid map using the positioning result of the laser SLAM module; In response to the laser sensor data of the current frame not satisfying a preset condition, the global grid map is constructed using the positioning result of the visual SLAM module.

11. The method according to claim 1, characterized in that: The laser SLAM module is deployed in the laser sensor, and the visual SLAM module is deployed in the visual sensor. The laser sensor is used to transmit the laser sensor data to the laser SLAM module, and the visual sensor is used to transmit the visual sensor data to the visual SLAM module. The laser sensor and the visual sensor work alternately during the mapping process.

12. A robot mapping device, characterized in that: The robot comprises a laser simultaneous positioning and mapping SLAM module and a visual SLAM module, wherein the laser SLAM module is used to construct a global grid map based on laser sensor data and determine the position of the robot in the global grid map, and the visual SLAM module is used to construct a global grid map based on visual sensor data and determine the position of the robot in the global grid map, and the device comprises: A starting module is used to start the visual SLAM module in an initialization state in response to the laser sensor data of the current frame not satisfying a preset condition during the process of the robot constructing a global grid map based on the laser SLAM module, so that the visual SLAM module uses the visual sensor data of the current frame for positioning to realize the construction of the global grid map. In the initialization state, the visual SLAM module starts working in response to receiving a start command.

13. A robot, characterized in that: include: A laser SLAM module, a visual SLAM module, a processor, and a memory communicatively connected to the processor; The laser SLAM module is used to construct a global grid map based on laser sensor data and determine the position of the robot in the global grid map, and the visual SLAM module is used to construct a global grid map based on visual sensor data and determine the position of the robot in the global grid map; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images