Robot mapping method and device, sweeping robot and storage medium

Abstract

The application discloses a kind of robot mapping method and device, floor cleaning robot, storage medium, wherein, robot mapping method includes: obtaining the pose information of robot;Current frame point cloud information is obtained by TOF sensor;Determine that current frame is key frame, then current frame point cloud information and pose information are updated to preset map.The application obtains current frame point cloud information using the TOF sensor arranged on the robot body, then obtains the current pose information of robot, and when determining that current frame is key frame, current frame point cloud information and the pose information of robot are updated to preset map.Compared with the camera-based visual SLAM scheme and the laser SLAM scheme based on single line laser radar in the related art, the TOF-based scheme in the embodiment of the application can construct the map of the environment where the robot is located at low cost, and can also improve the environmental adaptability of robot mapping, thereby improving the mapping success rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a robot mapping method, a sweeping robot, a computer readable storage medium and a robot mapping device. BACKGROUND

[0002] In the application of the sweeping robot, the SLAM (Simultaneous Localization and Mapping) technology and the 3DSLAM technology are very widely used.

[0003] In the related art, the SLAM of the sweeping robot has two kinds, which are a camera-based visual SLAM scheme and a single-line laser radar-based laser SLAM scheme. However, the camera-based visual SLAM scheme cannot work normally in insufficient light or low-texture scenes because it uses a camera to collect data. Although the laser used in the single-line laser radar-based laser SLAM scheme is not affected by light and environmental texture, it often leads to a higher body of the sweeping robot and greatly increases the cost of the sweeping robot. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art. To this end, one object of the present application is to provide a robot mapping method, which can construct a map of an environment in which a robot is located at a low cost and can also improve the environmental adaptability of the robot when mapping, thereby improving the mapping success rate.

[0005] A second object of the present application is to provide a sweeping robot.

[0006] A third object of the present application is to provide a computer readable storage medium.

[0007] A fourth object of the present application is to provide a robot mapping device.

[0008] To achieve the above objects, an embodiment of the first aspect of the present application provides a robot mapping method, wherein the robot is provided with a TOF (Time off Light) sensor, and the method comprises: acquiring pose information of the robot; acquiring point cloud information of a current frame through the TOF sensor; determining that the current frame is a key frame, and then updating the point cloud information of the current frame and the pose information to a preset map.

[0009] The robot mapping method of the embodiment of the present application uses a TOF sensor arranged on the robot body to obtain point cloud information of a current frame, then obtains current pose information of the robot, and when it is determined that the current frame is a key frame, updates the point cloud information of the current frame and the pose information of the robot into a preset map. Compared with the camera-based visual SLAM scheme and the single-line laser radar-based laser SLAM scheme in the related art, the TOF-based scheme in the embodiment of the present application can construct a map of an environment where the robot is located at a low cost, and can also improve the environmental adaptability of the robot when mapping, thereby improving the mapping success rate.

[0010] In some embodiments of the present application, the updating of the point cloud information of the current frame and the pose information into the preset map comprises: constructing an odometry factor; performing factor graph optimization on the pose information of the robot according to the odometry factor; and updating the point cloud information of the current frame and the pose information of the robot after the factor graph optimization into the preset map.

[0011] In some embodiments of the present application, before the determination that the current frame is a key frame, the method further comprises: obtaining plane point feature information of the current frame according to the point cloud information of the current frame, and constructing a loss function according to the plane point feature information; determining a forward direction and horizontal position information of the robot according to the pose information of the robot; and performing optimization on the pose information of the robot according to the forward direction and horizontal position information of the robot, the preset map, and the loss function, and taking the optimized pose information as the pose information of the robot.

[0012] In some embodiments of the present application, the obtaining of the plane point feature information of the current frame according to the point cloud information of the current frame comprises: calculating the curvature of each point in the point cloud information of the current frame; and determining the plane point feature information of the current frame according to the curvature of each point.

[0013] In some embodiments of the present application, after the obtaining of the plane point feature information of the current frame according to the point cloud information of the current frame, the method further comprises: performing loop detection according to the plane point feature information of the current frame and the preset map; determining that the current frame is a loop frame, then obtaining a loop factor of the loop frame; determining that the loop frame is a key frame, then performing factor graph optimization on the pose information of the robot based on the loop factor; and determining that the loop frame is a non-key frame, then accumulating the loop factor for the next factor graph optimization.

[0014] In some embodiments of the present application, the determining that the current frame is a loop-back frame comprises: obtaining a candidate key frame according to the pose information of the robot and historical key frames; performing registration on the candidate key frame and the current frame, and obtaining a registration value of the current frame; and if the registration value is less than a registration threshold, determining that the current frame is a loop-back frame.

[0015] In some embodiments of the present application, the determining that the current frame is a key frame comprises: if the moving distance of the robot is greater than a preset distance threshold and / or the rotation angle of the robot is greater than a preset angle threshold, determining that the current frame is a key frame.

[0016] In some embodiments of the present application, the obtaining the pose information of the robot comprises: obtaining first pose information of the robot through an inertial sensor, and obtaining second pose information of the robot through a wheel speed odometer; and performing fusion processing on the first pose information and the second pose information to obtain the pose information of the robot.

[0017] To achieve the above object, the second aspect of the present application provides a robot, comprising a memory, a processor and a robot mapping program stored in the memory and executable on the processor, wherein the processor executes the robot mapping program to implement the robot mapping method according to the above embodiments.

[0018] The robot of the embodiments of the present application comprises a memory and a processor, and the processor executes the robot mapping program stored in the memory, so that the robot can construct a map of the environment in which the robot is located at low cost, and the environmental adaptability of the robot during mapping can be improved, thereby improving the mapping success rate.

[0019] To achieve the above object, the third aspect of the present application provides a computer readable storage medium, which stores a robot mapping program, and the robot mapping program is executable on a processor to implement the robot mapping method according to the above embodiments.

[0020] The computer readable storage medium of the embodiments of the present application can construct a map of the environment in which the robot is located at low cost by executing the robot mapping program stored therein on the processor, and the environmental adaptability of the robot during mapping can be improved, thereby improving the mapping success rate.

[0021] To achieve the above object, the fourth aspect of the present application provides a robot mapping device, the robot is provided with a TOF sensor, the device comprises: a first acquisition module, configured to acquire pose information of the robot; a second acquisition module, configured to acquire point cloud information of a current frame by the TOF sensor; and an updating module, configured to determine that the current frame is a key frame, and then update the point cloud information of the current frame and the pose information into a preset map.

[0022] The robot mapping device provided by the embodiment of the present application comprises a first acquisition module, a second acquisition module and an updating module, wherein the point cloud information of a current frame acquired by a TOF sensor arranged on a robot body is acquired by the second acquisition module, the current pose information of the robot is acquired by the first acquisition module, and when it is determined that the current frame is a key frame, the point cloud information of the current frame and the pose information of the robot are updated into a preset map by the updating module. Compared with the camera-based visual SLAM scheme and the single-line laser radar-based laser SLAM scheme in the related art, the TOF-based scheme in the embodiment of the present application can construct a map of an environment in which the robot is located at a low cost, and can also improve the environmental adaptability of the robot when mapping, thereby improving the mapping success rate.

[0023] Additional aspects and advantages of the present application will be made apparent by the following description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a flowchart of a robot mapping method according to an embodiment of the present application;

[0025] Figure 2 is a flowchart of a robot mapping method according to an embodiment of the present application;

[0026] Figure 3 is a flowchart of a robot mapping method according to an embodiment of the present application;

[0027] Figure 4 is a flowchart of a robot mapping method according to an embodiment of the present application;

[0028] Figure 5 is a flowchart of a robot mapping method according to an embodiment of the present application;

[0029] Figure 6 is a flowchart of a robot mapping method according to an embodiment of the present application;

[0030] Figure 7 is a flowchart of a robot mapping method according to an embodiment of the present application;

[0031] Figure 8 is a flowchart of a robot mapping method according to an embodiment of the present application;

[0032] Figure 9 is a structural block diagram of a robot according to an embodiment of the present application;

[0033] Figure 10 is a structural block diagram of a robot mapping device according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which the same or similar components have the same or similar designations and functions throughout various figures and / or embodiments, and a description of the same or similar components is not repeated. Embodiments described below are examples in which the present application is applied to a robot mapping method and device, a robot, and a storage medium.

[0035] A robot mapping method and device, a robot, and a storage medium according to embodiments of the present application are described below with reference to the accompanying drawings.

[0036] Figure 1 is a flowchart of a robot mapping method according to an embodiment of the present application.

[0037] As shown in Figure 1 , the present application proposes a robot mapping method, which includes the following steps:

[0038] S10, obtaining pose information of the robot.

[0039] Specifically, the present embodiment is to obtain an image in a current environment by moving the robot, and then to establish a map of the environment in which the robot is currently located, so the pose information of the robot is crucial to whether the robot can completely map the environment. In the present embodiment, the pose information of the robot can include six degrees of freedom information of the robot, i.e., movement degrees of freedom along three orthogonal coordinate axes x, y, and z and rotation degrees of freedom around the three coordinate axes. In the present embodiment, the movement information of the robot, such as the forward direction and horizontal position of the robot, can be obtained according to the pose information of the robot. Figure 10

[0040] In the present embodiment, as shown in Figure 2 , the pose information of the robot is obtained, including:

[0041] S201, obtaining first pose information of the robot by an inertial sensor, and obtaining second pose information of the robot by a wheel speed odometer. S202, performing fusion processing on the first pose information and the second pose information to obtain the pose information of the robot.

[0042] ​Specifically, the embodiment can be provided with a fusion odometer connected with the inertial sensor and the wheel speed odometer respectively, and can receive the robot information acquired by the inertial sensor and the robot information acquired by the wheel speed odometer. The first pose information of the robot can be acquired by calculating the data acquired by the inertial sensor, and the first pose information can be the first six-degree-of-freedom information of the robot, i.e., x1, y1, z1, roll1, pitch1, yaw1. The data acquired by the inertial sensor can be specifically integrated. The second pose information of the robot can be acquired by calculating the data acquired by the wheel speed odometer, and the second pose information can be the second six-degree-of-freedom information of the robot, i.e., x2, y2, z2, roll2, pitch2, yaw2. It can be understood that in the six-degree-of-freedom information, the x direction and the y direction can represent the distance and direction of the robot moving forward, the z direction can represent the distance and direction of the robot moving up and down, the roll represents the roll angle of the robot, the pitch represents the pitch angle of the robot, and the yaw represents the yaw angle of the robot.

[0043] In the embodiment, the first pose information and the second pose information are fused by the fusion odometer. The processing method can extract x2 and y2 in the second six-degree-of-freedom information and yaw1 in the first six-degree-of-freedom information as the fusion result to obtain the moving direction and horizontal position information of the robot, wherein yaw1 represents the moving direction of the robot, and x2 and y2 represent the horizontal position information of the robot.

[0044] S20, acquiring point cloud information of a current frame by the TOF sensor.

[0045] First of all, it needs to be pointed out that the TOF sensor is a sensor that is currently equipped in most sweeping robots. It has the advantages of small volume, low cost, and is not affected by insufficient light and low texture environment. However, in most sweeping robots, it is only used for avoidance, and is rarely used for positioning and mapping of the sweeping robot. In addition, due to the problems of small field of view angle, small range, and large data volume of the TOF sensor, it is a great challenge to use it for positioning and mapping. Therefore, in the related art, the TOF sensor is rarely used for positioning and mapping.

[0046] The robot mapping method of the embodiment can overcome the above-mentioned problems of the TOF sensor and use the TOF sensor for positioning and mapping of the robot. Specifically, the TOF sensor is arranged on the body of the robot, and then the point cloud information of the current frame is acquired by the TOF sensor during mapping.

[0047] It should be noted that two TOF sensors can be arranged on the body of the robot in the embodiment, and are arranged on the left side and the front side of the robot respectively, and then the data detected by the two TOF sensors is spliced to obtain the current frame in the embodiment, and then the corresponding point cloud information is extracted from the current frame.

[0048] More specifically, in one specific embodiment, the robot is provided with a first TOF sensor and a second TOF sensor, wherein the first TOF sensor acquires point cloud information of a first current frame, the second TOF sensor acquires point cloud information of a second current frame, and then the point cloud information of the first current frame and the point cloud information of the second current frame are spliced to obtain a current frame.

[0049] In the embodiment, the first TOF sensor can be arranged in front of the robot, and the second TOF sensor can be arranged on the left side of the robot, and more specifically, the first TOF sensor arranged in front of the robot has a field of view angle of 110°×38°, i.e. a horizontal field of view angle of 110° and a vertical field of view angle of 38°, and has a resolution of 224×70; the second TOF sensor arranged on the left side of the robot has a field of view angle of 60°×45°, i.e. a horizontal field of view angle of 60° and a vertical field of view angle of 45°, and has a resolution of 240×180.

[0050] It can be understood that the first TOF sensor and the second TOF sensor are arranged on the robot at a suitable angle, which can make the point cloud information collected by the first TOF sensor and the second TOF sensor have no repeated information, i.e. the rightmost point cloud information in the second current frame collected by the second TOF sensor and the leftmost point cloud information in the first current frame collected by the first TOF sensor are spliced to form a complete and continuous current frame.

[0051] In a specific embodiment, splicing the first current frame and the second current frame can include the following specific steps: first, removing the perspective edge part point cloud in the first current frame and the second current frame. It can be understood that after the perspective edge part is removed, the resolution of the first current frame and the second current frame will change. For example, before the perspective edge part point cloud is removed, the resolution of the first current frame is 224x70, and after the perspective edge part point cloud is removed, the resolution of the first current frame can be 219x65, that is, 5 rows of pixel points of the perspective edge are deleted. After removing the perspective edge part point cloud, the vertical resolution of the first current frame is taken as one half, the vertical resolution of the second current frame is taken as one fourth, and finally, according to the appropriate angle between the first TOF sensor and the second TOF sensor, the point cloud information of the second current frame is converted into the coordinate system of the point cloud information of the first current frame, thereby completing the point cloud splicing between the first current frame and the second current frame to obtain the point cloud information of the current frame.

[0052] S30, determining that the current frame is a key frame, updating the point cloud information and the pose information of the current frame to the preset map.

[0053] Specifically, after obtaining the point cloud information of the current frame of the robot and the pose information of the robot, the current frame is judged. If it is determined that the current frame is a key frame, the obtained point cloud information and pose information of the current frame can be updated to the preset map. It can be understood that the preset map is composed of each key frame spliced, which is actually a local map of the current environment of the robot. After the robot completes mapping, the preset map is a global map of the current environment of the robot.

[0054] In this embodiment, determining that the current frame is a key frame includes: if the moving distance of the robot is greater than a preset distance threshold and / or the rotation angle of the robot is greater than a preset angle threshold, determining that the current frame is a key frame.

[0055] Specifically, when the moving distance of the robot is greater than a preset distance threshold and / or the rotation angle of the robot is greater than a preset angle threshold, it can be determined that the current frame is a key frame. That is, during the movement of the robot, if it is judged that the moving distance of the robot exceeds the preset distance threshold, it can be determined that the current frame is a key frame, or if it is judged that the rotation angle of the robot exceeds the preset angle threshold, it can be determined that the current frame is a key frame, of course, if the moving distance and the rotation angle of the robot meet at the same time, the current frame can also be determined as a key frame. That is, when one or more parameters of the moving distance and the rotation angle of the robot meet the conditions, the current frame can be determined as a key frame. It should be noted that the preset distance threshold and the preset angle threshold in the embodiment can be determined according to the size of the environment scene where the robot is located and the field of view angle of the TOF sensor of the robot.

[0056] After determining that the current frame is a key frame, the point cloud information and the pose information of the current frame can be updated to the preset map.

[0057] Specifically, after determining that the current frame is a key frame, it indicates that the current robot has moved more than a certain distance or rotated more than a certain angle, then in order to the integrity of the map, the current frame point cloud information and the pose information corresponding to the current robot need to be acquired, and then the preset map is updated.

[0058] In this embodiment, as shown in Figure 3 The point cloud information and the pose information of the current frame are updated to the preset map, including the following steps:

[0059] S301, constructing an odometry factor.

[0060] Specifically, when it is determined that the current frame is a key frame, an odometry factor can be constructed. It should be noted that the odometry factor in the embodiment can be obtained by fusing the data obtained by the wheel speed odometer and the inertial sensor through the extended Kalman filtering method, and the specific construction method can be referred to the detailed description in the related art, which will not be repeated here. More specifically, the odometry factor can be used to construct an odometry error term, it can be understood that by adjusting the pose information of the robot, the error term can be reduced, and in turn the current pose information of the robot can be reflected by the error term whether it is in the correct position.

[0061] S302, optimizing the pose information of the robot according to the odometry factor.

[0062] Specifically, the factor graph is an undirected graph composed of two kinds of nodes, namely variable nodes representing optimization variables and factor nodes representing factors, and the optimization of the factor graph is to adjust the values of the variables to maximize the product of their factors. In this embodiment, it is applied to adjust the pose information of the robot to minimize the error term constructed according to the odometry factor. Optionally, the embodiment can use the GTSAM open source library to perform factor graph optimization.

[0063] S303, update the point cloud information of the current frame and the pose information of the robot optimized by the factor graph to the preset map.

[0064] Specifically, after the pose information of the robot is optimized by the factor graph, the preset map can be updated according to the optimized pose information and the point cloud information of the current frame, which specifically includes updating the point cloud information on the current robot key frame to the local map according to the position information of the adjusted robot. It can be understood that the local map is composed of the point cloud information of the key frame, and the point cloud information of all key frames can be combined to form a global map. In each key frame, LM optimization and factor optimization operations need to be performed, and after optimization, the point cloud information of each key frame is updated to the preset map according to the pose information of the robot, so that the current key frame can be accurately updated to the preset map, and the construction of the global map is completed.

[0065] Optionally, after the preset map is updated, it is also sent through a TCP (Transmission Control Protocol, Transmission Control Protocol) server, so that external devices can receive and package and publish the point cloud information after receiving it for display.

[0066] In some embodiments of the present application, as shown in Figure 4 Before determining that the current frame is a key frame, the method further includes the following steps:

[0067] S401, obtain the plane point feature information of the current frame according to the point cloud information of the current frame, and construct a loss function according to the plane point feature information. S402, determine the advancing direction and horizontal position information of the robot according to the pose information of the robot. S403, optimize the pose information of the robot according to the advancing direction and horizontal position information of the robot, the preset map and the loss function, and use the optimized pose information as the pose information of the robot.

[0068] Specifically, after obtaining the robot's posture information and the point cloud information of the current frame, that is, before determining the current frame as a key frame, the robot's posture information and the point cloud information of the current frame are also subjected to LM (Levenberg-Marquardt) optimization. Specifically, the robot's current position can be tracked and predicted based on the robot's forward direction, horizontal position information and preset map, and then the plane point feature information of the current frame is extracted based on the point cloud information of the current frame. Then, a loss function is constructed based on the plane point feature information, and the Gauss-Newton iteration is used to optimize the robot's posture information to adjust the loss function.

[0069] Specifically, the robot's current pose is first tracked and predicted using odometer information and a preset map, and the plane feature information of the current frame is used to construct a point-surface constraint to obtain a loss function. It is understandable that the value of the loss function can be changed by adjusting the robot's pose. The smaller the value of the loss function, the more accurate the current robot's pose. The robot's pose information is optimized using the Gauss-Newton iterative method to adjust the loss function. More specifically, in this embodiment, the current pose is predicted by adding the robot's pose at the previous moment to the pose increment at the current moment. The robot's pose information at the previous moment can be obtained from the local map, while the pose increment at the current moment can be obtained through the odometer.

[0070] In this embodiment, Figure 5 As shown, obtaining the plane point feature information of the current frame according to the point cloud information of the current frame includes the following steps:

[0071] S501, calculating the curvature of each point in the point cloud information of the current frame. S502, determining the plane point feature information of the current frame according to the curvature of each point.

[0072] Specifically, after obtaining the point cloud information of the current frame, the curvature of each point cloud information in the current frame can be further calculated, and then the planar point feature information of the current frame can be extracted based on the curvature of the point cloud information. For example, when calculating the curvature corresponding to the current point cloud information, the point cloud on the same scan line as the point cloud information can be sampled, and the first five point cloud data and the last five point cloud data of the current point cloud data are taken respectively. Then, the sum of the depths of the first five point cloud data and the last five point cloud data is subtracted from ten times the depth of the current point cloud data, and the difference is squared. The square value is then used to represent the curvature of the current point cloud data. The curvature of all point cloud data in the current frame can represent the ups and downs of the current frame. It should be noted that the depth in this example refers to the distance between the TOF sensor and the detection point.

[0073] In some embodiments of the present invention, Figure 6As shown, after obtaining the planar point feature information of the current frame according to the point cloud information of the current frame, the mapping method of the robot further includes the following steps:

[0074] S601, loop detection is performed according to the planar point feature information of the current frame and the preset map.

[0075] Specifically, the embodiment further includes a loop detection step, and it should be noted that the loop detection is performed on each frame of information collected by the robot, not only on the key frame, that is, each frame of information collected by the robot through the TOF sensor needs to be loop detected to determine the difference between the planar point feature information of the current frame and the data in the local map, so as to determine whether the current frame is a loop frame, and if so, a loop factor is added to construct an error term through loop.

[0076] S602, if it is determined that the current frame is a loop frame, a loop factor of the loop frame is obtained.

[0077] Specifically, after loop detection of the current frame, it can be determined whether the current frame is a loop frame, and if the current frame is a loop frame, the loop factor of the loop frame is further determined.

[0078] In this embodiment, as shown, Figure 7 determining that the current frame is a loop frame includes the following steps:

[0079] S701, candidate key frames are obtained according to the pose information of the robot and historical key frames.

[0080] Specifically, the preset map in this embodiment includes a plurality of historical key frames, and through loop detection, candidate frames can be selected from the plurality of historical key frames. The specific selection method is to find historical key frames within a preset distance range around the current frame in the preset map, for example, to find historical key frames within one meter around the current frame. Then, the time difference between these historical key frames and the current frame is compared to filter out historical key frames with a recent time, for example, to filter out historical key frames within ten minutes. Then, according to the heading angle in the pose information of the robot, historical key frames with a heading angle close to that of the robot corresponding to the current frame are selected as candidate key frames, for example, if the heading angles of the two are within 10 degrees, it is determined that the heading angles are close.

[0081] S702, the candidate key frame and the current frame are registered, and a registration value of the current frame is obtained. S703, if the registration value is less than a registration threshold, it is determined that the current frame is a loop frame.

[0082] Specifically, after the candidate key frame is determined, the candidate key frame is registered with the current frame, and the registration manner can be ICP registration. It should be noted that the candidate key frame generally has multiple frames, and each frame of the candidate key frame is registered with the current frame by ICP registration. In the registration process, whether the current frame is successfully registered can be determined by an ICP registration threshold. More specifically, an ICP registration value can be obtained after the current frame is registered with the candidate key frame. When the ICP registration value is less than the ICP registration threshold, the current frame is determined to be a loop frame, and the ICP registration threshold can be 0.01.

[0083] In addition, it should be noted that loop detection has a very high risk and benefit. If the loop detection is wrong, the detection result will be greatly affected and the map will become very unclear, which will seriously affect the driving of the robot. If the loop detection is successful, the cumulative error can be significantly reduced, which helps the robot to more accurately and quickly avoid obstacles and navigate. Therefore, after the current frame is determined to be a loop frame, the loop factor of the loop frame is obtained, which can greatly improve the success rate of loop detection. The loop factor can be determined according to the ICP registration value of the loop frame, for example, the ICP registration value can be divided into several intervals, and each interval corresponds to a loop factor, so as to determine the loop factor of each loop frame. Of course, this embodiment only gives an implementation manner, and the loop factor can also be determined by other manners in related technologies, which is not limited herein.

[0084] S603, determining that the loop frame is a key frame, and then performing factor graph optimization on the pose information of the robot based on the loop factor. S604, determining that the loop frame is a non-key frame, and then accumulating the loop factor for the next factor graph optimization.

[0085] When the loop frame is a key frame, the loop factor is added, and in the above embodiment, when the loop frame is determined to be a key frame, the odometer factor can also be added. In this embodiment, the pose information of the robot can be optimized according to the odometer factor and the loop factor, that is, by adjusting the pose information of the robot, the error term constructed by the odometer and the error term constructed by the loop are adjusted. The odometer and the loop can also react whether the current pose information of the robot is accurate. If it is determined that the loop frame is not a key frame, the loop factor is accumulated, and when the next loop frame is determined to be a key frame, the accumulated loop factor is added to the process to perform factor graph optimization.

[0086] In summary, refer to Figure 8Firstly, it should be noted that the map module stores a preset map, i.e. a local map of the environment where the robot is currently located. When it is determined that the current frame is a key frame, the local map can be updated according to the information of the key frame. After each update, the map module sends the local map to the display module for display. Secondly, referring to Figure 8 The feature extraction module can extract the planar point feature information of the current frame according to the information collected by the left TOF sensor and the front TOF sensor. The fused odometry module fuses the data detected by the inertial sensor and the wheel speed odometry to obtain the advancing direction and horizontal position information of the robot. The LM optimization module can optimize the pose information of the robot according to the planar point feature information of the current frame, the advancing direction and horizontal position of the robot, and the local map obtained from the map module. The specific optimization method can be referred to the description in the above embodiment. After the LM optimization is completed, it is determined whether the current frame is a key frame. When the current frame is a key frame, the odometry factor is constructed. At the same time, the current frame is also subjected to loop detection. The loop detection module extracts the planar point feature information of the current frame from the feature extraction, and extracts the local map from the map module. Then, the loop detection is performed according to the information of the current frame and the local map to determine whether the current frame is a loop frame. If yes, the loop factor is added, and it is determined whether the loop frame is a key frame. When it is determined that the loop frame is a key frame, the loop factor is also added to the factor graph optimization for optimization processing, and the local map is updated after the processing. It should be noted that when the determination result of the determination block in the embodiment is no, the processing of the current frame is ended. In addition, the TOF sensor, the inertial sensor and the wheel speed odometry can be set to obtain data once every preset time.

[0087] In summary, the robot mapping method of the embodiment of the present application can construct a map of the environment where the robot is located at low cost, and can also improve the environmental adaptability of the robot when mapping, thereby improving the mapping success rate.

[0088] Figure 9 is a structural block diagram of the robot vacuum cleaner according to the embodiment of the present application.

[0089] Further, as shown in Figure 9 , the present application proposes a robot vacuum cleaner 10, which comprises a memory 11, a processor 12, and a robot mapping program stored on the memory 11 and executable on the processor 12. When the processor 12 executes the robot mapping program, the robot mapping method according to the above embodiment is realized.

[0090] The robot mapping device according to the embodiment of the present application can construct a map of an environment in which a robot is located at a low cost, and can also improve the environmental adaptability of the robot during mapping, thereby improving the mapping success rate.

[0091] Further, the present application provides a computer readable storage medium, which stores a robot mapping program, and the robot mapping program is executed by a processor to implement the robot mapping method according to the above embodiment.

[0092] The computer readable storage medium according to the embodiment of the present application can construct a map of an environment in which a robot is located at a low cost, and can also improve the environmental adaptability of the robot during mapping, thereby improving the mapping success rate.

[0093] Figure 10 is a structural block diagram of the robot mapping device according to the embodiment of the present application.

[0094] Further, as shown in Figure 10 the present application provides a robot mapping device 100, wherein the robot is provided with a TOF sensor, and the mapping device 100 comprises a first acquisition module 101, a second acquisition module 102 and an updating module 103.

[0095] The first acquisition module 101 is configured to acquire pose information of the robot; the second acquisition module 102 is configured to acquire point cloud information of a current frame through the TOF sensor; and the updating module 103 is configured to determine that the current frame is a key frame, and then update the point cloud information and the pose information of the current frame into a preset map.

[0096] In some embodiments of the present application, the updating module 103 is specifically configured to construct an odometry factor, perform factor graph optimization on the pose information of the robot according to the odometry factor, and update the point cloud information of the current frame and the pose information of the robot after the factor graph optimization into the preset map.

[0097] In some embodiments of the present application, the mapping device 100 further comprises a third acquisition module, configured to acquire plane point feature information of the current frame according to the point cloud information of the current frame before determining that the current frame is a key frame, and construct a loss function according to the plane point feature information; a first determination module, configured to determine a forward direction and horizontal position information of the robot according to the pose information of the robot; and a first optimization module, configured to optimize the pose information of the robot according to the forward direction and the horizontal position information of the robot, the preset map and the loss function, and take the optimized pose information as the pose information of the robot.

[0098] In some embodiments of the present application, the third acquisition module is specifically configured to calculate the curvature of each point in the point cloud information of the current frame; and determine the planar point feature information of the current frame according to the curvature of each point.

[0099] In some embodiments of the present application, the mapping device 100 further comprises: a detection module, configured to perform loop detection according to the planar point feature information of the current frame and a preset map after the planar point feature information of the current frame is acquired according to the point cloud information of the current frame; a second determination module, configured to determine that the current frame is a loop frame, and acquire a loop factor of the loop frame; a second optimization module, configured to determine that the loop frame is a key frame, and perform factor graph optimization on the pose information of the robot based on the loop factor; and an accumulation module, configured to determine that the loop frame is a non-key frame, and accumulate the loop factor for next time factor graph optimization.

[0100] In some embodiments of the present application, the second determination module is specifically configured to acquire a candidate key frame according to the pose information of the robot and a historical key frame; perform registration on the candidate key frame and the current frame, and obtain a registration value of the current frame; and determine that the current frame is a loop frame if the registration value is less than a registration threshold.

[0101] In some embodiments of the present application, the update module 103 is specifically configured to determine that the current frame is a key frame if the moving distance of the robot is greater than a preset distance threshold and / or the rotation angle of the robot is greater than a preset angle threshold.

[0102] In some embodiments of the present application, the first acquisition module 101 is specifically configured to acquire first pose information of the robot through an inertial sensor, and acquire second pose information of the robot through a wheel speed odometer; and perform fusion processing on the first pose information and the second pose information to obtain the pose information of the robot.

[0103] It should be noted that the specific implementation of the robot mapping device in the embodiments of the present application can refer to the specific implementation of the robot mapping method in the above embodiments, which will not be repeated here.

[0104] In summary, the robot mapping device in the embodiments of the present application can construct a map of the environment where the robot is located at low cost, and can also improve the environmental adaptability of the robot during mapping, thereby improving the mapping success rate.

[0105] It is to be appreciated that the above description and the examples that follow are intended to be illustrative only and that changes can be made to the description and examples without departing from the scope of the application. Note also that the use of particular brand names in the description is solely for illustration and should not be construed as an endorsement of such brands.

[0106] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following techniques, which are well known in the art of hardware implementation, can be used: a hybrid of the above techniques, a mixture of two or more of the above techniques, or a combination of the above techniques with other techniques not listed above.

[0107] In the description of the present application, the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative appearances of the above-mentioned terms in various places in the specification are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0108] In the description of the present application, it needs to be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0109] In addition, the terms "first", "second", and the like used in the embodiments of the present application are only for the purpose of description, and cannot be understood as indicating or implying relative importance, or implicitly indicating the number of technical features referred to in the embodiments. Therefore, the features defined with "first", "second" and the like in the embodiments of the present application can be explicitly or implicitly indicated to include at least one of the features. In the description of the present application, the meaning of the word "plurality" is at least two or two or more, such as two, three, four, etc., unless otherwise specifically limited in the embodiments.

[0110] In the present application, unless otherwise specifically provided or limited in the embodiments, the terms "mounting", "connecting", "connecting" and "fixing" and the like appearing in the embodiments should be understood broadly, for example, the connection can be fixed connection, or detachable connection, or integral, which can be understood, or can be mechanical connection, electrical connection, etc. Of course, it can also be directly connected, or indirectly connected through an intermediate medium, or the internal communication of two elements, or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific implementation situation.

[0111] In the present application, unless otherwise specifically provided and limited, the first feature "on" or "under" the second feature can be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature "above", "above" and "above" the second feature can be that the first feature is directly above or obliquely above the second feature, or only indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature "below", "below" and "below" the second feature can be that the first feature is directly below or obliquely below the second feature, or only indicates that the horizontal height of the first feature is less than that of the second feature.

[0112] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that variations, modifications, substitutions and changes can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A robot mapping method, applied to a sweeping robot, characterized in that: The sweeping robot is provided with a TOF sensor, and the method includes: Obtaining the position information of the sweeping robot; Obtaining point cloud information of the current frame through the TOF sensor, wherein the sweeping robot is provided with a first TOF sensor and a second TOF sensor, the first TOF sensor is provided in front of the sweeping robot, and the second TOF sensor is provided on the left of the sweeping robot, the first TOF sensor is used to obtain point cloud information of the first current frame, and the second TOF sensor is used to obtain point cloud information of the second current frame, and the first TOF sensor and the second TOF sensor are arranged on the sweeping robot at a suitable angle, the suitable angle ensures that the point cloud information collected by the first TOF sensor and the second TOF sensor does not contain repeated information, and the suitable angle ensures that the point cloud information collected by the first TOF sensor and the second TOF sensor can form a complete and continuous current frame; If the current frame is determined to be a key frame, the point cloud information and the posture information of the current frame are updated to a preset map.

2. The method according to claim 1, characterized in that The updating of the point cloud information and the pose information of the current frame into a preset map includes: Construct odometry factor; Performing factor graph optimization on the posture information of the robot according to the odometry factor; The point cloud information of the current frame and the posture information of the robot after factor graph optimization are updated to the preset map.

3. The method according to claim 1, characterized in that Before determining that the current frame is a key frame, the method further includes: Acquire plane point feature information of the current frame according to the point cloud information of the current frame, and construct a loss function according to the plane point feature information; Determining the forward direction and horizontal position information of the robot according to the posture information of the robot; The posture information of the robot is optimized according to the forward direction and horizontal position information of the robot, the preset map, and the loss function, and the optimized posture information is used as the posture information of the robot.

4. The method according to claim 3, characterized in that The acquiring the planar point feature information of the current frame according to the point cloud information of the current frame includes: Calculating the curvature of each point in the point cloud information of the current frame; The planar point feature information of the current frame is determined according to the curvature of each point.

5. The method according to claim 3, characterized in that After acquiring the plane point feature information of the current frame according to the point cloud information of the current frame, the method further includes: Perform loop detection based on the plane point feature information of the current frame and the preset map; Determining that the current frame is a loop frame, obtaining a loop factor of the loop frame; Determining that the loop frame is a key frame, and then performing factor graph optimization on the posture information of the robot based on the loop factor; If it is determined that the loop frame is a non-key frame, the loop factor is accumulated for the next factor graph optimization.

6. The method according to claim 5, characterized in that The determining that the current frame is a loop frame includes: Obtaining candidate key frames based on the robot's posture information and historical key frames; Registering the candidate key frame with the current frame, and obtaining a registration value of the current frame; If the registration value is less than the registration threshold, it is determined that the current frame is a loop frame.

7. The method according to any one of claims 1 to 6, characterized in that The determining that the current frame is a key frame includes: If the movement distance of the robot is greater than a preset distance threshold and / or the rotation angle of the robot is greater than a preset angle threshold, the current frame is determined to be a key frame.

8. The method according to claim 7, characterized in that The obtaining of the posture information of the robot includes: Acquiring a first posture information of the robot through an inertial sensor, and acquiring a second posture information of the robot through a wheel speed odometer; The first posture information and the second posture information are fused to obtain the posture information of the robot.

9. A sweeping robot, characterized in that: The robot mapping method comprises a memory, a processor, and a robot mapping program stored in the memory and executable on the processor. When the processor executes the robot mapping program, the robot mapping method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that A robot mapping program is stored thereon, and when the robot mapping program is executed by the processor, the robot mapping method according to any one of claims 1 to 8 is implemented.

11. A robot mapping device, applicable to the robot mapping method according to any one of claims 1 to 8, characterized in that: The robot is provided with a TOF sensor, and the device comprises: A first acquisition module is used to acquire the position and posture information of the robot; A second acquisition module is used to acquire point cloud information of the current frame through the TOF sensor; The updating module is used to determine that the current frame is a key frame, and then update the point cloud information and the posture information of the current frame into a preset map.

Images