Method and device for generating obstacle map, cleaning robot and storage medium

Abstract

The invention provides a method and device for generating an obstacle map, a cleaning robot and a storage medium, and the method comprises the steps: processing first point cloud data obtained through the scanning of a laser radar of the cleaning robot and second point cloud data obtained through the scanning of a depth camera, and obtaining the target point cloud data of a current frame representing the distribution condition of surrounding obstacles; determining current pose information of the cleaning robot according to the target point cloud data of the current frame, wherein the current pose information represents the position and the pose of the cleaning robot when the target point cloud data of the current frame is obtained through scanning; and generating obstacle map data of the current frame according to the current pose information and the target point cloud data of the current frame, and obtaining an obstacle map of the environment according to the obstacle map data of multiple frames including the current frame. According to the scheme, the obstacle map is generated according to the point cloud data obtained by simultaneous scanning of the laser radar and the depth camera, so that the problem that the laser radar cannot scan low obstacles is solved, and a more accurate obstacle map is obtained.

Description

Technical Field

[0001] The present application relates to the technical field of cleaning robots, and in particular to a method, device, cleaning robot and storage medium for generating an obstacle map. Background Art

[0002] With the development of electronic information technology, a variety of robotic devices have appeared in daily life, such as sweeping robots, food delivery robots, etc. In order to better plan the driving path, the robot often needs to scan the surrounding environment to establish an obstacle map that reflects the distribution of surrounding obstacles.

[0003] At present, the commonly used method of establishing an obstacle map is to use a laser radar to scan the surrounding environment, obtain a set of laser point cloud data reflecting the location of surrounding obstacles, and then establish an obstacle map based on the laser point cloud data. However, the laser radar itself can only scan a two-dimensional plane, and cannot measure obstacles below the height of the two-dimensional plane (for example, the laser installed on the top of the robot is about 105cm high and cannot identify common static obstacles such as chairs and stools). Therefore, the obstacle map established in this way cannot accurately reflect the distribution of obstacles in the surrounding environment, especially the distribution of low obstacles, which has an adverse effect on the efficiency of subsequent robot operations based on the obstacle map. Summary of the invention

[0004] In view of this, the present application provides a method, device, cleaning robot and storage medium for generating an obstacle map to solve the problem of inaccurate obstacle maps generated in the related art.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] A first aspect of the present application provides a method for generating an obstacle map, the method being applied to a cleaning robot, the method comprising:

[0007] Processing the first point cloud data and the second point cloud data to obtain target point cloud data of the current frame, wherein the target point cloud data of the current frame represents the distribution of obstacles around the cleaning robot, the first point cloud data is obtained by scanning with a laser radar of the cleaning robot, and the second point cloud data is obtained by scanning with a depth camera of the cleaning robot;

[0008] Determine the current posture information of the cleaning robot according to the target point cloud data of the current frame, wherein the current posture information represents the position and posture of the cleaning robot when the target point cloud data of the current frame is obtained by scanning;

[0009] Generate obstacle map data of the current frame according to the current posture information and the target point cloud data of the current frame;

[0010] Obstacle map data of multiple frames including the obstacle map data of the current frame are processed to obtain an obstacle map of the environment in which the cleaning robot is located.

[0011] Optionally, the processing of the first point cloud data and the second point cloud data to obtain target point cloud data of the current frame includes:

[0012] The first point cloud data and the second point cloud data are fused to obtain target point cloud data of the current frame.

[0013] Optionally, determining the current posture information of the cleaning robot according to the target point cloud data of the current frame includes:

[0014] Determining the posture change information of the cleaning robot according to the target point cloud data of the current frame;

[0015] The current posture information of the cleaning robot is determined based on the posture change information and the posture information of the cleaning robot at a previous moment. The posture information at the previous moment represents the position and posture of the cleaning robot when the target point cloud data of the previous frame is scanned and obtained. The posture information of the cleaning robot when the first frame of target point cloud data is scanned and obtained is the preset initial posture information.

[0016] Optionally, determining the posture change information of the cleaning robot according to the target point cloud data of the current frame includes at least one of the following:

[0017] Determining the posture change information of the cleaning robot according to the target point cloud data of the current frame and the target point cloud data of the previous frame;

[0018] Initial map data is determined according to the target point cloud data of the current frame, and position change information of the cleaning robot is determined according to the initial map data and the obstacle map data of the previous frame.

[0019] Optionally, also include:

[0020] When it is determined that the cleaning robot has reached any historical position, the generated multi-frame obstacle map data is optimized according to the current posture information and historical posture information of the cleaning robot, wherein the historical posture information represents the position and posture of the cleaning robot when it was at the historical position last time.

[0021] Optionally, determining that the cleaning robot has reached any historical position includes:

[0022] Matching the obstacle map data of the current frame with the generated multiple frames of historical obstacle map data;

[0023] If the obstacle map data of the current frame successfully matches any frame of generated historical obstacle map data, it is determined that the cleaning robot has reached a historical position corresponding to the successfully matched historical obstacle map data.

[0024] Optionally, generating obstacle map data of the current frame according to the current posture information and the target point cloud data of the current frame includes:

[0025] The extended Kalman filter algorithm is used to optimize the target point cloud data of the current frame according to the current posture information to obtain the obstacle map data of the current frame.

[0026] A second aspect of the present application provides a device for generating an obstacle map, the device being applied to a cleaning robot, the device comprising:

[0027] a processing unit, configured to process the first point cloud data and the second point cloud data to obtain target point cloud data of a current frame, wherein the target point cloud data of the current frame represents a distribution of obstacles around the cleaning robot, wherein the first point cloud data is obtained by scanning with a laser radar of the cleaning robot, and the second point cloud data is obtained by scanning with a depth camera of the cleaning robot;

[0028] A determination unit, configured to determine current posture information of the cleaning robot according to the target point cloud data of the current frame, wherein the current posture information represents the position and posture of the cleaning robot when the target point cloud data of the current frame is obtained by scanning;

[0029] A generating unit, configured to generate obstacle map data of a current frame according to the current posture information and the target point cloud data of the current frame;

[0030] The obtaining unit is used to process obstacle map data of multiple frames including the obstacle map data of the current frame to obtain an obstacle map of the environment in which the cleaning robot is located.

[0031] A third aspect of the present application provides a cleaning robot, comprising:

[0032] one or more processors;

[0033] a memory having one or more programs stored therein;

[0034] LiDAR and depth cameras;

[0035] When the one or more programs are executed by the one or more processors, the one or more processors control the lidar to scan to obtain first point cloud data, control the depth camera to scan to obtain second point cloud data, and implement the method for generating an obstacle map provided in any one of the first aspects of the present application based on the first point cloud data and the second point cloud data.

[0036] A fourth aspect of the present application provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for generating an obstacle map as provided in any one of the first aspects of the present application is implemented.

[0037] The beneficial effects of this solution are:

[0038] In this solution, when generating obstacle map data, the cleaning robot combines the lidar and depth camera to scan simultaneously to obtain the point cloud data required to generate the obstacle map data. Therefore, this solution can use the depth camera as a supplement to overcome the problem that the lidar cannot scan low obstacles and obtain a more comprehensive and accurate obstacle map. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0040] Figure 1 A flowchart of a method for generating an obstacle map disclosed in an embodiment of the present application;

[0041] Figure 2 A flowchart of scanning and obtaining target point cloud data disclosed in an embodiment of the present application;

[0042] Figure 3 A flowchart of determining the current posture information of a cleaning robot disclosed in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the structure of a device for generating an obstacle map provided in an embodiment of the present application;

[0044] Figure 5 A schematic diagram of the structure of a device for generating an obstacle map provided in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of the structure of a cleaning robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] In this application, the terms "comprises", "comprising" or any other variation thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0048] Furthermore, in this document, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0049] This application embodiment provides a method for generating an obstacle map. Figure 1 , is a flow chart of the method, and the method may include the following steps.

[0050] The method provided in this embodiment can be applied to a cleaning robot, that is, each of the following steps can be executed by the cleaning robot.

[0051] S101, processing the first point cloud data and the second point cloud data to obtain target point cloud data of the current frame, the target point cloud data of the current frame represents the distribution of obstacles around the cleaning robot, the first point cloud data is obtained by scanning the cleaning robot's laser radar, and the second point cloud data is obtained by scanning the cleaning robot's depth camera.

[0052] The cleaning robot of this embodiment may be a robot device equipped with a laser radar and a depth camera and capable of moving within a certain range, or may be other devices with corresponding scanning and moving capabilities.

[0053] Before starting to execute S101, the parameters of the cleaning robot can be configured by manual settings, including but not limited to configuring the initial posture information of the cleaning robot, the internal and external parameters of the lidar, the internal and external parameters of the depth camera, etc.

[0054] The initial position and posture information of the cleaning robot may include the position information of the cleaning robot when the cleaning robot starts to scan the first frame of target point cloud data and the posture information of the cleaning robot.

[0055] After completing the above settings, the cleaning robot can move in the spatial environment where it is located, and in the process of moving, combined with the set parameters, generate frame by frame of obstacle map data in real time according to the method of this embodiment. Each frame of obstacle map data reflects the distribution of obstacles around the position of the cleaning robot when the frame data is generated. The multiple frames of obstacle map data scanned thereby can constitute an obstacle map of the spatial environment where the cleaning robot is located.

[0056] The spatial environment in which the cleaning robot is located can be an indoor environment such as a classroom, a living room, a supermarket, a warehouse, etc., or an outdoor environment within a certain range, such as a playground, an outdoor basketball court, a park, etc., which is not limited in this embodiment.

[0057] A depth camera refers to a camera device that can obtain the distance information of the object relative to the camera (called the depth information of the object) when photographing an object. The depth image taken by the depth camera can not only reflect the position of the object on the image plane, but also reflect the distance from the object to the camera when shooting. Therefore, the position of the object in three-dimensional space can be restored based on the depth image.

[0058] In this embodiment, the target point cloud data of the current frame may include the first point cloud data and the second point cloud data. Correspondingly, the process of scanning to obtain the target point cloud data of the current frame may include the following steps: Figure 2 Steps shown.

[0059] S201, using the laser radar of the cleaning robot to scan and obtain the first point cloud data of the current frame.

[0060] In step S201, the cleaning robot may first remove noise and outliers from the signal obtained after the laser radar scan, and then determine the first point cloud data of the current frame based on the scanned signal and the internal and external parameters of the laser radar. The first point cloud data of the current frame may include the three-dimensional coordinate data of multiple points on the surface of obstacles around the cleaning robot scanned by the laser radar relative to the cleaning robot, and the distribution of obstacles around the cleaning robot may be determined through these data.

[0061] S202, using the depth camera of the cleaning robot to obtain a depth image of the current frame in the environment.

[0062] The order of the above steps is only an example. During actual scanning, the cleaning robot can use the lidar and the depth camera to scan at the same time. That is, the cleaning robot arrives at a location and scans the surrounding environment with the lidar and the depth camera at the same time. The lidar scans to obtain the first point cloud data of the current frame, and the depth camera captures to obtain the depth image of the current frame.

[0063] S203, obtaining second point cloud data of the current frame according to the depth image of the current frame.

[0064] In step S203, the cleaning robot may process the depth image of the current frame based on the three-dimensional reconstruction technology in combination with the internal and external parameters of the depth camera to obtain second point cloud data of the current frame.

[0065] The specific implementation method of the three-dimensional reconstruction technology can be found in the relevant existing technologies and will not be described in detail.

[0066] According to the depth image of the current frame, the cleaning robot can determine the three-dimensional coordinate data of the points on the surface of each object captured in the image relative to the cleaning robot, and the set of these three-dimensional coordinate data is equivalent to the second point cloud data of the current frame.

[0067] It should be noted that the depth image of the current frame may include one or more frames. For example, the cleaning robot may use a depth camera to rotate once to obtain depth images in multiple directions, and these depth images all belong to the depth image of the current frame.

[0068] S204: Fusing the first point cloud data and the second point cloud data to obtain target point cloud data of the current frame.

[0069] In step S204, the cleaning robot may fuse the first point cloud data and the second point cloud data in a variety of ways.

[0070] In some embodiments, the cleaning robot may delete duplicate three-dimensional coordinate data in the first point cloud data and the second point cloud data. Specifically, the cleaning robot may compare the first point cloud data and the second point cloud data to determine duplicate three-dimensional coordinate data in the two and delete the duplicate three-dimensional coordinate data. For example, assuming that both the first point cloud data and the second point cloud data include three-dimensional coordinate data of a point P in space, the three-dimensional coordinate data of a point P may be deleted from the first point cloud data or from the second point cloud data.

[0071] In some embodiments, the cleaning robot may merge every two three-dimensional coordinate data in the first point cloud data and the second point cloud data whose distance is less than a specific threshold into one three-dimensional coordinate data. Specifically, the cleaning robot may determine the points in the first point cloud data and the second point cloud data that are closer, merge the two points that are closer into one point, and delete the original two points; the merging method may be to merge the two points that are closer into the midpoint of the two points. Here, the closer distance can be represented by the distance between the two points being less than a specific threshold.

[0072] Exemplarily, assuming that the first point cloud data includes three-dimensional coordinate data of point P1 in space, the second point cloud data includes three-dimensional coordinate data of point P2 in space, and the distance between point P1 and point P2 is less than a specific threshold, the three-dimensional coordinate data of the midpoint P0 between points P1 and P2 can be determined, the three-dimensional coordinate data of point P1 in the first point cloud data is replaced with the three-dimensional coordinate data of point P0, and the original three-dimensional coordinate data of point P1 and point P2 are deleted.

[0073] By merging points that are close to each other in the above manner, on the one hand, the amount of data in the merged target point cloud data can be reduced to improve processing efficiency; on the other hand, since two points that are close to each other are merged into one point, the accuracy of the generated obstacle map will not be affected.

[0074] In some embodiments, the cleaning robot can also identify abnormal three-dimensional coordinate data in the first point cloud data and the second point cloud data and remove the abnormal three-dimensional coordinate data. Specifically, the cleaning robot can determine the space covered by the first point cloud data in the three-dimensional space, and then determine the points in the second point cloud data that are too far away from the space covered by the first point cloud data (for example, greater than a preset distance threshold) as abnormal points. Similarly, it can determine the space covered by the second point cloud data in the three-dimensional space, and then determine the points in the first point cloud data that are too far away from the space covered by the second point cloud data as abnormal points, and delete the abnormal three-dimensional coordinate data corresponding to these abnormal points from the first point cloud data and the second point cloud data, respectively.

[0075] After processing the first point cloud data and the second point cloud data in one or more of the above-mentioned ways, the cleaning robot may determine the union of the processed first point cloud data and the processed second point cloud data as the target point cloud data of the current frame.

[0076] Figure 2 In the method shown, steps S201 to S203 are equivalent to the process of obtaining the first point cloud data and the second point cloud data by scanning with a laser radar and a depth camera before step S101, and step S204 is equivalent to a specific implementation of step S101.

[0077] S102, determining current position and posture information of the cleaning robot according to the target point cloud data of the current frame, where the current position and posture information represents the position and posture of the cleaning robot when the target point cloud data of the current frame is obtained by scanning.

[0078] In step S102, if the current frame is the first frame of target point cloud data obtained by the cleaning robot when it starts scanning, the cleaning robot may determine the pre-configured initial pose information as the current pose information of the cleaning robot.

[0079] If the current frame is not the first frame of target point cloud data, the cleaning robot can press Figure 3 The method shown determines the current position information, that is, determines the position and posture of the cleaning robot when scanning to obtain the target point cloud data of the current frame.

[0080] S301, determining the posture change information of the cleaning robot according to the target point cloud data of the current frame.

[0081] S302, determine the current posture information of the cleaning robot based on the posture change information and the posture information of the cleaning robot at the previous moment, the posture information at the previous moment represents the position and posture of the cleaning robot when scanning to obtain the target point cloud data of the previous frame, and the posture information of the cleaning robot when scanning to obtain the first frame of target point cloud data is the preset initial posture information.

[0082] The posture change information represents the degree of change of the posture of the cleaning robot when scanning the current frame relative to the posture of the cleaning robot when scanning the previous frame. For example, the posture change information can represent that, relative to the posture of the cleaning robot when scanning the previous frame, the cleaning robot moves a certain distance in a certain direction when scanning the current frame, and the cleaning robot rotates a specific angle clockwise or counterclockwise.

[0083] In step S301, the cleaning robot may determine the posture change information in at least one of the following two ways.

[0084] The first method is to determine the posture change information of the cleaning robot according to the target point cloud data of the current frame and the target point cloud data of the previous frame.

[0085] In the first method, the cleaning robot can store each frame of target point cloud data obtained by scanning, or store the target point cloud data obtained by scanning the most recent frames. These data can be stored in the local memory of the cleaning robot or in other devices that are connected to the cleaning robot for communication.

[0086] After scanning and obtaining the target point cloud data of the current frame, the cleaning robot can read the target point cloud data of the previous frame, and use the iterative closest point (ICP) algorithm or other similar algorithms to process the target point cloud data of the current frame and the target point cloud data of the previous frame, thereby obtaining the optimal transformation matrix between the target point cloud data of the current frame and the target point cloud data of the previous frame. The optimal transformation matrix is ​​equivalent to the posture change information of the cleaning robot.

[0087] The second method is to determine the initial map data according to the target point cloud data of the current frame, and determine the posture change information of the cleaning robot according to the initial map data and the obstacle map data of the previous frame.

[0088] In the second method, the cleaning robot can first determine an estimated pose information based on its own moving speed and the pose information of the previous frame. The estimated pose information represents the rough position information and posture information of the cleaning robot when scanning the current frame. Then, the cleaning robot determines the initial map data of the current frame based on the estimated pose information and the target point cloud data of the current frame. Then, the cleaning robot can use the ICP algorithm or other similar algorithms to process the initial map data of the current frame and the obstacle map data of the previous frame to determine the optimal transformation matrix between the two. The optimal transformation matrix is ​​equivalent to the pose change information of the cleaning robot.

[0089] In step S302, the posture information at the previous moment can be represented in matrix form, which is called the posture matrix. After obtaining the optimal transformation matrix as the posture change information, the posture matrix representing the posture information at the previous moment can be transformed by the optimal transformation matrix. The transformed matrix obtained can be regarded as the posture matrix representing the current posture information.

[0090] S103, generating obstacle map data of the current frame according to the current posture information and the target point cloud data of the current frame.

[0091] In step S103, the cleaning robot can optimize the target point cloud data of the current frame according to the current posture information to obtain the obstacle map data of the current frame. The obstacle map data of the current frame may include a plurality of three-dimensional coordinate data, which represent the positions of the points on the obstacle surface around the cleaning robot in the spatial environment where the cleaning robot is located when scanning the current frame.

[0092] The above optimization can be achieved by using an extended Kalman filter (EKF) algorithm or other similar filtering algorithms.

[0093] S104, processing obstacle map data of multiple frames including obstacle map data of the current frame to obtain an obstacle map of the environment in which the cleaning robot is located.

[0094] In step S104, the plurality of frames of obstacle map data may include the obstacle map data of the current frame and several frames of obstacle map data obtained before the obstacle map data of the current frame.

[0095] Specifically, before executing step S104, the cleaning robot may repeatedly execute the process of steps S101 to S103 for multiple times. Each time the process is executed, the cleaning robot can obtain a frame of obstacle map data, and the frame of obstacle map data can be stored in the memory of the cleaning robot. Moreover, after obtaining a frame of obstacle map data each time, the cleaning robot can move a certain distance in any direction to obtain a frame of obstacle map data at a new position, and so on, until the cleaning robot stops scanning. In this way, the cleaning robot can obtain the above multiple frames of obstacle map data.

[0096] In some embodiments, when the cleaning robot stops scanning, the cleaning robot can create a blank grid map based on the initial posture information, and then add the obstacle map data of the current frame obtained by the last execution of steps S101 to S103, and the obstacle map data of multiple frames previously obtained and stored in the memory to the blank grid map one by one. The grid map with the obstacle map data of multiple frames added can be regarded as the obstacle map of the environment in which the cleaning robot is located. The initial posture information may include: the posture information of the cleaning robot and the laser radar and depth camera contained in the cleaning robot when the scanning starts.

[0097] The way to add obstacle map data to the grid map is to determine which grids of the grid map each three-dimensional coordinate data in the obstacle map data is in, mark the grids where the obstacle map data is located as grids occupied by obstacles, and mark the grids that do not contain obstacle map data as grids not occupied by obstacles. In this way, a grid map representing the distribution of obstacles in the spatial environment where the cleaning robot is located can be obtained, that is, an obstacle map is obtained.

[0098] Optionally, after obtaining the obstacle map, the trajectory information of the cleaning robot during the scanning process can also be added to the obstacle map so that other devices can refer to the trajectory information for navigation.

[0099] When the obstacle map is the aforementioned grid map, the way to add the trajectory information may be to mark the grids passed by the cleaning robot in the grid map as trajectory grids, and the set of these trajectory grids is equivalent to the trajectory information.

[0100] The cleaning robot can stop scanning in response to an external control command, or it can stop scanning after a specified cumulative scanning time, or it can stop scanning after a specified cumulative movement distance, or it can stop scanning after a certain number of frames of obstacle map data are accumulated.

[0101] The beneficial effect of this embodiment is that when the cleaning robot generates obstacle map data, it combines the laser radar and the depth camera to scan simultaneously to obtain the point cloud data required to generate the obstacle map data. Therefore, this solution can use the depth camera as a supplement to overcome the problem that the laser radar cannot scan low obstacles and obtain a more comprehensive and accurate obstacle map.

[0102] Optionally, during the scanning process, the cleaning robot may pass through repeated positions, that is, the position reached by the cleaning robot at a certain moment may be a historical position that has been reached before.

[0103] In this case, the cleaning robot can determine whether the current position it has reached belongs to a historical position it has reached before, that is, determine whether the cleaning robot has reached any historical position. The specific determination method can be:

[0104] Matching the obstacle map data of the current frame with the generated multiple frames of historical obstacle map data;

[0105] If the obstacle map data of the current frame successfully matches any frame of generated historical obstacle map data, it is determined that the cleaning robot has reached a historical position corresponding to the successfully matched historical obstacle map data.

[0106] The cleaning robot can match the obstacle map data of the current frame with the historical obstacle map data through a variety of existing algorithms. For example, a loop closure detection algorithm (Loop Closure Detection) can be used for matching. The specific implementation of the algorithm can refer to the relevant existing technology and will not be repeated here.

[0107] Exemplarily, after the cleaning robot reaches position A and generates a frame of obstacle map data of the current frame at position A in the aforementioned manner, the cleaning robot matches the obstacle map data of the current frame with the multiple frames of obstacle map data generated previously based on the closed-loop detection algorithm. After matching, if it is found that the obstacle map data of the current frame and a certain frame of obstacle map data generated previously can be successfully matched, the cleaning robot determines that position A is a historical position that has been reached at least once during the previous scanning process, and the current arrival at position A is the Nth time (N is greater than or equal to 2) that the cleaning robot has reached this position. If it is found that the obstacle map data of the current frame fails to match each frame of obstacle map data generated previously, it is determined that the cleaning robot has arrived at position A for the first time, and position A does not belong to the historical position that has been reached.

[0108] After determining that the cleaning robot has reached any historical position, for example, in the above example, the cleaning robot has reached position A and has determined that position A is a historical position, the cleaning robot may optimize the generated multi-frame obstacle map data as follows:

[0109] When it is determined that the cleaning robot has reached any historical position, the generated multi-frame obstacle map data is optimized according to the current posture information and historical posture information of the cleaning robot, where the historical posture information represents the position and posture of the cleaning robot when it was at the historical position last time.

[0110] First, when it is determined that the cleaning robot has reached a historical position, the cleaning robot may not press Figure 3 The current pose information is determined in the following way:

[0111] Process the target point cloud data of the current frame and the historical target point cloud data of the same position according to the ICP algorithm or other similar algorithms, or process the initial map data of the current frame and the historical obstacle map data of the same position according to the ICP algorithm or other similar algorithms to obtain historical posture change information, which represents the change of the posture of the cleaning robot when scanning the current frame relative to the posture of the cleaning robot when it last arrived at the current position;

[0112] The historical target point cloud data at the same location refers to a frame of target point cloud data scanned by the cleaning robot when it last arrived at the historical location, and the historical obstacle map data at the same location refers to a frame of obstacle map data generated when the cleaning robot last arrived at the historical location;

[0113] Then, the cleaning robot determines the current posture information according to the historical posture change information and the historical posture information; the implementation method of this step is similar to step S302, that is, the posture matrix corresponding to the historical posture information is subjected to matrix transformation processing according to the optimal transformation matrix representing the historical posture change information;

[0114] The historical position information refers to the position information of the cleaning robot when it last reached the historical position.

[0115] In combination with the above example, when the cleaning robot determines that it has reached position A for the second time, the cleaning robot processes the target point cloud data of the current frame and a frame of target point cloud data scanned when it reached position A the last time (i.e., the historical target point cloud data of the same position mentioned above) according to the ICP algorithm or other similar algorithms to obtain historical posture change information;

[0116] Then, the cleaning robot determines the posture information when the cleaning robot arrives at position A this time, that is, the current posture information, based on the historical posture change information and the historical posture information when the cleaning robot last arrived at position A.

[0117] After determining the current posture information in the above manner, the cleaning robot can determine the posture change information that characterizes the degree of change of the current posture information relative to the posture information at the previous moment in the manner of the aforementioned step S301, and then, reverse the process of determining the current posture information according to the posture information at the previous moment in step S302, and determine the optimized posture information at the previous moment based on the current posture information and the posture change information. Finally, the cleaning robot determines the optimized obstacle map data of the previous frame based on the optimized posture information at the previous moment and the target point cloud data of the previous frame obtained by scanning.

[0118] The above process can also continue to iterate forward, that is, the optimized posture information of the first two moments after optimization can be determined based on the optimized posture information of the first moment, and then the optimized first two frames of obstacle map data can be determined, and the optimized posture information of the first three moments after optimization can be determined based on the optimized posture information of the first two moments, and then the optimized first three frames of obstacle map data can be determined, and so on, so as to optimize the previously generated multiple frames of obstacle map data.

[0119] In combination with the above example, the cleaning robot passes through positions D, C, B, and A in sequence. After arriving at position A, it determines that position A is a historical position. Therefore, the current posture information of the cleaning robot when it arrives at position A this time is determined according to the above method. Then, based on a frame of target point cloud data scanned at position A this time and a frame of target point cloud data scanned at position B at the previous moment, the ICP algorithm or other similar algorithms are used to determine the degree of change of the cleaning robot's current posture at position A relative to the posture at the previous moment, that is, the posture change information of the cleaning robot when arriving at position A relative to the cleaning robot when arriving at position B is determined. Based on the posture change information and the current posture information, the new posture information of the cleaning robot at position B is determined, that is, the above-mentioned optimized posture information at the previous moment. Then, based on a frame of target point cloud data scanned at position B and the new posture information of the cleaning robot at position B, a new frame of obstacle map data corresponding to position B and the optimized previous frame of obstacle map data are determined.

[0120] Then, the cleaning robot can determine the new posture information of the cleaning robot at position C based on the new posture information of the cleaning robot at position B and the posture change of the cleaning robot at position B relative to the cleaning robot at position C, and then determine a new frame of obstacle map data corresponding to position C (that is, the first two frames of obstacle map data after the optimization mentioned above) in combination with a frame of target point cloud data scanned at position C. Similarly, the new posture information of the cleaning robot at position D and the new frame of obstacle map data corresponding to position D (that is, the first three frames of obstacle map data after the optimization mentioned above) can be determined based on the new posture information of the cleaning robot at position C.

[0121] The present application also provides a device for generating an obstacle map, see Figure 4 , the device can be applied to a cleaning robot, and the device can include the following units.

[0122] The processing unit 401 is used to process the first point cloud data and the second point cloud data to obtain target point cloud data of the current frame, where the target point cloud data of the current frame represents the distribution of obstacles around the cleaning robot, the first point cloud data is obtained by scanning with a laser radar of the cleaning robot, and the second point cloud data is obtained by scanning with a depth camera of the cleaning robot;

[0123] A determination unit 402 is used to determine current posture information of the cleaning robot according to the target point cloud data of the current frame, where the current posture information represents the position and posture of the cleaning robot when the target point cloud data of the current frame is obtained by scanning;

[0124] A generating unit 403, configured to generate obstacle map data of the current frame according to the current posture information and the target point cloud data of the current frame;

[0125] The obtaining unit 404 is used to process obstacle map data of multiple frames including obstacle map data of the current frame to obtain an obstacle map of the environment in which the cleaning robot is located.

[0126] The device for generating an obstacle map in this embodiment may be equivalent to one or more processors of the cleaning robot, and the above-mentioned units may be equivalent to corresponding functional modules included in the one or more processors.

[0127] Optionally, when the processing unit 401 processes the first point cloud data and the second point cloud data to obtain the target point cloud data of the current frame, it can be used to:

[0128] The first point cloud data and the second point cloud data are fused to obtain target point cloud data of the current frame.

[0129] Optionally, when the determination unit 402 determines the current posture information of the cleaning robot according to the target point cloud data of the current frame, it can be used to:

[0130] Determine the posture change information of the cleaning robot according to the target point cloud data of the current frame;

[0131] The current posture information of the cleaning robot is determined based on the posture change information and the posture information of the cleaning robot at the previous moment. The posture information at the previous moment represents the position and posture of the cleaning robot when the target point cloud data of the previous frame is scanned. The posture information of the cleaning robot when the first frame of target point cloud data is scanned is the preset initial posture information.

[0132] Optionally, when the determination unit 402 determines the posture change information of the cleaning robot according to the target point cloud data of the current frame, it can be used for at least one of the following:

[0133] Determine the position change information of the cleaning robot according to the target point cloud data of the current frame and the target point cloud data of the previous frame;

[0134] The initial map data is determined according to the target point cloud data of the current frame, and the posture change information of the cleaning robot is determined according to the initial map data and the obstacle map data of the previous frame.

[0135] Optionally, when the generating unit 403 generates obstacle map data of the current frame according to the current posture information and the target point cloud data of the current frame, it can be used to:

[0136] The extended Kalman filter algorithm is used to optimize the target point cloud data of the current frame according to the current posture information to obtain the obstacle map data of the current frame.

[0137] Optional, see Figure 5 The device may further include an optimization unit 501, configured to:

[0138] When it is determined that the cleaning robot has reached any historical position, the generated multi-frame obstacle map data is optimized according to the current posture information and historical posture information of the cleaning robot, where the historical posture information represents the position and posture of the cleaning robot when it was at the historical position last time.

[0139] Optionally, when the optimization unit 501 determines that the cleaning robot has reached any historical position, it can be used to:

[0140] Matching the obstacle map data of the current frame with the generated multiple frames of historical obstacle map data;

[0141] If the obstacle map data of the current frame successfully matches any frame of generated historical obstacle map data, it is determined that the cleaning robot has reached a historical position corresponding to the successfully matched historical obstacle map data.

[0142] The working principle of the device for generating an obstacle map provided in this embodiment can refer to the relevant steps in the method for generating an obstacle map provided in any embodiment of the present application, and will not be repeated here.

[0143] Another embodiment of the application also provides a cleaning robot, such as Figure 6 As shown, specifically including:

[0144] One or more processors 601 .

[0145] The memory 602 stores one or more programs.

[0146] LiDAR 603 and depth camera 604.

[0147] When one or more programs are executed by one or more processors 601, one or more processors 601 can control the laser radar 603 to scan and obtain the first point cloud data, and control the depth camera 604 to scan and obtain the second point cloud data, and then the one or more processors 601 can implement a method for generating an obstacle map such as any one of the above embodiments based on the first point cloud data and the second point cloud data.

[0148] Another embodiment of the present application further provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for generating an obstacle map as described in any one of the above embodiments is implemented.

[0149] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0150] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0151] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating an obstacle map, characterized in that: The method is applied to a cleaning robot, and the method comprises: Processing the first point cloud data and the second point cloud data to obtain target point cloud data of the current frame, wherein the target point cloud data of the current frame represents the distribution of obstacles around the cleaning robot, the first point cloud data is obtained by scanning with a laser radar of the cleaning robot, and the second point cloud data is obtained by scanning with a depth camera of the cleaning robot; Determine the current posture information of the cleaning robot according to the target point cloud data of the current frame, wherein the current posture information represents the position and posture of the cleaning robot when the target point cloud data of the current frame is obtained by scanning; Generate obstacle map data of the current frame according to the current posture information and the target point cloud data of the current frame; Obstacle map data of multiple frames including the obstacle map data of the current frame are processed to obtain an obstacle map of the environment in which the cleaning robot is located.

2. The method according to claim 1, characterized in that: The processing of the first point cloud data and the second point cloud data to obtain target point cloud data of the current frame includes: The first point cloud data and the second point cloud data are fused to obtain target point cloud data of the current frame.

3. The method according to claim 1, characterized in that Determining the current posture information of the cleaning robot according to the target point cloud data of the current frame includes: Determining the posture change information of the cleaning robot according to the target point cloud data of the current frame; The current posture information of the cleaning robot is determined based on the posture change information and the posture information of the cleaning robot at a previous moment. The posture information at the previous moment represents the position and posture of the cleaning robot when the target point cloud data of the previous frame is scanned and obtained. The posture information of the cleaning robot when the first frame of target point cloud data is scanned and obtained is the preset initial posture information.

4. The method according to claim 3, characterized in that The determining of the posture change information of the cleaning robot according to the target point cloud data of the current frame includes at least one of the following: Determining the posture change information of the cleaning robot according to the target point cloud data of the current frame and the target point cloud data of the previous frame; Initial map data is determined according to the target point cloud data of the current frame, and position change information of the cleaning robot is determined according to the initial map data and the obstacle map data of the previous frame.

5. The method according to claim 1, characterized in that Also includes: When it is determined that the cleaning robot has reached any historical position, the generated multi-frame obstacle map data is optimized according to the current posture information and historical posture information of the cleaning robot, wherein the historical posture information represents the position and posture of the cleaning robot when it was at the historical position last time.

6. The method according to claim 5, characterized in that The determining that the cleaning robot has arrived at any historical position includes: Matching the obstacle map data of the current frame with the generated multiple frames of historical obstacle map data; If the obstacle map data of the current frame successfully matches any frame of generated historical obstacle map data, it is determined that the cleaning robot has reached a historical position corresponding to the successfully matched historical obstacle map data.

7. The method according to claim 1, characterized in that The generating obstacle map data of the current frame according to the current posture information and the target point cloud data of the current frame comprises: The extended Kalman filter algorithm is used to optimize the target point cloud data of the current frame according to the current posture information to obtain the obstacle map data of the current frame.

8. A device for generating an obstacle map, characterized in that: The device is applied to a cleaning robot, and comprises: a processing unit, configured to process the first point cloud data and the second point cloud data to obtain target point cloud data of a current frame, wherein the target point cloud data of the current frame represents a distribution of obstacles around the cleaning robot, wherein the first point cloud data is obtained by scanning with a laser radar of the cleaning robot, and the second point cloud data is obtained by scanning with a depth camera of the cleaning robot; A determination unit, configured to determine current posture information of the cleaning robot according to the target point cloud data of the current frame, wherein the current posture information represents the position and posture of the cleaning robot when the target point cloud data of the current frame is obtained by scanning; A generating unit, configured to generate obstacle map data of a current frame according to the current posture information and the target point cloud data of the current frame; The obtaining unit is used to process obstacle map data of multiple frames including the obstacle map data of the current frame to obtain an obstacle map of the environment in which the cleaning robot is located.

9. A cleaning robot, characterized in that: include: one or more processors; a memory having one or more programs stored therein; LiDAR and depth cameras; When the one or more programs are executed by the one or more processors, the one or more processors control the lidar scan to obtain first point cloud data, control the depth camera scan to obtain second point cloud data, and implement the method for generating an obstacle map as described in any one of claims 1 to 7 based on the first point cloud data and the second point cloud data.

10. A computer storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the method for generating an obstacle map as claimed in any one of claims 1 to 7 is implemented.

Images