Robot mapping method and related device, robot and storage medium

Abstract

The present application discloses a robot mapping method and related devices, a robot, and a storage medium, wherein the robot mapping method includes: obtaining a captured image and radar data of the robot at the current point; wherein the radar data includes a plurality of measurement point data; aligning the captured image and the radar data, and identifying the image area of ​​each object in the captured image; based on the measurement point data in the radar data that are respectively aligned with the image area of ​​each object, determining the target grid occupied by each object in the grid map; in response to the presence of an obstruction in each object, determining the area to be explored that is blocked by the obstruction based on at least the current point and the target grid occupied by the obstruction in the grid map, and determining the exploration point to be reached by the robot based on the area to be explored. The above scheme can improve the accuracy of robot mapping.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a robot mapping method and related device, robot and storage medium. BACKGROUND

[0002] With the rapid development of electronic technology, robots have been increasingly applied to all aspects of daily life, industrial production, etc. For example, sweeping robots, companion robots, etc.

[0003] Before formally working, a robot usually needs to map first in order to execute formal instructions such as cleaning the ground. However, the existing robot usually has a large mapping error. In view of this, how to improve the mapping accuracy of the robot becomes a problem to be solved. SUMMARY

[0004] The technical problem solved by the present application is to provide a robot mapping method and related device, robot and storage medium, which can improve the mapping accuracy of the robot.

[0005] In order to solve the above technical problem, the first aspect of the present application provides a robot mapping method, comprising: acquiring a shooting image and radar data of a robot at a current point; wherein the radar data comprises a plurality of measurement point data; aligning the shooting image with the radar data, and identifying image regions of each object in the shooting image; determining target grids respectively occupied by each object in a grid map based on measurement point data in the radar data respectively aligned with the image regions of each object; in response to the existence of an occluder in each object, determining a to-be-explored region occluded by the occluder based on at least the current point and the target grids occupied by the occluder in the grid map, and determining an exploration point to be reached by the robot based on the to-be-explored region.

[0006] In order to solve the above technical problem, the second aspect of the present application provides a robot mapping device, comprising: a data acquisition module, an alignment and identification module, a grid determination module and a point determination module, the data acquisition module is used for acquiring a shooting image and radar data of a robot at a current point; wherein the radar data comprises a plurality of measurement point data; the alignment and identification module is used for aligning the shooting image with the radar data, and identifying image regions of each object in the shooting image; the grid determination module is used for determining target grids respectively occupied by each object in a grid map based on measurement point data in the radar data respectively aligned with the image regions of each object; the point determination module is used for determining a to-be-explored region occluded by an occluder based on at least the current point and the target grids occupied by the occluder in the grid map in response to the existence of the occluder in each object, and determining an exploration point to be reached by the robot based on the to-be-explored region.

[0007] To solve the above technical problems, the third aspect of the present application provides an electronic device comprising at least a memory and a processor coupled to each other, the memory at least storing program instructions, and the processor being configured to execute the program instructions to implement the robot mapping method in the first aspect.

[0008] To solve the above technical problems, the fourth aspect of the present application provides a robot comprising at least a camera device, a radar device, a motion device and a control device, the camera device, the radar device and the motion device being electrically connected to the control device, and the control device being the electronic device in the third aspect.

[0009] To solve the above technical problems, the fifth aspect of the present application provides a computer-readable storage medium storing program instructions capable of being executed by a processor, the program instructions being configured to implement the robot mapping method in the first aspect.

[0010] The above scheme acquires the photographed image and the radar data of the robot at the current point, and the radar data comprises a plurality of measurement point data. Then, the photographed image and the radar data are aligned, and the image regions of each object in the photographed image are identified. Thus, based on the measurement point data in the radar data respectively aligned with the image regions of each object, the target grid occupied by each object in the grid map is determined. Then, in response to the existence of the occlusion in each object, the to-be-explored region occluded by the occlusion is determined based on at least the current point and the target grid occupied by the occlusion in the grid map, and the exploration point to which the robot is to be moved is determined based on the to-be-explored region. Therefore, on the one hand, the mapping can be performed by combining the image data and the radar data, which helps to improve the mapping accuracy to a certain extent. On the other hand, in the mapping process, it is also detected whether there is an occlusion in each object detected, and when there is an occlusion, the to-be-explored region occluded by the occlusion is further determined based on the current detection result, so as to determine the exploration point to which the robot is to be moved. This helps to supplement the mapping of the occluded region and can avoid the mapping loss as much as possible. Therefore, the mapping accuracy of the robot can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a flowchart of an embodiment of the robot mapping method of the present application;

[0012] Figure 2a is a schematic diagram of an embodiment of determining the to-be-explored region;

[0013] Figure 2b is a schematic diagram of an embodiment of determining the exploration point when the category of the occlusion is furniture;

[0014] Figure 3 is a schematic diagram of an embodiment of the robot mapping device of the present application;

[0015] Figure 4is a frame schematic diagram of an embodiment of the electronic device of the present application;

[0016] Figure 5 is a frame schematic diagram of an embodiment of the robot of the present application;

[0017] Figure 6 is a frame schematic diagram of an embodiment of the computer readable storage medium of the present application. DETAILED DESCRIPTION

[0018] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0019] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. The present application may, however, be practiced without these details. In other instances, well-known methods have not been described in detail in order not to unnecessarily obscure the present application.

[0020] The terms "system" and "network" are often used interchangeably herein. The term "and / or" herein is merely used to represent an associated relationship between associated objects, and can represent three cases, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. In addition, the segment " / " herein generally represents that the associated objects before and after the " / " are in an "or" relationship. In addition, "multiple" herein represents two or more than two.

[0021] Please refer to Figure 1 , Figure 1 is a flow schematic diagram of an embodiment of the robot mapping method of the present application. Specifically, it can include the following steps:

[0022] Step S11: Obtain the shooting image and radar data of the robot at the current point.

[0023] In the embodiments of the present disclosure, the radar data includes a plurality of measurement point data. It should be noted that the radar rays are reflected on the surface of the object and received by the radar to form the measurement point data. The measurement point data includes the relative angle and relative distance between the reflection point on the surface of the object and the radar, and the like, and the spatial position of the reflection point on the surface of the object can be located accordingly. Of course, the above description is only a principle description of the measurement point data, and the specific process of obtaining the measurement point data can be referred to the technical details of the radar, which will not be described here. In addition, as a possible example, the robot can determine the measurement point positions for mapping by using a preset strategy during the mapping process. When the robot performs environment detection at a certain measurement point position, the measurement point position is used as the current point position. For example, a measurement point position can be determined every time a certain distance is traveled, or a measurement point position can be determined after traveling a certain distance along the edge every time after trying to travel close to the edge of the space to be mapped. Of course, the above examples are only a few possible examples of the preset strategy for determining the measurement point positions by the robot, and other possible ways of the preset strategy are not limited by this. The specific ways of the preset strategy will not be exemplified here.

[0024] Step S12: aligning the photographed image and the radar data, and identifying the image area of each object in the photographed image.

[0025] In an implementation scenario, before the alignment and identification, the photographed image and the radar data can be first de-distorted. It should be noted that the de-distortion of the photographed image can be implemented in combination with the camera distortion coefficient and the like, and the specific implementation can be referred to the technical details of the camera distortion correction. The de-distortion of the radar data can be implemented in combination with the pose information provided by the laser radar odometer and the like, and the specific implementation can be referred to the technical details of the radar distortion correction, which will not be described here.

[0026] In an implementation scenario, the camera and the radar on the robot can be pre-calibrated to obtain the external parameters (such as the coordinate system conversion parameters) between the camera and the radar, and to obtain the internal parameters of the camera and the radar. On this basis, after the photographed image and the radar data are obtained, the photographed image and the radar data can be aligned in combination with the internal parameters and the external parameters, and then the correspondence between the pixel points in the photographed image and the measurement point data can be established. That is, a certain pixel point in the photographed image and the measurement point data are both sensed from a certain three-dimensional point in the physical space, and the above correspondence exists between them. It should be noted that the specific process of aligning the photographed image and the radar data can be referred to the technical details of the data alignment, which will not be described here.

[0027] In one implementation scenario, in order to identify the image region of each object in the photographed image, a target detection algorithm such as FasterRCNN can be used to perform target detection on the photographed image to obtain the image region (such as a rectangular box bbox) of each object in the photographed image. It should be noted that each object can include but is not limited to: furniture, building components, and the like, without limitation. In addition, in the target detection process, not only the image region of each object can be output, but also the respective category of each object, such as furniture such as bed, table, etc., or building components such as wall, partition, etc.

[0028] In another implementation scenario, different from the foregoing implementation, in order to identify the image region of each object in the photographed image, an image segmentation algorithm such as U-Net can be used to perform image segmentation on the photographed image to obtain the image region (such as a contour mask) of each object in the photographed image. The respective category of each object can be referred to the foregoing related description, which will not be repeated here. In addition, in the image segmentation process, not only the image region of each object can be output, but also the respective category of each object, such as furniture such as bed, table, etc., or building components such as wall, partition, etc.

[0029] In yet another implementation scenario, different from the foregoing implementation, in order to identify the image region of each object in the photographed image, target detection can also be performed on the photographed image, and image segmentation can also be performed on the photographed image. Based on this, the image region of each object can be obtained by combining the detection result of target detection and the segmentation result of image segmentation. Exemplarily, if the detection result of target detection does not detect the image region of any object at a certain position, but the segmentation result of image segmentation segments the image region of a certain object at the same position, then it can be finally determined that the image region of a certain object is identified at the position. In addition, as described before, in the process of target detection and image segmentation, the respective category of each object can be output, so the final category of the object at the same position can be determined by combining the object category obtained by target detection and the image category obtained by image segmentation. Exemplarily, if the object category obtained by target detection at a certain position is furniture, and the object category obtained by image segmentation is unknown, then the final object category at the position can be determined as furniture. Of course, the above examples are only a few possible examples of the combination of the segmentation result of image segmentation and the detection result of target detection, and do not limit other possible combination modes of the segmentation result of image segmentation and the detection result of target detection. Here, they will not be exemplified one by one.

[0030] Step S13: determining the target grid respectively occupied by each object in the grid map based on the measurement point data respectively aligned with the image region of each object in the radar data.

[0031] Specifically, the measurement point data respectively aligned with the image regions of the objects in the radar data can be acquired as the target data, and the target data can include at least one of the relative angle and the relative distance with the current point. For example, if an image region of an object contains N pixel points, the measurement point data corresponding to the N pixel points can be obtained according to the correspondence established after the data alignment, and then the measurement point data can be used as the target data of the object. On this basis, the target position corresponding to the target data and the current point in the grid map can be used to determine the target grid occupied by each object in the grid map. For example, if the target data includes the relative angle and the relative distance, for any target data of an object, the target position in the grid map can be used as the reference point, a straight line passing through the reference point and having the same orientation as the robot at the current point can be used as the reference line, a straight line passing through the reference point and deviating from the reference line by the relative angle in the target data can be used as the deviation line, and the reference point on the deviation line can be used as the starting point to deviate from the detection direction of the robot at the current point by the relative distance in the target data, and then the grid where the position is located is the target grid occupied by the object in the grid map. The above method, which acquires the measurement point data respectively aligned with the image regions of the objects in the radar data as the target data, and the target data includes at least one of the relative angle and the relative distance with the current point, and then determines the target grid occupied by each object in the grid map based on the target position corresponding to the target data and the current point in the grid map, can improve the accuracy of determining the target grid occupied by each object in the grid map.

[0032] Step S14: In response to the existence of the occlusion in the objects, the to-be-explored region occluded by the occlusion is determined based on at least the current point and the target grid occupied by the occlusion in the grid map, and the exploration point to be reached by the robot is determined based on the to-be-explored region.

[0033] In one implementation scenario, for any object, whether it is an occlusion object can be determined in combination with the image region of the object in the captured image. Illustratively, whether it is an occlusion object can be determined according to factors such as the area occupied by the image region in the captured image, the proportion of the length of the image region in the captured image, the proportion of the width of the image region in the captured image, etc. For example, through the foregoing image recognition, it can be recognized that the image region of the furniture object "bed" in the captured image occupies 90% of the length of the captured image, or 85% of the area of the captured image, or 80% of the width of the captured image, and it can be determined that the furniture object "bed" in the captured image is an occlusion object. It should be noted that the foregoing example is only one possible example in actual application, and does not limit the discrimination threshold of the area, length, width, etc. in determining whether it is an occlusion object, nor does it limit the specific category of the occlusion object (such as furniture object or building component), and other possible cases will not be exemplified here.

[0034] In another implementation scenario, for any object, whether it is an occlusion object can be determined in combination with the radar data. Illustratively, the measurement point data can also include the time of arrival (characterizing the time difference from the transmission of the signal to the reception of the signal), and whether it is an occlusion object can be determined according to the time of arrival in each measurement point data. For example, the time of arrival in the measurement point data of a large proportion of rays successively starting from a certain ray suddenly decreases, and these measurement point data all belong to the same object, and it can be considered that the object is an occlusion object.

[0035] In yet another implementation scenario, for any object, whether it is an occlusion object can be determined in combination with the captured image and the radar data, so as to complement each other according to whether the object is an occlusion object determined according to the captured image and whether the object is an occlusion object determined according to the radar data. It should be noted that the specific manner of complementing each other can refer to the foregoing specific manner of determining the image region in combination with target detection and image segmentation, and will not be described here.

[0036] In one implementation scenario, in the case where there is no occlusion object in each object, the next point can be moved to continue the foregoing step S11 to continue to perfect the grid map, and the process is repeated until the mapping is completed. It should be noted that the determination manner of the next point can refer to the foregoing description of the preset strategy, and will not be described here.

[0037] In one implementation scenario, in the case where there is an occlusion object in each object, the to-be-explored region occluded by the occlusion object can be determined first. Please refer to Figure 2a , Figure 2a is a schematic view of one embodiment of determining the to-be-explored region. As Figure 2aAs shown, specifically, the target position corresponding to the current point in the grid map can be acquired first (such as Al), and a connected domain formed by the target grid occupied by the occlusion in the grid map (such as the grid filled with diagonal shading) is acquired as a target region, then a target ray is formed by connecting the end of the target region from the target position (such as ray AlC and ray AlF), and finally, the to-be-explored region (such as trapezoid BCDE) can be determined based on at least the target ray, a first edge line of the target region (such as BE), and a second edge line of the grid map (such as CD and DE) surrounding the region. Of course, Figure 2a The to-be-explored region shown is only one possible example in the actual application process, and does not limit the specific shape and specific position of the to-be-explored region. The above method, acquiring the target position corresponding to the current point in the grid map, acquiring the connected domain formed by the target grid occupied by the occlusion in the grid map as the target region, connecting the end of the target region from the target position to form the target ray, and determining the to-be-explored region based on at least the target ray, the first edge line of the target region, and the second edge line of the grid map surrounding the region, can determine the to-be-explored region only through point-line operation, which helps to reduce the complexity of determining the to-be-explored region as much as possible while ensuring the accuracy of the to-be-explored region as much as possible.

[0038] In a specific implementation scenario, during the mapping of the robot, the real-time position of the robot in the grid map can be determined according to the pose information and other parameters of the robot. For details, refer to technologies such as SLAM (Simultaneous Localization And Mapping, real-time positioning and mapping), which will not be described here.

[0039] In a specific implementation scenario, as one possible example, the region surrounded by the target ray, the first edge line, and the second edge line can be directly used as the to-be-explored region. Alternatively, as another possible example, the region surrounded by the target ray, the first edge line, and the second edge line can be acquired as a candidate region, and then the connected domain formed by the unoccupied grid in the candidate region can be obtained as the to-be-explored region of the robot, that is, the connected domain formed by the unoccupied grid in the candidate region can be specifically selected as the to-be-explored region. It should be noted that if the grid is not occupied, it means that the grid position is more likely to have not been explored, or although it has been explored, no actual contact has been established with the object. The above method, after acquiring the region surrounded by the target ray, the first edge line, and the second edge line as a candidate region, and then using the connected domain formed by the unoccupied grid in the candidate region as the to-be-explored region, helps to further improve the accuracy of the to-be-explored region.

[0040] In one implementation scenario, please continue to refer to Figure 2aAfter the region to be explored is determined, a third edge line (such as BC) other than the first edge line and the second edge line in the region to be explored can be obtained, and a point (such as A2) to be explored can be determined on a side of the third edge line away from the region to be explored. That is, the third edge line does not coincide with any of the map edge of the grid map or the object edge of the obstacle. In the above manner, the point to be explored is determined on the side of the third edge line away from the region to be explored, which helps the robot to cover the region to be explored as comprehensively as possible after the robot reaches the point to be explored, and thus the region to be explored in the grid map can be perfected as efficiently as possible.

[0041] In one specific implementation scenario, as one possible implementation, please refer to Figure 2a , specifically, a point on a median line GA2 of the third edge line (such as BC) and outside the region to be explored can be selected as the point to be explored (such as A2), and the distance between the point and the third edge line is a preset distance. It should be noted that the preset distance can be determined according to the camera parameters (such as the field of view angle) of the robot. Taking the determination according to the field of view angle as an example, the smaller the field of view angle, the larger the preset distance can be set, so that the field of view angle of the robot can cover the region to be explored as comprehensively as possible. Conversely, the larger the field of view angle, the smaller the preset distance can be set, so that the field of view angle of the robot can cover the region to be explored as comprehensively as possible, and the region to be explored can be imaged as close as possible to improve the imaging quality.

[0042] In one specific implementation scenario, as another possible implementation, after the median line is determined, the shortest path (such as the perpendicular line from the target position to the median line when there is no obstruction) from the target position of the robot to the median line can be determined, and the robot can move along the shortest path to the median line, and determine whether the field of view angle of the robot at this position can cover the region to be explored. If yes, this position can be directly determined as the point to be explored. Otherwise, the robot can continue to move away from the region to be explored along the median line, and the determination is continuously made until the field of view angle of the robot can cover the region to be explored, and then the position at this time is determined as the point to be explored.

[0043] In one implementation scenario, as a possible implementation, after determining an exploration point, the robot can proceed to the exploration point and use it as the new current point. In response to the obstruction being classified as a building component (such as a wall), the robot returns to the step of acquiring the robot's captured imagery and radar data at the current point, continuing until the step of determining the target grid cells occupied by each object in the grid map based on the measured point data in the radar data that aligns with the image area of ​​each object. That is, if the obstruction is classified as a building component such as a wall, the robot can use the captured imagery and radar data at the exploration point to complete the unexplored area in the grid map according to the aforementioned description, and then continue to advance to the next point according to a preset strategy. In this manner, if the obstruction is classified as a building component, the robot re-executes the aforementioned mapping process based on the captured imagery and radar data at the exploration point to complete the unexplored area in the grid map, thereby improving the completeness of the robot's mapping.

[0044] In another implementation scenario, as another possible implementation method, after determining the exploration point, the robot can move to the exploration point and use the exploration point as the new current point. Different from the above implementation method, in response to the category of the obstruction being a furniture object, the robot returns to the step of obtaining the captured image and radar data of the robot at the current point and iterates until all perspectives of the furniture object to which the obstruction belongs are detected by the robot at all the exploration points it has traveled to. Please refer to Figure 2a and Figure 2b , Figure 2b FIG. 1 is a schematic diagram of an embodiment of determining exploration points when the category of the obstructing object is furniture. Figure 2a For example, if the occluder is a furniture object, the robot moves to the exploration point A2 and takes the exploration point A2 as the new current point. In response to the category of the occluder being a furniture object, the robot can return to the aforementioned step S11 and iterate, that is, the grid map can be improved (such as Figure 2b After that, a new area to be explored is determined according to the above method (such as Figure 2b and determine a new exploration point according to the above method (such as Figure 2b A3 in the figure), then move to the exploration point A3, and use the exploration point A3 as the new current point. In response to the category of the occluder being furniture, return to the aforementioned step S11 and iterate, so that the grid map can be improved again (such as Figure 2b A grid filled with grid shadows, and its corresponding object belongs to the category of furniture objects). At this time, since the furniture objects ( Figure 2b ) of various viewing angles (shown as a shaded grid in Figure 2bThe lower middle view, right view and upper view (except the left view because it is blocked) are all obtained by the robot at each exploration point it has reached (such as Figure 2b The exploration points A1, A2, and A3 are explored, so the iteration can be terminated. For example, at this time, the next point can be moved to according to the above preset strategy. It should be noted that Figure 2b For the specific meanings of various styles of lines, please refer to Figure 2a For example, the unbold dashed line starting from the exploration point represents the target ray, the bold dashed line represents the first edge line, the bold solid line represents the second edge line, and the black dot represents the exploration point. Of course, Figure 2b The example shown here is just one possible example of multi-view mapping when the occluder is a furniture object. Other possible scenarios will not be listed here one by one. The above method is required. When the occluder is a furniture object, exploration points are continuously selected around the furniture object to iteratively execute the aforementioned mapping process to improve the furniture objects in the to-be-explored area of ​​the grid map from multiple perspectives. On the one hand, this can improve the accuracy and completeness of the robot's mapping of the furniture objects, and on the other hand, it does not lead to a significant increase in mapping time. Therefore, the accuracy, completeness, and efficiency of the robot's mapping can be improved at the same time.

[0045] In one implementation scenario, after the map is created using the aforementioned method, the robot can also execute the work instruction based on the grid map in response to receiving a work instruction. For example, the robot can be a cleaning robot, and the work instruction can be a cleaning instruction. Based on the grid map, the robot can accurately clean around and under furniture items such as tables and beds. Alternatively, the robot can be a companion robot, and the work instruction can be a companion instruction. Based on the grid map, the robot can accurately accompany a target person. Alternatively, the robot can be a pet robot, and the work instruction can be a patrol instruction. Based on the grid map, the robot can accurately patrol a desired indoor location (such as a balcony). Of course, the above examples are only a few possible examples of how the robot can execute work instructions in actual applications, and other possible scenarios will not be listed here. In this approach, after the grid map is created, based on the accurate and complete grid map created above, the robot can execute the work instruction based on the work instruction, which helps improve the robot's work efficiency.

[0046] The above scheme acquires the photographed image and the radar data of the robot at the current point, and the radar data includes a plurality of measurement point data. The photographed image and the radar data are aligned, and the image area of each object in the photographed image is identified. The target grid occupied by each object in the grid map is determined based on the measurement point data in the radar data that is aligned with the image area of each object. In response to the existence of an occlusion object in each object, the to-be-explored area occluded by the occlusion object is determined based on at least the current point and the target grid occupied by the occlusion object in the grid map. The exploration point to which the robot is to be moved is determined based on the to-be-explored area. On the one hand, the scheme can combine image data and radar data to perform mapping, which helps to improve the mapping accuracy to a certain extent. On the other hand, the scheme also detects whether there is an occlusion object in each object detected during mapping, and further determines the to-be-explored area occluded by the occlusion object based on the current detection result when the occlusion object exists. The exploration point to which the robot is to be moved is determined based on the to-be-explored area, which helps to supplement the mapping of the occluded area and avoids mapping loss as much as possible. Therefore, the mapping accuracy of the robot can be improved.

[0047] Please refer to Figure 3 , Figure 3 is a schematic diagram of the framework of an embodiment of the robot mapping device. The robot mapping device 30 includes a data acquisition module 31, an alignment and identification module 32, grid determination modules 33 and 34, the data acquisition module 31 is configured to acquire a photographed image and radar data of a robot at a current point; the radar data includes a plurality of measurement point data; the alignment and identification module 32 is configured to align the photographed image and the radar data, and identify the image area of each object in the photographed image; the grid determination module 33 is configured to determine the target grid occupied by each object in a grid map based on the measurement point data in the radar data that is aligned with the image area of each object; and the point determination module 34 is configured to determine the to-be-explored area occluded by the occlusion object based on at least the current point and the target grid occupied by the occlusion object in the grid map in response to the existence of the occlusion object in each object, and determine the exploration point to which the robot is to be moved based on the to-be-explored area.

[0048] The above scheme, the robot mapping device 30 acquires the shooting image and the radar data of the robot at the current point, and the radar data includes a plurality of measurement point data, and then aligns the shooting image and the radar data, and identifies the image area of each object in the shooting image, so as to determine the target grid occupied by each object in the grid map based on the measurement point data in the radar data which is aligned with the image area of each object respectively, and then in response to the existence of the shelter in each object, at least based on the current point and the target grid occupied by the shelter in the grid map, the to-be-explored area blocked by the shelter is determined, and based on the to-be-explored area, the exploration point to be reached by the robot is determined. On the one hand, the mapping can be performed in combination with image and radar data, which helps to improve the mapping accuracy to a certain extent. On the other hand, in the mapping process, it is also detected whether there is a shelter in each object detected, and when there is a shelter, the to-be-explored area blocked by the shelter is further determined in combination with the current detection result, so as to determine the exploration point to be reached by the robot, which helps to supplement the mapping of the blocked area and can avoid mapping loss as much as possible. Therefore, the mapping accuracy of the robot can be improved.

[0049] In some disclosed embodiments, the point position determination module 34 includes an acquisition sub-module for acquiring a target position corresponding to the current point in the grid map, and acquiring a connected domain formed by the target grid occupied by the shelter in the grid map as a target area; the point position determination module 34 includes a connection sub-module for connecting the end of the target area with the target position as a starting point to form a target ray; the point position determination module 34 includes a determination sub-module for determining the to-be-explored area based on at least the target ray, a first edge line of the target area, and a second edge line of the grid map.

[0050] In some disclosed embodiments, the determination sub-module includes a candidate area acquisition unit for acquiring the area enclosed by the target ray, the first edge line, and the second edge line as a candidate area; the determination sub-module includes an exploration area determination unit for obtaining the to-be-explored area of the robot based on the connected domain formed by the unoccupied grid in the candidate area.

[0051] In some disclosed embodiments, the point position determination module 34 includes an edge sub-module for acquiring a third edge line outside the first edge line and the second edge line in the to-be-explored area; the point position determination module 34 includes a point position sub-module for determining the exploration point on the side of the third edge line away from the to-be-explored area.

[0052] In some disclosed embodiments, the point position sub-module is specifically configured to select a point position on the median line of the third edge line, which is a preset distance away from the third edge line and is located outside the to-be-explored area, as the exploration point.

[0053] In some disclosed embodiments, the robot mapping device 30 further comprises an exploration travel module for traveling to an exploration point and taking the exploration point as a new current point; the robot mapping device 30 further comprises a first loop module for returning to the steps of acquiring the photographed image and the radar data of the robot at the current point and iterating until each perspective of the furniture object to which the occlusion belongs is detected by the robot at each exploration point that has been traveled to, in response to the category to which the occlusion belongs being furniture.

[0054] In some disclosed embodiments, the robot mapping device 30 further comprises an exploration travel module for traveling to an exploration point and taking the exploration point as a new current point; the robot mapping device 30 further comprises a second loop module for returning to the step of acquiring the photographed image and the radar data of the robot at the current point until the step of determining the target grid occupied by each object in the grid map respectively based on the measurement point data in the radar data respectively aligned with the image region of each object, in response to the category to which the occlusion belongs being a building component.

[0055] In some disclosed embodiments, the grid determination module 33 comprises a measurement point acquisition submodule for acquiring measurement point data in the radar data respectively aligned with the image region of each object as target data; wherein the target data comprises at least one of a relative angle and a relative distance of the current point; the grid determination module 33 comprises a grid determination submodule for determining the target grid occupied by each object in the grid map respectively based on the target data and the target position corresponding to the current point in the grid map.

[0056] In some disclosed embodiments, the robot mapping device 30 further comprises an instruction execution module for executing a work instruction based on the grid map in response to receiving the work instruction after the grid map is mapped.

[0057] Please refer to Figure 4 , Figure 4 is a schematic diagram of the framework of an embodiment of the electronic device. The electronic device 40 at least comprises a memory 41 and a processor 42 coupled to each other, the memory 41 at least stores program instructions, and the processor 42 is configured to execute the program instructions to implement the steps in any of the above robot mapping method embodiments. The robot mapping method can refer to the foregoing disclosed embodiments for details, which will not be described here.

[0058] In particular, the processor 42 is configured to control itself and the memory 41 to implement the steps in any of the above robot mapping method embodiments. The processor 42 can also be referred to as a CPU (Central Processing Unit). The processor 42 can be an integrated circuit chip having a processing capability of signals. The processor 42 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like. In addition, the processor 42 can be implemented by an integrated circuit chip together.

[0059] The above scheme, the electronic device 40 acquires the photographed image of the robot at the current point and the radar data, and the radar data includes a plurality of measurement point data, then aligns the photographed image and the radar data, and identifies the image area of each object in the photographed image, so as to determine the target grid occupied by each object in the grid map respectively based on the measurement point data in the radar data respectively aligned with the image area of each object, and then in response to the existence of the occlusion in each object, at least based on the current point and the target grid occupied by the occlusion in the grid map, determine the to-be-explored area occluded by the occlusion, and based on the to-be-explored area, determine the exploration point to be reached by the robot. Therefore, on the one hand, it can combine image and radar data for mapping, which helps to improve the mapping accuracy to a certain extent. On the other hand, in the mapping process, it also detects whether there is an occlusion in each object detected, and when there is an occlusion, it further combines the current detection result to determine the to-be-explored area occluded by the occlusion, so as to determine the exploration point to be reached by the robot, which helps to supplement the mapping of the occluded area and can avoid mapping missing as much as possible. Therefore, the robot mapping accuracy can be improved.

[0060] Please refer to Figure 5 , Figure 5is a frame diagram of an embodiment of the robot of the present application. The robot 50 at least includes a camera device 51, a radar device 52, a motion device 53 and a control device 54, the camera device 51, the radar device 52 and the motion device 53 are respectively electrically connected to the control device 54, and the control device 54 is the electronic device in the above-mentioned electronic device embodiment, which can be specifically referred to the foregoing disclosed embodiments and will not be described here. It should be noted that the camera device 51 is used to collect the photographed image, the radar device 52 is used to collect the radar data, and the motion device 53 is used to drive the whole robot 50 to move. In addition, the robot 50 can specifically include but is not limited to a cleaning robot such as a sweeping robot, a sweeping and mopping robot and a mopping robot, and can also include but is not limited to a companion robot and a pet robot, and the specific type of the robot 50 is not limited here.

[0061] The above-mentioned scheme, the robot 50 at least includes a camera device 51, a radar device 52, a motion device 53 and a control device 54, the camera device 51, the radar device 52 and the motion device 53 are respectively electrically connected to the control device 54, and the control device 54 is the electronic device in the above-mentioned electronic device embodiment, which can be specifically referred to the foregoing disclosed embodiments and will not be described here. It should be noted that the camera device 51 is used to collect the photographed image, the radar device 52 is used to collect the radar data, and the motion device 53 is used to drive the whole robot 50 to move. In addition, the robot 50 can specifically include but is not limited to a cleaning robot such as a sweeping robot, a sweeping and mopping robot and a mopping robot, and can also include but is not limited to a companion robot and a pet robot, and the specific type of the robot 50 is not limited here.

[0062] Please refer to Figure 6 , Figure 6 is a frame diagram of an embodiment of the computer readable storage medium 60 of the present application. The computer readable storage medium 60 stores program instructions 61 capable of being run by a processor, and the program instructions 61 are used to implement the steps in any of the above-mentioned robot mapping method embodiments.

[0063] The above scheme, the computer readable storage medium 60 acquires the shooting image of the robot at the current point and the radar data, and the radar data includes a plurality of measurement point data, and then aligns the shooting image and the radar data, and identifies the image area of each object in the shooting image, so as to determine the target grid occupied by each object in the grid map based on the measurement point data in the radar data respectively aligned with the image area of each object, and then in response to the existence of the shelter in each object, at least based on the current point and the target grid occupied by the shelter in the grid map, the to-be-explored area blocked by the shelter is determined, and based on the to-be-explored area, the exploration point to be reached by the robot is determined. On the one hand, the image and radar data can be combined for mapping, which helps to improve the mapping accuracy to a certain extent. On the other hand, in the mapping process, it is also detected whether there is a shelter in each object detected, and when there is a shelter, the to-be-explored area blocked by the shelter is further determined based on the current detection result, so as to determine the exploration point to be reached by the robot, which helps to supplement the mapping of the blocked area and can avoid mapping loss as much as possible. Therefore, the mapping accuracy of the robot can be improved.

[0064] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be repeated here.

[0065] The above description of each embodiment tends to emphasize the differences between each embodiment, and the same or similar parts can be mutually referred to. For brevity, it will not be repeated here.

[0066] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the above-described device implementation is only schematic. For example, the division of the module or unit is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other form.

[0067] The unit described as a separate component can or can not be physically separated, and the component shown as a unit can or can not be a physical unit, that is, it can be located in one place, or it can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the present embodiment scheme.

[0068] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0069] If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0070] If the technical solutions of the present application involve personal information, the product applying the technical solutions of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solutions of the present application involve sensitive personal information, the product applying the technical solutions of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as a camera, a clear and conspicuous sign is set to inform that the personal information collection range has been entered, and the personal information will be collected. If the individual voluntarily enters the collection range, it is considered to agree to collect the personal information. Or, on the device for processing personal information, the individual is informed of the personal information processing rules by using obvious signs / information, and the individual is authorized by means of pop-up information or by uploading the personal information by the individual himself. The personal information processing rules can include personal information processor, processing purpose, processing method, and processing personal information type, etc.

Claims

1. A robot mapping method, characterized in that: include: Obtaining the captured image and radar data of the robot at the current position; wherein the radar data includes data of several measurement points; aligning the captured image with the radar data, and identifying image regions of respective objects in the captured image; Determining target grids occupied by the respective objects in the grid map based on measurement point data in the radar data that are respectively aligned with image areas of the respective objects; In response to the presence of an occluder in each of the objects, determining, based at least on the current point and a target grid occupied by the occluder in the grid map, an area to be explored that is blocked by the occluder, and determining, based on the area to be explored, an exploration point to be reached by the robot; The step of determining the area to be explored that is blocked by the occluder based at least on the current point and the target grid occupied by the occluder in the grid map includes: Obtaining a target position in the grid map corresponding to the current point position, and obtaining a connected domain formed by the target grid occupied by the occluder in the grid map as a target area; Connecting the ends of the target area with the target position as the starting point to form a target ray; The area to be explored is determined based on at least an area enclosed by the target ray, a first edge line of the target area, and a second edge line of the grid map.

2. The method according to claim 1, characterized in that The determining the area to be explored based at least on an area enclosed by the target ray, the first edge line of the target area, and the second edge line of the grid map includes: Acquire an area enclosed by the target ray, the first edge line, and the second edge line as a candidate area; Based on the connected domain formed by the unoccupied grids in the candidate area, the area to be explored by the robot is obtained.

3. The method according to claim 1, characterized in that The step of determining the exploration point to be reached by the robot based on the area to be explored includes: Acquire a third edge line in the to-be-explored area; wherein the third edge line does not overlap with any of a map edge of the grid map and an object edge of the obstruction; The exploration point is determined on a side of the third edge line away from the area to be explored.

4. The method according to claim 3, characterized in that Determining the exploration point on a side of the third edge line away from the to-be-explored area includes: A point on the perpendicular midline of the third edge line that is a preset distance away from the third edge line and is located outside the area to be explored is selected as the exploration point.

5. The method according to claim 1, wherein After determining the exploration point to be reached by the robot based on the area to be explored, the method further includes: Traveling to the exploration point and setting the exploration point as the new current point; In response to the category of the obstruction being a furniture object, return to the step of obtaining the robot's captured images and radar data at the current point and iterate until all perspectives of the furniture object to which the obstruction belongs are detected by the robot at all the exploration points it has reached.

6. The method according to claim 1, characterized in that After determining the exploration point to be reached by the robot based on the area to be explored, the method further includes: Traveling to the exploration point and setting the exploration point as the new current point; In response to the category of the obstruction being a building component, the method returns to the step of obtaining the captured image and radar data of the robot at the current point, until the step of determining the target grids occupied by each object in the grid map based on the measurement point data in the radar data that are respectively aligned with the image areas of each object is completed.

7. The method according to claim 1, characterized in that The determining, based on the measurement point data in the radar data that are respectively aligned with the image areas of the respective objects, the target grids occupied by the respective objects in the grid map comprises: Acquire measurement point data from the radar data that are respectively aligned with the image areas of the respective objects as target data; wherein the target data includes at least one of a relative angle and a relative distance from the current point position; Based on the target data and the current point position corresponding to the target position in the grid map, the target grids occupied by the respective objects in the grid map are determined.

8. The method according to any one of claims 1 to 7, characterized in that After the grid map is constructed, the method further includes: In response to receiving a work order, the work order is executed based on the grid map.

9. A robot mapping device, characterized in that: include: A data acquisition module is used to acquire the robot's captured images and radar data at the current point; wherein the radar data includes data of several measurement points; an alignment and recognition module, configured to align the captured image with the radar data and recognize image regions of various objects in the captured image; a grid determination module, configured to determine target grids occupied by each object in the grid map based on measurement point data in the radar data that are respectively aligned with the image areas of each object; a point determination module for determining, in response to the presence of an occluder in each of the objects, an area to be explored that is blocked by the occluder based at least on the current point and a target grid occupied by the occluder in the grid map, and determining an exploration point to be reached by the robot based on the area to be explored; The step of determining the area to be explored that is blocked by the occluder based at least on the current point and the target grid occupied by the occluder in the grid map includes: Obtaining a target position in the grid map corresponding to the current point position, and obtaining a connected domain formed by the target grid occupied by the occluder in the grid map as a target area; Connecting the ends of the target area with the target position as the starting point to form a target ray; The area to be explored is determined based on at least an area enclosed by the target ray, a first edge line of the target area, and a second edge line of the grid map.

10. An electronic device, characterized in that: The robot mapping method comprises at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the robot mapping method according to any one of claims 1 to 8.

11. A robot, characterized in that: The electronic device comprises at least a camera device, a radar device, a motion device and a control device, wherein the camera device, the radar device and the motion device are electrically connected to the control device respectively, and the control device is the electronic device according to claim 10.

12. A computer-readable storage medium, characterized in that Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the robot mapping method according to any one of claims 1 to 8.

Images