Step recognition method, mobile robot, cleaning system and storage medium
By projecting point cloud data onto the target map, identifying the position and height of the step, solving the problem that mobile robots cannot actively cross low steps, and improving work efficiency.
Patent Information
- Application Number
- CN202411060101.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-08
AI Technical Summary
Mobile robots ignore low steps in their marching routes, resulting in the inability to actively cross, affecting work efficiency.
By obtaining point cloud data, projecting it into the target map, determining the position and height of the steps, updating the information of the map points, and determining whether it belongs to the step area, thereby realizing the identification and path planning of the steps.
Accurately identify the position of the step, allowing the mobile robot to actively leap, avoid stuck problems, and improve work efficiency.
Smart Images

Figure CN118915765A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of mobile robots, and more specifically to a step recognition method, a mobile robot, a cleaning system, and a computer-readable storage medium. Background Art
[0002] With the rapid development of science and technology and the continuous improvement of people's living standards, more and more families are beginning to use mobile robots such as sweeping robots, which improve the comfort and convenience of people's lives and have a very broad market prospect. Mobile robots can identify information about the surrounding environment so as to plan routes and avoid obstacles based on the information.
[0003] In some application scenarios, there are steps on the route of the mobile robot. In related technologies, the mobile robot usually ignores low steps and treats them as the ground. However, since this method does not recognize the steps, the mobile robot cannot actively cross the steps and may get stuck. As a result, the working efficiency of the mobile robot is affected. Summary of the invention
[0004] The present disclosure aims to solve at least one of the technical problems existing in the prior art. To this end, one purpose of the present disclosure is to propose a step recognition method that can identify the location and height direction of the steps and represent them in a map to facilitate subsequent path planning of the mobile robot.
[0005] According to one aspect of the present disclosure, there is provided a step recognition method for a mobile robot, comprising: acquiring point cloud data, wherein the point cloud data includes height information of a measured point, the measured point being a point located outside a shell of the mobile robot and within a preset distance from a projection center of the mobile robot, the height information being used to indicate the height of the measured point relative to a robot plane, the robot plane being a plane where points of the mobile robot in contact with the ground are located; projecting the point cloud data into a target map to acquire projection points, wherein the target map is a map of a preset area where the mobile robot is located, the projection points corresponding one-to-one to the measured points in the point cloud data; and determining the position of the steps in the target map based on the projection points in the target map.
[0006] Exemplarily, determining the position of the steps in the target map based on the projection points in the target map includes: updating map point information of the map points in the target map based on the projection points, wherein the map point information includes at least at least one of the height and type of the map points and the number of projection points contained therein; determining the position of the steps in the target map based at least on the map point information of the map points in the target map.
[0007] Exemplarily, updating the map point information of the map point in the target map based on the projection point includes: updating the map point information of the map point in the target map based on the projection point and its height information, wherein the height information of the projection point is the height information of the corresponding measured point; determining the position of the step in the target map at least based on the map point information of the map point in the target map includes: determining the step candidate point in the target map based on the height of the map point in the target map and the number of projection points it contains; judging whether the step candidate point belongs to a step area to determine the position of the step in the target map, wherein the step area is the area where the step is located.
[0008] Exemplarily, determining the step candidate point in the target map based on the height of the map point in the target map and the number of projection points it contains includes: determining the map point in the target map whose height is less than or equal to a first height threshold as the target map point; determining the target map point whose number of projection points it contains is greater than or equal to a preset number threshold as the step candidate point; or determining the map point in the target map whose number of projection points it contains is greater than or equal to a preset number threshold as the target map point; determining the target map point among the target map points whose height is less than or equal to the first height threshold as the step candidate point.
[0009] Exemplarily, the determining whether the step candidate point belongs to the step area includes: based on the height of the step candidate point, determining at least one to-be-processed area in the target map, each of the to-be-processed areas being an area in which the height difference between the height of the included map points and the height of at least some of the step candidate points is less than or equal to a second height threshold; for each to-be-processed area in the at least one to-be-processed area, determining whether the to-be-processed area meets a first target requirement; when the to-be-processed area meets the first target requirement, determining that the step candidate point in the to-be-processed area belongs to the step area.
[0010] Exemplarily, the point cloud data is the current frame point cloud data; before judging whether the area to be processed meets the first target requirement, judging whether the step candidate point belongs to the step area, further includes: for each of the at least one area to be processed, judging whether there is a historical step area within a preset range centered on the step candidate point in the area to be processed, the historical step area being a step area determined based on at least one frame of point cloud data before the current frame point cloud data; when the historical step area exists, judging whether the consistency between the area to be processed and the historical step area meets the second target requirement; when the consistency between the area to be processed and the historical step area meets the second target requirement, determining that the step candidate point in the area to be processed belongs to the step area.
[0011] Exemplarily, the second target requirement includes: the height difference between the height of the area to be processed and the height of the historical step area is less than or equal to a third height threshold, and the angle between the height extension direction of the area to be processed and the height extension direction of the historical step area is less than or equal to a preset angle threshold; wherein the heights of the area to be processed and the historical step area are determined based on the heights of the map points they each contain, and the height extension direction is a direction perpendicular to a straight line obtained by fitting the step candidate point and the step candidate point.
[0012] Exemplarily, the first target requirement includes: the length of the area to be processed is greater than a length threshold; and, the width of the area to be processed is greater than a width threshold; and, the area of the area to be processed is greater than an area threshold; and / or, based on the height of the step candidate point, determining at least one area to be processed in the target map, including: taking each of the determined step candidate points as a seed point, and using the region growing method to determine in the target map a growth area adjacent to the seed point and with a height difference between the seed point and the seed point that is less than or equal to the second height threshold, so as to obtain the at least one area to be processed.
[0013] Exemplarily, the point cloud data is the point cloud data of the current frame, and the projection point corresponding to the point cloud data of the current frame is the current projection point; the updating of the map point information of the map point in the target map based on the projection point includes: determining the initial map point information corresponding to the current projection point for each map point in the target map based on the current projection point, the initial map point information at least including the initial type and initial height of the map point, the initial type including ground and convex; for each map point in the target map, when the initial type of the map point is convex or ground, comparing the initial type of the map point with the initial type of the map point; map point information and historical map point information of the map point; based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, determine the latest map point information of the map point to update the map point information of the map point; wherein the historical map point information of each map point in the target map is map point information determined based on historical projection points, the historical projection points are each projection point corresponding to each frame of point cloud data in at least one frame of point cloud data before the current frame of point cloud data, and the historical map point information includes the historical type and historical height of the map point, and the historical type includes ground and protrusion.
[0014] Exemplarily, the historical map point information is first historical map point information determined based on first historical projection points, the first historical projection points are a group of projection points corresponding to a frame of point cloud data before the current frame of point cloud data, the first map point information includes a first historical type and a first historical height of the map point, the first historical type includes ground and protrusion; the determining the latest map point information of the map point based on the comparison result of the initial map point information of the map point with the historical map point information of the map point includes: when the initial type of the map point is ground and the first historical type is protrusion, determining the latest type of the map point is ground, and determining the latest height of the map point is the initial height of the map point; and / or, when the initial type of the map point is ground and the first historical type is ground, maintaining the latest type of the map point as ground, and determining the latest height of the map point as the initial height of the map point. The latest height of the map point is the initial height of the map point; and / or, when the initial type of the map point is raised and the first historical type is ground, determine whether the height difference between the initial height of the map point and the first historical height of the map point is greater than or equal to a fourth height threshold; if the height difference between the initial height of the map point and the first historical height of the map point is greater than or equal to the fourth height threshold, determine that the latest type of the map point is raised, and determine the latest height of the map point to be the initial height of the map point; if the height difference between the initial height of the map point and the first historical height of the map point is less than the fourth height threshold, maintain the latest type of the map point as ground; and / or, when the initial type of the map point is raised and the historical type is raised, maintain the latest type of the map point as raised, and determine the latest height of the map point to be the initial height of the map point.
[0015] Exemplarily, the historical map point information is second historical map point information determined based on second historical projection points, the second historical projection points are each group of projection points corresponding to each frame of point cloud data in at least two frames of point cloud data before the current frame of point cloud data, the second historical map point information includes a second historical type and a second historical height of the map point, the second historical type includes ground and protrusion; the determining the latest map point information of the map point based on the comparison result of the initial map point information of the map point with the historical map point information of the map point includes: when the initial type of the map point is ground and the second historical type is protrusion, performing a type determination operation to determine the map point The latest type of the map point, and performing a height determination operation to determine the latest height of the map point; and / or, when the initial type of the map point is ground and the second historical type is ground, maintaining the latest type of the map point as ground; and / or, when the initial type of the map point is convex and the second historical type is ground, determining whether the height difference between the initial height of the map point and the second historical height of the map point is greater than or equal to a fourth height threshold; if the height difference between the initial height of the map point and the second historical height of the map point is greater than or equal to the fourth height threshold, performing the type determination operation to determine the latest type of the map point, and performing the height determination operation to determine the the latest height of the map point; if the height difference between the initial height of the map point and the second historical height of the map point is less than the fourth height threshold, maintaining the latest type of the map point as ground; and / or, when the initial type of the map point is raised and the second historical type is raised, maintaining the latest type of the map point as raised, and performing the height determination operation to determine the latest height of the map point; wherein the type determination operation includes: for each map point in the target map, determining the latest type of the map point based on a preset number of initial types of the map point, the preset number of initial types corresponding to a preset number of point cloud data one by one, and the preset number of point cloud data The target map comprises a map point in the target map, the map point having a preset number of initial heights, the initial type being the initial type of the map point determined based on the corresponding frame point cloud data; the height determination operation comprises: for each map point in the target map, determining the latest height of the map point based on a preset number of initial heights of the map point, the preset number of initial heights corresponding one-to-one to a preset number of point cloud data, the preset number of point cloud data comprising the current frame point cloud data and the at least two frames of point cloud data before the current frame point cloud data, each initial height being the initial height of the map point determined based on the corresponding frame point cloud data.
[0016] Exemplarily, the point cloud data is the current frame point cloud data, and the projection point corresponding to the current frame point cloud data is the current projection point; determining the position of the steps in the target map based on the projection point in the target map includes: determining the initial map point information corresponding to the current projection point for each map point in the target map based on the current projection point, the initial map point information including at least the initial type and initial height of the map point, and the initial type including depression; determining the edge area of the depression area based on the initial map point information of at least some of the map points in the target map, the depression area being the area where the map points whose initial type belongs to depression are located; determining that the edge area belongs to the step area, so as to determine the position of the steps in the target map, the step area being the area where the steps are located.
[0017] Exemplarily, the at least some of the map points are map points that include the current projection point when the current frame point cloud data is projected into the target map; determining the edge area of the sunken area based on the initial map point information of at least some of the map points in the target map includes: judging, based on the initial map point information of at least some of the map points in the target map, whether the initial heights corresponding to the map points arranged along a preset direction in the at least some of the map points jump from large to small along the preset direction, and the jump is that the height difference between the initial heights of two adjacent map points is greater than or equal to a fifth height threshold; when the initial heights corresponding to the map points arranged along the preset direction jump from large to small along the preset direction, determining that the map point with a larger initial height among the map points that have undergone the jump belongs to the edge area.
[0018] Exemplarily, after determining the position of the steps in the target map based on the projection points in the target map, the method further includes: determining an area of equal height after the mobile robot climbs the steps based on the projection points and the determined position of the steps, wherein the height of each map point in the area of equal height is within a preset height range; and marking the area of equal height after the mobile robot climbs the steps as the ground in the target map to update the target map.
[0019] According to another aspect of the present disclosure, a mobile robot is provided, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the computer program to implement the above-mentioned step recognition method.
[0020] According to another aspect of the present disclosure, a cleaning system is provided, comprising the above-mentioned mobile robot and a base station used in conjunction with the mobile robot, wherein the mobile robot is a cleaning robot, and the cleaning robot performs maintenance in the base station.
[0021] According to another aspect of the present disclosure, a computer-readable storage medium is provided, storing a computer program / instruction. When the computer program / instruction is executed by one or more processors, the above-mentioned step identification method is implemented.
[0022] The above technical solution can accurately determine the location of the step by projecting the point cloud data into the target map and based on the projection points in the target map. This solution can provide a relatively accurate basis for the path planning of the mobile robot. It can be understood that since the location of the step is obtained in this solution, when the mobile robot passes the step, it can actively cross the step (such as accelerating to cross the step, etc.), thereby helping to prevent the mobile robot from getting stuck and improving the working efficiency of the mobile robot.
[0023] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0025] Figure 1 A schematic flow chart of a step recognition method according to an embodiment of the present disclosure is shown;
[0026] Figure 2 A schematic diagram showing a step recognition method according to a specific embodiment of the present disclosure;
[0027] Figure 3 A schematic diagram showing a step recognition method according to another specific embodiment of the present disclosure;
[0028] Figure 4 A schematic diagram showing a mobile robot climbing stairs according to an embodiment of the related art;
[0029] Figure 5 A schematic diagram showing a step according to an embodiment of the present disclosure;
[0030] Figure 6 A schematic flow chart showing a step recognition method according to a specific embodiment of the present disclosure is shown;
[0031] Figure 7A schematic diagram showing a step recognition method according to another specific embodiment of the present disclosure;
[0032] Figure 8 A schematic flow chart showing a step recognition method according to another specific embodiment of the present disclosure;
[0033] Fig. 9 A schematic block diagram of a cleaning robot according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solutions and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein. Based on the embodiments of the present disclosure described in the present disclosure, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present disclosure.
[0035] As mentioned above, in the related art, mobile robots usually ignore low steps and treat the steps as equivalent to the ground. Since the steps are not recognized, the mobile robot does not have the ability to actively cross the steps. In other words, the mobile robot may pass through the steps at the same speed as when traveling on the ground. However, due to the height difference between the steps and the ground, this way of passing may cause the mobile robot to get stuck. Thus, the working efficiency of the mobile robot is affected. In view of this, the present disclosure provides a step recognition method, a mobile robot, a cleaning system and a computer-readable storage medium, which can more accurately determine the position of the steps in the target map and their height direction, thereby facilitating the mobile robot to actively cross the steps when passing through the steps, thereby helping to avoid the problem of the mobile robot getting stuck. The step recognition method, mobile robot, cleaning system and computer-readable storage medium are described in detail below.
[0036] According to one aspect of the present disclosure, a step recognition method is provided, which is used for a mobile robot. The mobile robot may be any one of a sweeping robot, a mopping robot, a sweeping and mopping robot, a transport robot, a lawn mowing robot, a service robot, etc., and the present disclosure does not limit the type of the mobile robot.
[0037] Figure 1 FIG. 2 is a schematic flow chart of a step recognition method according to an embodiment of the present disclosure. Figure 1 As shown, the step recognition method 100 may include step S110 , step S120 , and step S130 .
[0038] In step S110, point cloud data is obtained; the point cloud data includes height information of the measured point, the measured point is a point located outside the shell of the mobile robot and within a preset distance from the projection center of the mobile robot, and the height information is used to indicate the height of the measured point relative to the robot plane, and the robot plane is the plane where the point where the mobile robot contacts the ground is located. It can be understood by those skilled in the art that the projection center of the mobile robot can be the projection point when the center of the mobile robot is projected onto the ground.
[0039] Optionally, the point cloud data can be obtained by any existing or future developed sensor that can detect height information. For example, the sensor can be any one of a radar sensor, a depth sensor, and a structured light sensor. In a specific embodiment, the sensor can be a depth camera, which can be set at the front of the mobile robot.
[0040] In this article, the robot plane is the plane where the mobile robot contacts the ground. In some embodiments, the mobile robot includes a robot body and wheels disposed below the robot body. In this embodiment, the robot plane is the point where the mobile robot contacts the ground and can be the point where the wheels contact the ground.
[0041] Optionally, the preset distance can be set as needed. For example, the preset distance can be set according to the detection accuracy of the sensor used to collect point cloud data.
[0042] Optionally, the point cloud data may be point cloud data originally collected by a sensor for collecting point cloud data, or may be point cloud data obtained after preprocessing the original collected point cloud data. The preprocessing method may be filtering, denoising, or the like. In a specific embodiment, the original collected point cloud data may be subjected to noise filtering to obtain preprocessed point cloud data. This solution helps to reduce noise interference and improve the accuracy of point cloud data by preprocessing the original collected point cloud data. It also helps to reduce the amount of calculation in subsequent steps. The point cloud data may include height information of any number of measured points, and the number of measured points may depend on the collection range and collection interval of the sensor used to collect the point cloud data.
[0043] In step S120, the point cloud data is projected into a target map to obtain projection points, wherein the target map is a map of a preset area where the mobile robot is located, and the projection points correspond one-to-one to the measured points in the point cloud data.
[0044] Optionally, the target map may be a grid map, and the map points on the target map may be grids on the grid map. In this case, the number of projected points in the map point hereinafter is the number of projected points in the grid.
[0045] The size of the preset area can be set as needed. In some embodiments, the preset area can be the entire area corresponding to the mobile robot performing the current task. In this case, the target map can be regarded as a global map. For example, when the mobile robot is a sweeping robot, if the sweeping robot currently performs the task of cleaning the living room, then the entire living room can be the preset area.
[0046] In other embodiments, the preset area may be an area within a preset range centered on the projection center of the mobile robot. In this case, the preset area may change with the movement of the mobile robot, and accordingly, the target map may be regarded as a local map. In one embodiment, the preset area may be a circular area centered on the projection center of the mobile robot and having a preset radius. The preset radius may be set as needed, for example, the preset radius may be determined according to the detection accuracy of the sensor used to collect point cloud data. The preset radius may be less than or equal to the maximum detection distance indicated by the detection accuracy. In another embodiment, the preset area may be a rectangular area centered on the projection center of the mobile robot and having a preset width and a preset length. The setting method of the preset width and the preset length is similar to that of the preset radius, and will not be repeated. In a specific embodiment, the maximum detection distance indicated by the detection accuracy of the sensor is 3 meters (m). In this case, the preset width and the preset length may both be 2m. It can be understood that the closer the distance to the sensor, the higher the accuracy of the point cloud data obtained by the sensor. This embodiment helps to ensure the accuracy of the target map by making the preset width and the preset length both less than the maximum detection distance.
[0047] In step S130, the position of the step in the target map is determined based on the projection point in the target map.
[0048] In the present disclosure, a step refers to an area that a mobile robot can cross and that has a height difference from the ground the mobile robot is currently located at. The location of the step is a vertical plane between two areas with a height difference.
[0049] After projecting the point cloud data into the target map, the location of the steps can be determined based on the projected points in the target map.
[0050] In some embodiments, the height of each map point in the target map can be determined based on the projection point in each map point. Then, the height of each map point can be traversed in a preset order, and the map point with a sudden change in height can be determined based on the height of each map point. Then, it is determined whether the height difference between the two map points with sudden changes is less than or equal to a preset height threshold (e.g., the first height threshold below). If the result is yes, it is determined that the map point with a height inconsistent with the robot plane in the two map points belongs to a step. In other words, in this embodiment, it can be considered that the position of the map point is the position of the step in the target map. In this embodiment, the preset order can be set as needed. For example, the preset order can be an order arranged along a direction that is at a preset angle to the driving direction of the mobile robot and close to the mobile robot. For another example, the preset order can be an order arranged along a direction that is at a preset angle to the driving direction of the mobile robot and away from the mobile robot. The preset angle can be any angle, for example, it can be any angle in [0,180] degrees. It can be understood that the position of the step is an area where there is a height difference with the robot plane, and the height difference is less than or equal to the preset height threshold (e.g., the first height threshold below). In this embodiment, the map points belonging to the step area can be determined relatively quickly by traversing the heights of the map points, and the positions of the steps in the target map can be determined relatively accurately based on the positions of the map points belonging to the step area.
[0051] It is understood that the projection point may have height information, and its height information is the height information of the corresponding measured point. In other embodiments, the height information of each projection point and the position information of the map point to which each projection point belongs may be input into a trained neural network model, and the neural network model outputs the position information of the step in the target map.
[0052] The above embodiment of determining the position of the steps in the target map based on the projection points in the target map is only an example. In other embodiments, the position of the steps in the target map can also be determined based on the number of projection points in the map points in the target map and the height of each map point. This solution is described in detail below and will not be repeated here.
[0053] The above technical solution can accurately determine the location of the step by projecting the point cloud data into the target map and based on the projection points in the target map. This solution can provide a relatively accurate basis for the path planning of the mobile robot. It can be understood that since the location of the step is obtained in this solution, when the mobile robot passes the step, it can actively cross the step (such as accelerating to cross the step, etc.), thereby helping to prevent the mobile robot from getting stuck and improving the working efficiency of the mobile robot.
[0054] Exemplarily, determining the position of steps in the target map based on the projection points in the target map includes: updating map point information of map points in the target map based on the projection points, the map point information including at least one of the height and type of the map points and the number of projection points contained therein; determining the position of the steps in the target map based at least on the map point information of the map points in the target map.
[0055] In the solution of this example, the map point information of the map point in the target map can be updated first based on the projection point. For example, the height of the map point can be updated based on the height information of the projection point in the map point. For another example, the type of the map point can be updated based on the height information of the projection point in the map point, and the type of the map point can include but is not limited to the ground, obstacles (or protrusions). For another example, the number of projection points contained in the map point can be updated based on the number of projection points in the map point.
[0056] In the scheme of this example, only the map points with projection points are updated. It can be understood that in the process of the mobile robot acquiring point cloud data, each frame of point cloud data may only include a part of the target map. Therefore, when the map point information of the map points in the target map is updated based on the projection points, only the map points corresponding to the part of the area are updated. For other map points, their map point information can be determined based on historical point cloud data, which will not be repeated.
[0057] After updating the map point information of the map point in the target map, the position of the step in the target map can be determined based on at least the map point information of the map point in the target map. For example, the position of the step in the target map can be determined by traversing the map points. For another example, the position of the step in the target map can be determined based on the number of projection points in the map point in the target map and the height of each map point. No further details are given.
[0058] In the above technical solution, the map point information of the map point in the target map can be updated first, and then the position of the step in the target map can be determined based on at least the map point information of the map point in the target map. This solution can more accurately determine the map point information in the target map by combining the current frame point cloud data and the historical frame point cloud data, thereby helping to more accurately determine the position of the step in the target map.
[0059] Exemplarily, based on the projection points, map point information of the map points in the target map is updated, including: based on the projection points and their height information, the map point information of the map points in the target map is updated, the height information of the projection points is the height information of the corresponding measured points; based at least on the map point information of the map points in the target map, the position of the steps in the target map is determined, including: based on the heights of the map points in the target map and the number of projection points contained in the map points in the target map, determining the step candidate points in the target map; judging whether the determined step candidate points belong to the step area to determine the position of the steps in the target map, the step area being the area where the steps are located.
[0060] As mentioned above, the location of the step is the vertical plane between two areas with a height difference. It can be understood that when using sensors such as depth cameras to collect point cloud data of the mobile robot's surroundings, if there are steps in the mobile robot's surroundings, and the mobile robot is located on the ground on the lower side of the steps, in this case, there may be multiple measured points on the vertical plane corresponding to the steps. When projecting the point cloud data into the target map, at least some of the corresponding multiple measured points on the vertical plane may be projected onto the same map point. Therefore, it is possible to determine whether the map point is a step candidate point based on the height of the map point and the number of projection points contained in the map point.
[0061] Optionally, determining the step candidate points in the target map based on the height of the map points in the target map and the number of projection points contained in the map points in the target map may specifically include the following steps: determining the map points in the target map whose height is less than or equal to a first height threshold as candidate map points; determining the candidate map points whose number of projection points contained in the determined candidate map points is greater than or equal to a preset number threshold as step candidate points. The above embodiments for determining the step candidate points in the target map are only examples and are not intended to be a limitation of the present disclosure. In other implementation schemes of the present disclosure, it is also possible to first determine the map points whose number of projection points contained in the target map is greater than or equal to a preset number threshold, and then determine the map points whose height is less than or equal to the first height threshold among these map points as step candidate points. This scheme is described in detail below and will not be repeated here.
[0062] By performing the above operation on each map point in the target map, all step candidate points in the target map can be determined. After determining the step candidate points in the target map, it can be further determined whether the determined step candidate points belong to the step area, and after determining the step candidate points belonging to the step area, the position of the step in the target map can be determined according to the position of the step candidate points belonging to the step area. In other words, the position of the step in the target map can be represented by the position of the step candidate points belonging to the step area.
[0063] In the above technical solution, the number of step candidate points can be determined according to the height of the map point in the target map and the number of projection points contained in the map point in the target map, and the position of the step in the target map can be determined by judging whether the determined step candidate point belongs to the step area. In short, the solution can more accurately determine the step candidate point by comprehensively considering the height of the map point in the target map and the number of projection points contained in the map point in the target map, and then can more accurately determine the position of the step in the target map according to the step candidate point.
[0064] Exemplarily, based on the height of the map point in the target map and the number of projection points contained in the map point in the target map, determining the step candidate point in the target map includes: determining the map point whose number of projection points contained in the target map is greater than or equal to a preset number threshold as the target map point; and determining the target map point whose height is less than or equal to a first height threshold as the step candidate point.
[0065] Optionally, for each map point containing a projection point, it can be first determined whether the number of projection points in the map point is greater than a preset number threshold; if the number of projection points in the map point is greater than the preset number threshold, the map point is determined to be a target map point. Optionally, the preset number threshold can be set as needed. For example, it can be a theoretical value or an empirical value measured by experiment. In the solution of this example, by comparing the number of projection points in the map point with the preset number threshold, it can be more accurately determined whether the map point is a target map point, thereby providing a more reliable basis for determining the step candidate points in the subsequent steps.
[0066] After determining the target map point, the height of the target map point can be compared with the first height threshold to determine whether the target map point is a step candidate point. It can be understood that in addition to steps, map points corresponding to some insurmountable obstacles may also be classified as target map points. For example, walls, cabinets, etc. In this case, the target map point can be further distinguished as a step candidate point based on the height. For insurmountable obstacles such as walls and cabinets, their heights are usually higher, while the heights of steps are usually lower. Therefore, by comparing the height of the target map point with the first height threshold, the target map points belonging to insurmountable obstacles can be removed more accurately, thereby avoiding misidentification of insurmountable obstacles as steps as much as possible. The first height threshold can be set according to actual needs and will not be elaborated.
[0067] The above technical solution can avoid misidentifying insurmountable obstacles as steps as much as possible by judging whether a map point is a step candidate point according to the number of projection points contained in the map point and the height of the map point, thereby helping to more accurately determine the step candidate points.
[0068] Exemplarily, determining whether a determined step candidate point belongs to a step area includes: determining at least one to-be-processed area in a target map based on the height of the step candidate point, each to-be-processed area being an area in which the height difference between the height of the included map points and the height of at least some of the step candidate points is less than or equal to a second height threshold; for each to-be-processed area in at least one to-be-processed area, determining whether the to-be-processed area meets a first target requirement; and when the to-be-processed area meets the first target requirement, determining that the step candidate point in the to-be-processed area belongs to a step area.
[0069] After determining the step candidate point, at least one area to be processed can be determined in the target map based on the height information corresponding to the determined step candidate point. Optionally, for each step candidate point, the area to be processed corresponding to the step candidate point can be determined based on the height information corresponding to the step candidate point and the map points within a preset range around the step candidate point. For example, the height information of the step candidate point can be compared with the height information corresponding to each map point within the preset range. When the height difference between the two map points is less than or equal to the second height threshold, it can be determined that the step candidate point and the map point belong to the same area. The second height threshold can be determined according to actual needs and will not be repeated. Optionally, methods such as the regional growing method can also be used to determine the area to be processed. This scheme is described in detail below and will not be repeated.
[0070] It can be understood that for any area to be processed, the area to be processed may correspond to at least one step candidate point. As described above, each area to be processed is an area with the same height as at least some of the step candidate points. In other words, when the step candidate points are adjacent and the height information corresponding to the adjacent step candidate points is the same (that is, the height difference is less than or equal to the second height threshold), the adjacent step candidate points may correspond to the same area to be processed.
[0071] After determining at least one area to be processed, it can be determined in turn whether each area to be processed meets the first target requirement. The target requirement can be set as needed. For example, it can be any one or more of the requirements of the length requirement of the area to be processed, the width requirement of the area to be processed, the length requirement of the area occupied by the step candidate point in the area to be processed, the width requirement of the area occupied by the step candidate point in the area to be processed, and the length-to-width ratio requirement of the area occupied by the step candidate point in the area to be processed. When the area to be processed meets the first target requirement, it can be determined that the step candidate point in the area to be processed belongs to the step area. The step area is the location of the step.
[0072] Figure 2 FIG. 1 is a schematic diagram showing a step recognition method according to a specific embodiment of the present disclosure. Figure 2As shown in FIG. 1 , after determining the step candidate points, at least one area to be processed can be determined based on the step candidate points. Figure 2 In the embodiment shown, the number of the area to be processed is one, i.e., the raised area in the figure. After determining the area to be processed, it can be determined whether the area to be processed meets the target condition. When the area to be processed meets the target condition, the step candidate point in the area to be processed can be determined to be a step area. At the same time, since the area to be processed is an equal height area, the area to be processed can be marked as the ground at this time.
[0073] In the above technical solution, at least one area to be processed can be determined based on the step candidate point, and then the step area can be determined by judging whether each area to be processed meets the first target requirement. The solution can combine the map point information of the map points around the step candidate point to more accurately judge whether the step candidate point belongs to the step area, thereby helping to more accurately determine the location of the step in the target map.
[0074] Exemplarily, the point cloud data is the point cloud data of the current frame; before determining whether the area to be processed meets the first target requirement, determining whether the determined step candidate point belongs to the step area, and also including: for each area to be processed in at least one area to be processed, determining whether there is a historical step area within a preset range centered on the step candidate point in the area to be processed, the historical step area being a step area determined based on at least one frame of point cloud data before the current frame of point cloud data; when there is a historical step area, determining whether the consistency between the area to be processed and the historical step area meets the second target requirement; when the consistency between the area to be processed and the historical step area meets the second target requirement, determining that the step candidate point in the area to be processed belongs to the step area.
[0075] Optionally, the preset range can be set as needed. It can be understood that when the height and the height extension direction of the area to be processed are the same as those of the historical step area, the step candidate point in the area to be processed may belong to the same step area as the historical step area. At this time, it can be directly determined that the step candidate point in the area to be processed belongs to the step area.
[0076] In the scheme of this example, for each area to be processed, the preset range centered on each step candidate point in the area to be processed can be first determined, and then it is determined whether there is a historical step area in these preset ranges corresponding to the area to be processed. Alternatively, for each area to be processed, the preset ranges corresponding to two step candidate points at the edge of the area to be processed can be first determined, and then it is determined whether there is a historical step area in these two preset ranges. This embodiment only needs to determine two preset ranges. When the number of step candidate points in the area to be processed is large, the scheme of this embodiment is used to help improve the efficiency of step recognition.
[0077] When there is a historical step area, it can be determined whether the consistency between the area to be processed and the historical step area meets the second target requirement. In other words, it can be determined whether the area to be processed is consistent with the historical step area. If the area to be processed is consistent with the historical step area, the two areas may be the same area. In this case, it can be directly determined that the step candidate point in the area to be processed belongs to the step area. The second target requirement can be set as needed. For example, the second target requirement may include: the height extension direction of the two areas is consistent, and / or, whether the height between the two areas is consistent (that is, the height difference between the two areas is less than or equal to the third height threshold).
[0078] In the above technical solution, when there is a historical step region within a preset range centered on the step candidate point in the area to be processed, it is possible to determine whether the step candidate point in the area to be processed belongs to the step region by determining whether the consistency between the area to be processed and the historical step region meets the second target requirement. In this solution, if the consistency between the area to be processed and the historical step region meets the second target requirement, it is not necessary to determine again whether the area to be processed meets the first target requirement. This method helps to simplify the calculation process of step recognition and improve the efficiency of step recognition.
[0079] Exemplarily, the second target requirement includes: the height difference between the height of the area to be processed and the height of the historical step area is less than or equal to a third height threshold, and the angle between the height extension direction of the area to be processed and the height extension direction of the historical step area is less than or equal to a preset angle threshold, wherein the heights of the area to be processed and the historical step area are determined based on the heights of the map points they each contain, and the height extension direction is a direction starting from the step candidate point and perpendicular to the straight line obtained by fitting the step candidate point.
[0080] In the solution of this example, the height of the area to be processed is consistent with the height of the historical step area, which is equivalent to the height difference between the height of the area to be processed and the height of the historical step area being less than or equal to the third height threshold. Similarly, the height extension direction of the area to be processed is consistent with the height extension direction of the historical step area, which is equivalent to the angle between the height extension direction of the area to be processed and the height extension direction of the historical step area being less than or equal to the preset angle threshold. Both the third height threshold and the preset angle threshold can be determined according to actual needs and will not be described in detail.
[0081] In the solution of this example, the height of the area to be processed and the historical step area are determined based on the height of the map points contained in each. The area to be processed is used as an example for explanation. In some embodiments, the height of the area to be processed can be any one of the mean, median, maximum or minimum values of the heights of all map points in the area to be processed. The height of the historical step area is determined in a similar manner to the height of the area to be processed, and will not be repeated.
[0082] In the scheme of this example, the height extension direction of the area to be processed is perpendicular to the straight line obtained by fitting the step candidate points in the area to be processed. The height extension direction of the historical step area is perpendicular to the straight line obtained by fitting the step candidate points in the historical step area. Figure 2 Taking the embodiment shown as an example, Figure 2 The height extension direction of the ground (h=1) on the right side is the height extension direction of the step ( Figure 2 left and right directions in the image).
[0083] Figure 3 FIG. 1 is a schematic diagram showing a step recognition method according to another specific embodiment of the present disclosure. Figure 3 As shown in FIG. 1 , first, the point cloud data is collected by the depth camera set on the mobile robot. Then, the point cloud data is projected into the target map, and the step candidate points are determined based on the height of the map points in the target map and the number of projection points included. Then, the area to be processed (i.e. Figure 3 The method for determining the area to be processed has been described in detail above and will not be repeated here. After determining the area to be processed, it can be determined whether there is a historical step area within a preset range centered on the step candidate point in the area to be processed. Figure 5 In the illustrated embodiment, there is a historical step area within a preset range centered on the step candidate point in the raised area, that is, a step area between the ground (h=0) and the ground (h=1). Then, it can be determined whether the height difference between the height of the area to be processed and the height of the historical step area is less than or equal to the third height threshold (that is, whether the heights are consistent), and whether the angle between the height extension direction of the area to be processed and the height extension direction of the historical step area is less than or equal to the preset angle threshold (that is, whether the height extension direction is consistent). Since the height and height extension direction of the area to be processed are consistent with the historical step area, at this time, it can be determined that the step candidate point in the area to be processed belongs to the step area. In this case, the step candidate point in the area to be processed and the historical step area can be updated to the same step area, and at the same time, the raised area can be marked as the ground.
[0084] In the above technical solution, the second target requirement includes: the height difference between the height of the area to be processed and the height of the historical step area is less than or equal to the third height threshold, and the angle between the height extension direction of the area to be processed and the height extension direction of the historical step area is less than or equal to the preset angle threshold. In other words, the solution can more accurately determine whether the step candidate point in the area to be processed belongs to the step area by judging whether the height and height extension direction between the area to be processed and the historical step area are consistent. The solution is simple to calculate and helps to improve the efficiency of step recognition.
[0085] Exemplarily, the first target requirement includes: the length of the area to be processed is greater than a length threshold; and, the width of the area to be processed is greater than a width threshold; and, the area of the area to be processed is greater than an area threshold; and / or, based on the height of the determined step candidate points, at least one area to be processed is determined in the target map, including: taking each of the determined step candidate points as a seed point, and using the region growing method to determine in the target map a growth area adjacent to the seed point and with a height difference between the seed point and the seed point that is less than or equal to a second height threshold, so as to obtain at least one area to be processed.
[0086] Exemplarily, the first target requirement includes: the length of the region to be processed is greater than a length threshold; and, the width of the region to be processed is greater than a width threshold; and, the area of the region to be processed is greater than an area threshold.
[0087] Optionally, the length threshold, the width threshold and the area threshold can be set as required, for example, they can be theoretical values determined through experiments or empirical values determined based on experience.
[0088] It can be understood that when the length of the area to be processed is less than or equal to the length threshold, the area to be processed may not be a step. For example, it may be a book or a wooden board dropped on the ground. In this case, the area to be processed can be used as an obstacle area to prevent the mobile robot from passing through the area to be processed and causing contamination or damage to the object corresponding to the area to be processed.
[0089] It can be understood that when the width of the area to be processed is less than or equal to the width threshold, the area to be processed may not be a step. For example, it may be an object such as an electric wire. In some embodiments, when the mobile robot is actually used, in order to prevent the mobile robot from being entangled by electric wires, it is necessary to control the mobile robot to bypass the area where the electric wires are located. In the scheme of this embodiment, by judging whether the width of the area to be processed is greater than the width threshold, it is possible to avoid judging the area where the electric wires are located as a step area, thereby helping to prevent the mobile robot from crossing the electric wires as steps during movement and causing them to be entangled. This scheme helps to improve the accuracy of the determined steps.
[0090] It can be understood that when the area of the area to be processed is less than or equal to the area threshold, the area to be processed may not be a step. For example, it may be a small protrusion that falls on the ground. Therefore, this solution can prevent areas such as small protrusions from being determined as step areas, which helps to improve the accuracy of the determined steps.
[0091] In the above technical scheme, by respectively judging whether the length of the area to be processed is greater than the length threshold, whether the width of the area to be processed is greater than the width threshold, and whether the area of the area to be processed is greater than the area threshold, it helps to avoid misidentifying other areas (such as small protrusions, wires, etc.) as step areas, thereby helping to provide a more accurate basis for the path planning of the mobile robot.
[0092] Exemplarily, based on the height of the determined step candidate points, at least one area to be processed is determined in the target map, including: taking each of the determined step candidate points as a seed point, and using the region growing method to determine in the target map a growth area adjacent to the seed point and with a height difference between the seed point and the seed point less than or equal to a second height threshold, so as to obtain at least one area to be processed.
[0093] In a specific embodiment, each step candidate point can be used as a seed point in turn. For any seed point, the neighborhood map points of the seed point can be traversed with the height difference with the seed point being less than the second height difference threshold as the growth criterion. If the neighborhood map point meets the growth criterion, the neighborhood map point is used as a growth point and the neighborhood map points of the growth point are traversed with the above growth criterion. The growth is continued according to this principle until no map point that meets the growth criterion is found, thereby generating a region to be processed; and then a step candidate point that does not belong to the determined region to be processed is selected from the remaining step candidate points as a seed point to start growth.
[0094] The above technical solution can quickly and accurately determine at least one region to be processed based on the region growing method. This solution helps to improve the efficiency of step recognition.
[0095] Exemplarily, after determining the position of the steps in the target map based on the projection points in the target map, the method further includes: determining the equal-height area after the mobile robot climbs the steps based on the projection points and the determined position of the steps; marking the equal-height area after the mobile robot climbs the steps as the ground in the target map to update the target map.
[0096] In this article, an equal-height area means that the heights of each map point in the area are within a preset height range, that is, the height difference between each map point is less than or equal to a preset height threshold (for example, it can be a second height threshold).
[0097] The inventors discovered during their research that when a mobile robot can recognize steps, if the plane where the mobile robot is located changes (for example, after the mobile robot goes up or down the steps), the reference plane for the mobile robot to detect the height of the object changes before and after the plane where it is located, resulting in inconsistent detection of the object's height, which in turn may cause the height of the area in the obtained target map to lose consistency, thereby affecting the path planning and obstacle avoidance functions of the mobile robot. Figure 4 FIG. 2 is a schematic diagram showing a mobile robot climbing stairs according to an embodiment of the present disclosure. Figure 4 As shown, after the robot goes up the stairs, the most recently detected area is updated to the ground, while the previously detected part that has not been updated is a protrusion. On the local map (i.e., the target map), although the two are on the same plane, they have different height values, and the unupdated part will be treated as an obstacle for path planning. Although it will be gradually updated as the mobile robot moves forward, it will still affect the working efficiency of the mobile robot. In the scheme of this example, the equal-height area after climbing the stairs is marked as the ground. In this case, even if the plane where the mobile robot is located changes, the structural unity can still be guaranteed, that is, the area around the mobile robot after it goes up the stairs is still the ground. As a result, it helps to maintain the consistency of the structural information around the mobile robot in the target map after the plane where the mobile robot is located changes, avoiding the problem of height consistency.
[0098] In the scheme of this example, obstacles represent objects that exceed the crossing ability of the mobile robot (for example, the height of the protrusion can be used to determine whether it can be crossed, and a certain height threshold can be set for comparison and judgment. If the height of the protrusion exceeds the height threshold, it can be regarded as an obstacle. If the height of the protrusion does not exceed the height threshold, it can be regarded as a step if other conditions are met). The unification of the structure described in this example means that all planes that the mobile robot can reach in the target map are marked as the ground. It can be understood that for obstacles in the target map, even if the reference plane changes, the mobile robot still cannot cross the obstacle. In other words, changes in the plane where the mobile robot is located will not affect the planning of the mobile robot's path (that is, regardless of whether the reference plane changes, the mobile robot needs to bypass the obstacle when planning the path). However, for the steps that the mobile robot can cross in the target map, if the structural unification cannot be achieved, it may affect the planning of the mobile robot's path and the working efficiency of the mobile robot. Figure 4Taking the embodiment shown as an example, after the mobile robot climbs the stairs, the areas on both sides of the mobile robot are still obstacles in the target map because the areas on both sides of the mobile robot are not updated. Then, when performing path planning, the mobile robot may be controlled to bypass the areas on both sides of the mobile robot that could have been passed directly, thereby affecting the working efficiency of the mobile robot. The solution of this example effectively avoids the above situation by making the planes that the mobile robot can reach in the target map all be the ground.
[0099] In the solution of this example, for the equal-height area after climbing the stairs, only its identification information is updated to the ground, and its own height information is not changed. Figure 5 A schematic diagram of a step according to an embodiment of the present disclosure is shown. Figure 5 As shown in the figure, the step is the vertical plane between the left and right ground in the figure. The mobile robot moves along the arrow direction in the figure. The equal height area after climbing the step is the plane with height h=1 on the right side of the step in the figure. In this case, the plane with height h=1 can be marked as the ground, that is, the ground (h=1).
[0100] Optionally, in the above embodiment of updating the location of the steps to the target map, after determining the location of the steps, the location of the steps can be updated in the target map, and the equal-height area after climbing the steps can be marked as the ground. In this embodiment, the update of the location of the steps in the target map and the update of the equal-height area after the steps can be completed through one map update, thereby helping to improve the update efficiency of the target map.
[0101] In the above technical solution, the equal height area after climbing the steps is marked as the ground to update the target map. Thus, the planes that the mobile robot can reach in the target map can maintain a unified structure, which helps to avoid the problem of height consistency.
[0102] For example, after determining the position of the step in the target map based on the projection point, the method may further include: marking the height direction of the step. It can be understood that the height direction of the step is the ascending / descending direction of the step relative to the plane where the mobile robot is currently located. Figure 5 The height direction is explained by taking as an example. When the mobile robot is on the ground on the left side of the figure, the height direction of the step is the ascending direction. When the mobile robot is on the ground on the right side of the figure, the height direction of the step is the descending direction. The height direction can be represented by "+" or "-", where "+" indicates that the height direction of the step is the ascending direction and "-" indicates that the height direction of the step is the descending direction.
[0103] Figure 6FIG. 1 is a schematic flow chart of a step recognition method according to a specific embodiment of the present disclosure. In this embodiment, the target map is a 2D grid map, and the map points are grids. Figure 6 As shown, the method includes step S610, step S620, step S630, step S640, step S650, step S660 and step S670.
[0104] In step S610, after acquiring the 3D point cloud data, the point cloud data is projected onto a 2D grid map. In step S620, the grids (i.e., grids) whose number of points (i.e., the number of projected points) in the grid is greater than a threshold (i.e., a preset number threshold) are counted and marked as step candidate points.
[0105] In step S620, the grids counted are grids whose heights are less than or equal to a first height threshold.
[0106] In step S630, the point cloud data is updated to the map (ie, the target map). In this step, the map point information of the map points in the target map may be updated based on the projection points.
[0107] In step S640, region growing is performed on all step candidate points. In this step, at least one region to be processed can be obtained by region growing. The method of region growing has been described in detail above and will not be repeated here. For each region to be processed in at least one region to be processed obtained by region growing, step S650 can be executed to determine whether the step condition (i.e., the first target requirement) is met. The specific method for determining whether the region to be processed meets the step condition has been described in detail above and will not be repeated here. When the region to be processed does not meet the step condition, no operation is performed on the region to be processed (i.e., no operation). When the region to be processed meets the step condition, step S660 can be executed to mark these step candidate points (i.e., the step candidate points on the region to be processed) as steps and record their height direction.
[0108] Next, step S670 is executed to mark the connected protrusions of equal height behind the steps as the ground.
[0109] It is understandable that when planning the path of the mobile robot, different strategies may be required for the mobile robot to go up and down stairs. This solution can provide more information for the path planning of the mobile robot by marking the height direction of the stairs, thereby helping to improve the working efficiency of the mobile robot.
[0110] Exemplarily, the point cloud data is the point cloud data of the current frame, and the projection point corresponding to the point cloud data of the current frame is the current projection point; updating the map point information of the map point in the target map based on the projection point includes: determining the initial map point information corresponding to the current projection point for each map point in the target map based on the current projection point, the initial map point information at least including the initial type and initial height of the map point, and the initial type includes ground and protrusion; for each map point in the target map, when the initial type of the map point is protrusion or ground, comparing the initial map point information of the map point with the historical map point information of the map point; based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, determining the latest map point information of the map point to update the map point information of the map point; wherein the historical map point information of each map point in the target map is the map point information determined based on the historical projection point, the historical projection point is each projection point corresponding to each frame of point cloud data in at least one frame of point cloud data before the current frame of point cloud data, the historical map point information includes the historical type and historical height of the map point, and the historical type includes ground and protrusion.
[0111] Optionally, the height of each map point can be determined based on the height information of the current projection point, and the initial type of the map point can be determined based on the height of each map point. For example, based on the current projection point, determining the initial map point information corresponding to the current projection point for each map point in the target map can include the following steps: for any map point, determining the height of the current projection point with the largest height value among the current projection points contained in the map point as the height of the map point; determining whether the height of the map point is within a preset height range; when the height of the map point is greater than the upper limit of the preset height range, the initial type of the map point can be determined to be a protrusion; when the height of the map point is within the preset height range, the initial type of the map point can be determined to be the ground. The preset height range can be set as needed and will not be elaborated on.
[0112] Alternatively, the height of each map point can be determined based on the height information of the current projection point, and the initial type of the map point can be determined based on the height of each map point and the number of current projection points contained in the map point. For example, based on the current projection point, determining the initial map point information corresponding to the current projection point of each map point in the target map can include the following steps: for any map point, determining the height of the projection point with the largest height value among the projection points contained in the map point as the height of the map point; judging whether the height of the map point is within a preset height range, and judging whether the number of current projection points contained in the map point is within a preset number range; when the height of the map point is greater than the upper limit of the preset height range and the number of current projection points contained in the map point is greater than the upper limit of the preset number range, the initial type of the map point can be determined to be a protrusion; when the height of the map point is within the preset height range and the number of current projection points contained in the map point is within the preset number range, the initial type of the map point can be determined to be the ground. The preset height range and the preset number range can be set as needed and will not be repeated.
[0113] After determining the initial map point information of the map point, the historical map point information of the map point can be obtained from the target map, and the historical map point information is compared with the initial map point information. Then, the map point information of the map point is updated according to the comparison result. The specific update strategy is described in detail below.
[0114] In the above technical solution, the historical map point information and the initial map point information of the map point can be combined to update the map point information of the map point. This helps to more accurately determine the map point information of each map point, thereby providing a more accurate basis for tasks such as path planning for mobile robots.
[0115] Exemplarily, the historical map point information is first historical map point information determined based on first historical projection points, the first historical projection points are a group of projection points corresponding to a frame of point cloud data before the current frame of point cloud data, the first historical map point information includes a first historical type and a first historical height of the map point, the first historical type includes ground and protrusion; based on the comparison result of the initial map point information of the map point and the first historical map point information of the map point, determining the latest map point information of the map point includes: when the initial type of the map point is ground and the first historical type is protrusion, determining the latest type of the map point is ground, and determining the latest height of the map point is the initial height of the map point; and / or, when the initial type of the map point is ground and the first historical type is ground, keeping the latest type of the map point unchanged, that is, still ground; and / or, when the initial type of the map point is a projection and the first historical type is the ground, determining whether the height difference between the initial height of the map point and the first historical height of the map point is greater than or equal to a fourth height threshold; if the height difference between the initial height of the map point and the first historical height of the map point is greater than or equal to the fourth height threshold, determining that the latest type of the map point is a projection, and determining the latest height of the map point to be the initial height of the map point; if the height difference between the initial height of the map point and the first historical height of the map point is less than the fourth height threshold, keeping the latest type of the map point unchanged, as the ground; and / or, when the initial type of the map point is a projection and the first historical type is a projection, keeping the latest type of the map point unchanged, as the projection, and determining the latest height of the map point to be the initial height of the map point.
[0116] Optionally, based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, the latest map point information of the map point is determined, including: when the initial type of the map point is ground and the historical type is convex, the latest type of the map point is determined to be ground, and the latest height of the map point is determined to be the initial height of the map point. In the scheme of this embodiment, if the initial type of the map point is ground and the historical type is convex, the latest type of the map point can be determined to be ground, and the latest height of the map point is updated to the initial height of the map point. It can be understood that compared with the historical results (i.e., the determined historical map point information), the confidence of the initial map point information determined based on the current frame point cloud data is higher. Therefore, in this embodiment, the accuracy of the current frame point cloud data is more trusted, so the map point information of the map point can be updated according to the initial map point information obtained based on the current frame point cloud data.
[0117] Optionally, based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, determining the latest map point information of the map point includes: when the initial type of the map point is ground and the historical type is ground, keeping the latest type of the map point unchanged. In this embodiment, the initial type and the historical type of the map point are the same, so the latest type of the map point may not be updated. At this time, the latest type of the map point is still the historical type, i.e., ground. In some embodiments, when the initial type of the map point is ground and the historical type is ground, the latest height of the map point may also be determined to be the initial height of the map point.
[0118] Optionally, based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, determining the latest map point information of the map point includes: when the initial type of the map point is a protrusion and the historical type is ground, determining whether the height difference between the initial height of the map point and the historical height of the map point is greater than or equal to a fourth height threshold; if the height difference between the initial height of the map point and the historical height of the map point is greater than or equal to the fourth height threshold, determining the latest type of the map point to be a protrusion, and determining the latest height of the map point to be the initial height of the map point; if the height difference between the initial height of the map point and the historical height of the map point is less than the fourth height threshold, keeping the latest type of the map point unchanged. In the scheme of this embodiment, if the initial type of the map point is a protrusion and the historical type is ground, the initial height of the map point and the historical height of the map point can be compared. If the initial height of the map point is inconsistent with the historical height (i.e., the height difference is greater than or equal to the fourth height threshold), at this time, it can be determined that the latest type of the map point is a protrusion, and the latest height of the map point is determined to be the initial height of the map point. If the initial height of the map point is consistent with the historical height (i.e., the height difference is less than the fourth height threshold), the latest type of the map point can be kept unchanged, that is, the latest type of the map point is still ground. In some embodiments, when the initial height of the map point is consistent with the historical height, the latest height of the map point can be kept unchanged, or the latest height of the map point can be determined as the initial height.
[0119] Optionally, based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, the latest map point information of the map point is determined, including: when the initial type of the map point is convex and the historical type is convex, the latest type of the map point is kept unchanged, and the latest height of the map point is determined to be the initial height of the map point. In the scheme of this embodiment, if the initial type of the map point is convex and the historical type is convex, it can be determined that the latest type of the map point is still convex, that is, the latest type of the map point is kept unchanged. At the same time, since the height reference corresponding to the acquisition of the current frame point cloud data and the acquisition of the historical frame point cloud data may have changed, at this time, in order to improve the accuracy of the map point height, the latest height of the map point can be determined to be the initial height of the map point.
[0120] The above technical solution can more accurately determine the latest map point information of the map point based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, thereby helping to ensure the accuracy of the map point information of the map point. Therefore, it can further provide a more accurate basis for tasks such as path planning for mobile robots, thereby helping to improve the working efficiency of mobile robots.
[0121] Exemplarily, the historical map point information is second historical map point information determined based on the second historical projection point, the second historical projection point is each group of projection points corresponding to each frame of point cloud data in at least two frames of point cloud data before the current frame of point cloud data, the second historical map point information includes a second historical type and a second historical height of the map point, the second historical type includes ground and protrusion; based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, determining the latest map point information of the map point includes: when the initial type of the map point is ground and the second historical type is protrusion, performing a type determination operation to determine the latest map point information of the map point A new type is determined, and a height determination operation is performed to determine the latest height of the map point; and / or, when the initial type of the map point is ground and the second historical type is ground, the latest type of the map point is kept unchanged and is still ground; and / or, when the initial type of the map point is raised and the second historical type is ground, it is determined whether the height difference between the initial height of the map point and the second historical height of the map point is greater than or equal to a fourth height threshold; if the height difference between the initial height of the map point and the second historical height of the map point is greater than or equal to the fourth height threshold, a type determination operation is performed to determine the latest type of the map point, and a height determination operation is performed to determine the latest height of the map point; if the height difference between the initial height of the map point and the second historical height of the map point is less than a fourth height threshold, keeping the latest type of the map point unchanged, as ground; and / or, when the initial type of the map point is convex and the second historical type is convex, keeping the latest type of the map point unchanged, as convex, and performing a height determination operation to determine the latest height of the map point; wherein the type determination operation includes: for each map point in the target map, determining the latest type of the map point based on a preset number of initial types of the map point, the preset number of initial types and the preset number of point cloud data One-to-one correspondence, the preset number of point cloud data includes the current frame point cloud data and at least two frames of point cloud data before the current frame point cloud data, and each initial type is the initial type of the map point determined based on the corresponding frame point cloud data; the height determination operation includes: for each map point in the target map, determining the latest height of the map point based on the preset number of initial heights of the map point, the preset number of initial heights corresponds one-to-one to the preset number of point cloud data, the preset number of point cloud data includes the current frame point cloud data and at least two frames of point cloud data before the current frame point cloud data, and each initial height is the initial height of the map point determined based on the corresponding frame point cloud data.
[0122] Optionally, based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, the latest map point information of the map point is determined, including: when the initial type of the map point is ground and the second historical type is convex, performing a type determination operation to determine the latest type of the map point, and performing a height determination operation to determine the latest height of the map point. In the scheme of this embodiment, if the initial type of the map point is ground and the second historical type is convex, a type determination operation can be performed to determine the latest type of the map point, and a height determination operation can be performed to determine the latest height of the map point. In this embodiment, the type determination operation and the height determination operation can be used to realize the integration of the current frame point cloud data and the historical frame point cloud data, so that the latest map point information of the map point can be determined more accurately.
[0123] Optionally, based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, determining the latest map point information of the map point includes: when the initial type of the map point is ground and the second historical type is ground, keeping the latest type of the map point unchanged, that is, still ground.
[0124] Optionally, based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, determining the latest map point information of the map point includes: when the initial type of the map point is a protrusion and the second historical type is the ground, determining whether the height difference between the initial height of the map point and the second historical height of the map point is greater than or equal to a fourth height threshold; if the height difference between the initial height of the map point and the second historical height of the map point is greater than or equal to the fourth height threshold, performing a type determination operation to determine the latest type of the map point, and performing a height determination operation to determine the latest height of the map point; if the height difference between the initial height of the map point and the second historical height of the map point is less than the fourth height threshold, keeping the latest type of the map point unchanged, which is still the ground. In the scheme of this embodiment, if the initial type of the map point is a protrusion and the second historical type is the ground, the initial height of the map point can be compared with the second historical height of the map point. If the initial height of the map point is inconsistent with the historical height (that is, the height difference is greater than or equal to the fourth height threshold), at this time, it can be determined to perform a type determination operation to determine the latest type of the map point, and perform a height determination operation to determine the latest height of the map point. If the initial height of the map point is consistent with the historical height (i.e., the height difference is less than the fourth height threshold), the latest type of the map point can be kept unchanged, that is, the latest type of the map point is still ground. In some embodiments, when the initial height of the map point is consistent with the historical height, the latest height of the map point can be kept unchanged, or the latest height of the map point can be determined as the initial height.
[0125] Optionally, based on the comparison result of the initial map point information of the map point and the historical map point information of the map point, the latest map point information of the map point is determined, including: when the initial type of the map point is convex and the second historical type is convex, the latest type of the map point is kept unchanged, and it is still convex, and a height determination operation is performed to determine the latest height of the map point. In the scheme of this embodiment, if the initial type of the map point is convex and the second historical type is convex, it can be determined that the latest type of the map point is still convex, that is, the latest type of the map point is kept unchanged. At the same time, since the height reference corresponding to the acquisition of the current frame point cloud data and the acquisition of the historical frame point cloud data may have changed, at this time, in order to improve the accuracy of the map point height, a height determination operation can be performed to determine the latest height of the map point.
[0126] In the scheme of this example, the type determination operation includes: for each map point in the target map, the latest type of the map point is determined based on a preset number of initial types of the map point, the preset number of initial types corresponds to a preset number of point cloud data, the preset number of point cloud data includes the current frame point cloud data and at least two frames of point cloud data before the current frame point cloud data, and each initial type is the initial type of the map point determined based on the corresponding frame point cloud data. The preset number can be set as needed and will not be described in detail.
[0127] Optionally, determining the latest type of the map point based on a preset number of initial types of the map point includes: when the preset number of initial types includes different initial types, determining the initial type with the largest number of repetitions as the latest type of the map point. In a specific embodiment, the preset number is 3. The preset number of initial types of the map point is [ground, ground, raised], in which case it can be determined that the latest type of the map point is ground.
[0128] In the scheme of this example, the height determination operation includes: for each map point in the target map, determining the latest height of the map point based on a preset number of initial heights of the map point, the preset number of initial heights corresponding one-to-one to a preset number of point cloud data, the preset number of point cloud data including the current frame point cloud data and at least two frames of point cloud data before the current frame point cloud data, each initial height is the initial height of the map point determined based on the corresponding frame point cloud data.
[0129] Optionally, determining the latest height of the map point based on a preset number of initial heights of the map point includes: calculating a weighted sum of the preset number of initial heights to determine the latest height of the map point. In this embodiment, different weights can be assigned to the preset number of initial heights. For example, the closer the point cloud data corresponding to the initial height is to the current frame point cloud data, the larger the corresponding weight can be. The above method of determining the latest height of the map point is only an example. In other embodiments, the median, maximum or minimum value of the preset number of initial heights can also be determined as the latest height of the map point.
[0130] The above technical solution utilizes the height determination operation and / or the type determination operation to re-determine the latest height and / or the latest type of the map point. Thus, the map point information of the map point can be more accurately determined based on multi-frame point cloud data, which helps to avoid erroneous results caused by errors in single-frame point cloud data.
[0131] Exemplarily, the point cloud data is the current frame point cloud data, and the projection point corresponding to the current frame point cloud data is the current projection point; determining the position of the steps in the target map based on the projection points in the target map includes: determining the initial map point information corresponding to the current projection point for each map point in the target map based on the current projection point, the initial map point information including the initial type and initial height of the map point, the initial type including depression; determining the edge area of the depression area based on the initial map point information of at least some of the map points in the target map, the depression area being the area where the map points whose initial type is depression are located; determining that the edge area belongs to the step area, so as to determine the position of the steps in the target map, the step area being the area where the steps are located.
[0132] Optionally, the height of each map point can be determined based on the height information of the current projection point, and the initial type of the map point can be determined based on the height of each map point. For example, determining the initial type of the map point based on the height information of each map point in the target map corresponding to the current frame point cloud data can also include the following steps: for any map point, when the height of the map point is less than the lower limit of a preset height range (refer to the preset height range described above), determining the initial type of the map point as a concave.
[0133] Alternatively, the height of each map point can be determined based on the height information of the current projection point, and the initial type of the map point can be determined based on the height of each map point and the number of current projection points contained in the map point. For example, based on the current projection point, determining the initial map point information corresponding to the current projection point for each map point in the target map can also include the following steps: for any map point, determining the height of the projection point with the largest height value among the projection points contained in the map point as the height of the map point; determining whether the height of the map point is within a preset height range (refer to the preset height range described above), and determining whether the number of current projection points contained in the map point is within a preset number range (refer to the preset number range described above); when the height of the map point is less than the lower limit of the preset height range and the number of current projection points contained in the map point is less than the lower limit of the preset number range, it can be determined that the initial type of the map point is concave.
[0134] When the initial type of the map point is a depression, the edge area of the depression area can be determined based on the initial map point information of at least some map points in the target map. Then, it can be determined that the edge area belongs to the step area to determine the location of the step in the target map.
[0135] In the above scheme, when the initial type of the map point is a depression, the initial map point information of at least some map points in the target map can be used to more accurately determine the edge area of the limited area, thereby determining the step area based on the edge area. The scheme is simple to calculate and helps to improve the efficiency of step recognition.
[0136] Exemplarily, at least some of the map points are map points that include the current projection point when the current frame point cloud data is projected into the target map; based on the initial map point information of at least some of the map points in the target map, the edge area of the sunken area is determined, including: based on the initial map point information of at least some of the map points in the target map, judging whether the initial heights corresponding to the map points arranged along a preset direction in at least some of the map points jump from large to small along the preset direction, and the height difference between the initial heights of two adjacent map points is greater than or equal to a fifth height threshold; when the initial heights corresponding to the map points arranged along the preset direction jump from large to small along the preset direction, determining that the map points with larger initial heights among the map points that have jumped belong to the edge area.
[0137] Optionally, the preset direction can be set according to actual needs, for example, it can be a direction from the center of gravity of the mobile robot to the recessed area.
[0138] Optionally, the fifth height threshold may be set as required and will not be described in detail.
[0139] Optionally, determining the edge area of the sunken area based on the initial map point information of at least some of the map points in the target map may further include the following steps: when the initial heights corresponding to the map points arranged along the preset direction do not jump from large to small along the preset direction, determining that at least some of the map points do not belong to the step area. When the initial heights corresponding to the map points arranged along the preset direction do not jump from large to small along the preset direction, it may be that the step area is not within the field of view of the mobile robot (i.e., the field of view of the sensor used to obtain point cloud data). In this case, the edge area of the sunken area cannot be determined based on the current frame point cloud data. At this time, it can be determined that at least some of the map points corresponding to the current frame point cloud data do not belong to the step area.
[0140] Figure 7 A schematic diagram of a step recognition method according to another specific embodiment of the present disclosure is shown. In this embodiment, the sensor used to obtain point cloud data is a depth camera. Figure 7 It consists of two parts: upper and lower. Figure 7 In the upper part of the figure, the step is not within the field of view of the mobile robot (i.e., the step is in the robot's blind spot). At this point, there is a difference between the step position inferred based on the point cloud data and the actual step position. In this case, the actual position of the step cannot be determined based on the point cloud data. Figure 7 In the lower part of , the actual step position is within the field of view of the mobile robot. That is, the actual step position is consistent with the inferred step position. Therefore, at this time, the position of the step can be determined more accurately based on the point cloud data. The specific method for determining the step area has been described in detail above and will not be repeated here.
[0141] Figure 8 FIG. 2 is a schematic flow chart of a step recognition method according to another specific embodiment of the present disclosure. In this embodiment, the target map is a grid map, and the map points are grids (which can be referred to as grids for short). Figure 8As shown, after projecting the point cloud data onto the target map, each grid in the target map receives new data (i.e., the height information of each measured point). After the grid receives the new data, the height information of each grid can be determined based on the new data content, and the initial type of each grid can be determined based on the height information. For any grid, when the initial type of the grid is ground, if the historical type (historical label content) of the grid is ground, the grid is not updated; if the historical type of the grid is convex, the latest type of the grid is updated to ground. When the initial type of the grid is concave, the position of the edge (i.e., the edge area of the concave area) is inferred, and the map point corresponding to the position of the edge area is updated to a step candidate point. The position of the step candidate point is the position of the step area. When the initial type of the grid is convex, if the historical type of the grid is convex, the latest height of the grid is updated to the initial height of the grid determined by the current frame point cloud data. When the initial type of the grid is a convexity, if the historical type of the grid is the ground, it can be determined whether the convexity and the ground are at the same height. In this step, it can be determined whether the initial height of the grid is consistent with the historical height of the grid (i.e., whether the height difference is less than or equal to the fourth height threshold) based on the initial height of the grid determined by the current frame point cloud data; if the initial height of the grid is inconsistent with the historical height of the grid, the latest type of the grid is updated to a convexity; otherwise, the type of the grid is not updated. The specific steps for updating the map point information of the grid have been described in detail above and will not be repeated.
[0142] In the above technical solution, when the initial type of the map point is a depression, the edge area of the depression area corresponding to the map point can be more accurately determined by judging whether the initial height corresponding to the map points arranged along the preset direction in at least part of the map points jumps from large to small along the preset direction, thereby more accurately determining the step area.
[0143] According to another aspect of the embodiment of the present disclosure, a mobile robot is also provided. The mobile robot 900 includes: a processor 910 and a memory 920. The memory 920 stores a computer program, and the processor 910 is used to execute the computer program to implement the above-mentioned step recognition method. Optionally, the mobile robot can be any one of a sweeping robot, a mopping robot, a sweeping and mopping robot, a transport robot, etc.
[0144] Fig. 9 A schematic block diagram of a cleaning robot according to an embodiment of the present disclosure is shown. The cleaning robot includes a robot body, a drive motor 102, a sensor unit 103, a controller 104, a cleaning member 105, a walking unit 106, a memory 107, a communication unit 108, an interaction unit 109, an energy storage unit 110, and the like.
[0145] The sensor unit 103 provided on the robot body may include at least one of the following sensors: a radar sensor (such as a laser radar), a collision sensor, a distance sensor, a drop sensor, a counter, a gyroscope, etc. For example, the laser radar is provided on the top or the side of the robot body, and when working, it can obtain the surrounding environment information, such as the distance and angle of the obstacle relative to the laser radar. In addition, the laser radar can also be replaced by a visual sensor such as a camera, and the distance and angle of the obstacle relative to the camera can also be obtained by analyzing the obstacle in the image captured by the camera. The collision sensor, for example, includes a collision shell and a trigger sensor; when the cleaning robot collides with the obstacle through the collision shell, the collision shell moves toward the inside of the cleaning robot and compresses the elastic buffer to play a buffering role. After the collision shell moves a certain distance into the cleaning robot, the collision shell contacts the trigger sensor, and the trigger sensor is triggered to generate a signal, which can be sent to the controller 104 in the robot body for processing. After colliding with the obstacle, the cleaning robot moves away from the obstacle, and under the action of the elastic buffer, the collision shell moves back to its original position. The distance sensor can be specifically an infrared detection sensor, which can be used to detect the distance from the obstacle to the distance sensor. The distance sensor can be set on the side of the robot body, so that the distance sensor can measure the distance value from the obstacle located near the side of the cleaning robot to the distance sensor. The distance sensor can also be an ultrasonic distance sensor, a laser distance sensor or a depth sensor. The drop sensor is set at the bottom edge of the robot body. When the cleaning robot moves to the edge of the ground, the drop sensor can detect that the cleaning robot is at risk of falling from a height, so as to perform corresponding anti-fall reactions, such as the cleaning robot stops moving, or moves away from the falling position. A counter and a gyroscope are also provided inside the robot body. The counter is used to detect the distance the cleaning robot moves. The gyroscope is used to detect the angle of rotation of the cleaning robot, so as to determine the direction of the cleaning robot.
[0146] The controller 104 is disposed inside the robot body and is used to control the cleaning robot to perform specific operations. The controller 104 may be, for example, a central processing unit (CPU) or a microprocessor. Figure 7 As shown, the controller 104 is electrically connected to the energy storage unit 110, the memory 107, the drive motor 102, the walking unit 106, the sensor unit 103, the interaction unit 109 and the cleaning member 105 to control these components.
[0147] The cleaning member 105 can be used to clean the floor, and the number of the cleaning members 105 can be one or more. The cleaning member 105 includes, for example, a mop. The mop includes, for example, at least one of the following: a rotating mop, a flat mop, a roller mop, a crawler mop, etc., but is certainly not limited thereto. The mop is disposed at the bottom of the robot body, and specifically can be at the rear of the bottom of the robot body. Taking the cleaning member as a rotating mop as an example, a drive motor 102 is provided inside the robot body, two rotating shafts extend from the bottom of the robot body, and the mop is sleeved on the rotating shaft. The drive motor 102 can drive the rotating shaft to rotate, so that the rotating shaft drives the mop to rotate. The cleaning member 105 can also be a side brush, a roller brush, a roller, etc., which are not limited here.
[0148] The walking unit 106 is a component related to the movement of the cleaning robot, and the walking unit 106 includes, for example, a driving wheel and a universal wheel. The universal wheel and the driving wheel cooperate to realize the steering and movement of the cleaning robot.
[0149] The memory 107 is provided on the robot body, and a program is stored in the memory 107, and the corresponding operation is realized when the program is executed by the controller 104. The memory 107 is also used to store parameters used by the cleaning robot. The memory 107 includes but is not limited to a disk memory, a compact disc read-only memory (CD-ROM), an optical memory, etc.
[0150] The communication unit 108 is arranged on the robot body, and is used to allow the cleaning robot to communicate with external equipment, such as a terminal or a base station, wherein the base station is a cleaning equipment used in conjunction with the cleaning robot.
[0151] The interaction unit 109 is arranged on the robot body, and the user can interact with the cleaning robot through the interaction unit 109. The interaction unit 109 includes, for example, at least one of a touch screen, a switch button, a speaker, etc. For example, the user can control the cleaning robot to start or stop working by pressing the switch button.
[0152] The energy storage unit 110 is disposed inside the robot body, and is used to provide power to the cleaning robot. For example, the energy storage unit 110 is a battery.
[0153] The robot body is also provided with a charging component, which is used to obtain power from an external device (such as a base station) to charge the energy storage unit 110 of the cleaning robot.
[0154] A dust container (such as a dust box or dust bag) is also provided inside the robot body. The dust container is used to store dust and other garbage collected by the robot during the cleaning task, and when the robot returns to the base station to perform the dust collection process, the garbage is sucked into the dust container of the base station.
[0155] It should be understood that Fig. 9 The cleaning robot described in the above is only a specific example in the embodiment of the present application, and does not constitute a specific limitation on the cleaning robot in the embodiment of the present application. The cleaning robot in the embodiment of the present application may also be implemented in other specific ways. In other implementations, the cleaning robot may be more Fig. 9 The cleaning robot shown has more or fewer components; for example, the cleaning robot may include a clean water chamber for storing clean water and / or a dirt holding portion for storing dirt. The cleaning robot can transport the clean water stored in the clean water chamber to the mop and / or the ground to wet the mop, and clean the ground based on the wetted mop. The cleaning robot can also collect dirt on the ground or sewage containing dirt into the dirt holding portion; the cleaning robot can also transport the clean water stored in the clean water chamber to the mop to clean the mop, and the sewage containing dirt after cleaning the mop can also be transported to the dirt holding portion.
[0156] According to another aspect of the embodiments of the present disclosure, a cleaning system is also provided. The cleaning system includes the above-mentioned mobile robot and a base station used in conjunction with the mobile robot, and the mobile robot is a cleaning robot. Optionally, the cleaning robot can be any one of a sweeping robot, a mopping robot, a sweeping and mopping robot, etc. The cleaning robot is maintained in the base station, and the maintenance includes but is not limited to: charging, dust collection, cleaning of cleaning parts, replenishing fresh water, pumping sewage, etc.
[0157] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is also provided. The storage medium stores a computer program / instruction, and the computer program / instruction implements the above-mentioned step identification method when executed by the processor. The storage medium may include, for example, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.
[0158] A person skilled in the art can easily understand the implementation structure, working principle and beneficial effects of the mobile robot, cleaning system and computer-readable storage medium by reading the above step recognition method. For the sake of brevity, it will not be described in detail here.
[0159] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present disclosure to this. A person of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present disclosure. All such changes and modifications are intended to be included within the scope of the present disclosure as required by the appended claims.
[0160] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.
[0161] In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
[0162] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0163] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various disclosed aspects, in the description of the exemplary embodiments of the present disclosure, the various features of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present disclosure should not be interpreted as reflecting the following intention: the claimed disclosure requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the point of disclosure is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present disclosure.
[0164] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.
[0165] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present disclosure and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.
[0166] The various component embodiments of the present disclosure can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules in the mobile robot and cleaning system according to the embodiments of the present disclosure. The present disclosure can also be implemented as a device program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present disclosure can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0167] It should be noted that the above embodiments illustrate the present disclosure rather than limit the present disclosure, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present disclosure may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.
[0168] The above description is only a specific embodiment of the present disclosure or an explanation of a specific embodiment. The protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. The protection scope of the present disclosure shall be based on the protection scope of the claims.
Claims
1. A step recognition method, applied to a mobile robot, characterized in that: include: Acquire point cloud data, wherein the point cloud data includes height information of a measured point, the measured point being a point located outside the shell of the mobile robot and within a preset distance from a projection center of the mobile robot, and the height information is used to indicate the height of the measured point relative to a robot plane, the robot plane being a plane where a point of the mobile robot in contact with the ground is located; Projecting the point cloud data into a target map to obtain projection points, wherein the target map is a map of a preset area where the mobile robot is located, and the projection points correspond one-to-one to the measured points in the point cloud data; Based on the projection point in the target map, the position of the step in the target map is determined.
2. The step recognition method according to claim 1, characterized in that: The step of determining the position of the step in the target map based on the projection point in the target map comprises: Based on the projection point, updating the map point information of the map point in the target map, wherein the map point information includes at least one of the height and type of the map point and the number of projection points contained therein; The position of the step in the target map is determined based at least on the map point information of the map point in the target map.
3. The step recognition method according to claim 2, characterized in that: The updating of the map point information of the map point in the target map based on the projection point includes: Based on the projection point and its height information, updating the map point information of the map point in the target map, wherein the height information of the projection point is the height information of the corresponding measured point; The determining the position of the step in the target map based at least on the map point information of the map point in the target map comprises: Determining a step candidate point in the target map based on the height of the map point in the target map and the number of projection points it contains; It is determined whether the step candidate point belongs to a step area to determine the position of the step in the target map, and the step area is the area where the step is located.
4. The step recognition method according to claim 3, characterized in that: The step of determining the step candidate point in the target map based on the height of the map point in the target map and the number of projection points contained therein comprises: Determine a map point in the target map whose height is less than or equal to a first height threshold as a target map point; Determine a target map point whose number of included projection points is greater than or equal to a preset number threshold as the step candidate point; or, Determine a map point in the target map whose number of projection points is greater than or equal to a preset number threshold as a target map point; A target map point whose height is less than or equal to a first height threshold among the target map points is determined as the step candidate point.
5. The step recognition method according to claim 3, characterized in that: The determining whether the step candidate point belongs to a step area includes: Based on the heights of the step candidate points, at least one to-be-processed area is determined in the target map, each of the to-be-processed areas being an area in which the height difference between the heights of the included map points and the heights of at least some of the step candidate points is less than or equal to a second height threshold; For each of the at least one to-be-processed area, Determine whether the area to be processed meets the first target requirement; When the area to be processed meets the first target requirement, it is determined that the step candidate point in the area to be processed belongs to the step area.
6. The step recognition method according to claim 5, characterized in that: The point cloud data is the current frame point cloud data; Before determining whether the area to be processed meets the first target requirement, determining whether the step candidate point belongs to the step area further includes: For each of the at least one to-be-processed area, Determine whether there is a historical step area within a preset range centered on the step candidate point in the area to be processed, wherein the historical step area is a step area determined based on at least one frame of point cloud data before the current frame of point cloud data; When the historical step region exists, determining whether the consistency between the region to be processed and the historical step region meets the second objective requirement; When the consistency between the region to be processed and the historical step region meets the second target requirement, it is determined that the step candidate point in the region to be processed belongs to the step region.
7. The step recognition method according to claim 6, characterized in that: The second target requirement includes: the height difference between the height of the area to be processed and the height of the historical step area is less than or equal to a third height threshold, and the angle between the height extension direction of the area to be processed and the height extension direction of the historical step area is less than or equal to a preset angle threshold; wherein the heights of the area to be processed and the historical step area are determined based on the heights of the map points they each contain, and the height extension direction is a direction perpendicular to a straight line obtained by fitting the step candidate point and the step candidate point.
8. The step recognition method according to claim 5, characterized in that: The first target requirement includes: the length of the area to be processed is greater than a length threshold; and, the width of the area to be processed is greater than a width threshold; and, the area of the area to be processed is greater than an area threshold; and / or, The determining at least one area to be processed in the target map based on the height of the step candidate point includes: Each of the determined step candidate points is used as a seed point, and a growth area adjacent to the seed point and having a height difference with the seed point less than or equal to the second height threshold is determined in the target map using the region growing method to obtain the at least one area to be processed.
9. The step recognition method according to any one of claims 2 to 8, characterized in that: The point cloud data is the current frame point cloud data, and the projection point corresponding to the current frame point cloud data is the current projection point; and updating the map point information of the map point in the target map based on the projection point includes: Based on the current projection point, determining initial map point information corresponding to the current projection point for each map point in the target map, the initial map point information at least including an initial type and an initial height of the map point, the initial type including ground and convex; For each map point in the target map, When the initial type of the map point is convex or ground, comparing the initial map point information of the map point with the historical map point information of the map point; Determining the latest map point information of the map point based on a comparison result of the initial map point information of the map point and the historical map point information of the map point, so as to update the map point information of the map point; Among them, the historical map point information of each map point in the target map is map point information determined based on historical projection points, and the historical projection points are each projection point corresponding to each frame of point cloud data in at least one frame of point cloud data before the current frame of point cloud data. The historical map point information includes the historical type and historical height of the map point, and the historical type includes ground and protrusion.
10. The step recognition method according to claim 9, characterized in that: The historical map point information is first historical map point information determined based on first historical projection points, the first historical projection points are a group of projection points corresponding to a frame of point cloud data before the current frame of point cloud data, the first map point information includes a first historical type and a first historical height of the map point, the first historical type includes ground and protrusion; The determining the latest map point information of the map point based on the comparison result of the initial map point information of the map point and the historical map point information of the map point includes: When the initial type of the map point is ground and the first history type is convex, determining the latest type of the map point is ground, and determining the latest height of the map point as the initial height of the map point; and / or, When the initial type of the map point is ground and the first historical type is ground, maintaining the latest type of the map point as ground, and determining the latest height of the map point as the initial height of the map point; and / or, When the initial type of the map point is raised and the first history type is ground, Determine whether a height difference between an initial height of the map point and a first historical height of the map point is greater than or equal to a fourth height threshold; If the height difference between the initial height of the map point and the first historical height of the map point is greater than or equal to the fourth height threshold, determining the latest type of the map point is a protrusion, and determining the latest height of the map point as the initial height of the map point; If the height difference between the initial height of the map point and the first historical height of the map point is less than the fourth height threshold, keeping the latest type of the map point as ground; and / or, When the initial type of the map point is raised and the historical type is raised, the latest type of the map point is maintained as raised, and the latest height of the map point is determined as the initial height of the map point.
11. The step recognition method according to claim 9, characterized in that: The historical map point information is second historical map point information determined based on second historical projection points, the second historical projection points are each group of projection points corresponding to each frame of point cloud data in at least two frames of point cloud data before the current frame of point cloud data, the second historical map point information includes a second historical type and a second historical height of the map point, the second historical type includes ground and protrusion; The determining the latest map point information of the map point based on the comparison result of the initial map point information of the map point and the historical map point information of the map point includes: When the initial type of the map point is ground and the second historical type is raised, performing a type determination operation to determine a latest type of the map point, and performing a height determination operation to determine a latest height of the map point; and / or, When the initial type of the map point is ground and the second historical type is ground, maintaining the latest type of the map point as ground; and / or, When the initial type of the map point is raised and the second history type is ground, Determine whether a height difference between an initial height of the map point and a second historical height of the map point is greater than or equal to a fourth height threshold; If the height difference between the initial height of the map point and the second historical height of the map point is greater than or equal to the fourth height threshold, performing the type determination operation to determine the latest type of the map point, and performing the height determination operation to determine the latest height of the map point; If the height difference between the initial height of the map point and the second historical height of the map point is less than the fourth height threshold, keeping the latest type of the map point as ground; and / or, When the initial type of the map point is convex and the second historical type is convex, maintaining the latest type of the map point as convex, and performing the height determination operation to determine the latest height of the map point; The type determination operation includes: For each map point in the target map, determining the latest type of the map point based on a preset number of initial types of the map point, the preset number of initial types corresponding to a preset number of point cloud data, the preset number of point cloud data including the current frame point cloud data and the at least two frames of point cloud data before the current frame point cloud data, each initial type being the initial type of the map point determined based on the corresponding frame point cloud data; The height determination operation includes: For each map point in the target map, the latest height of the map point is determined based on a preset number of initial heights of the map point, the preset number of initial heights corresponds one-to-one to a preset number of point cloud data, the preset number of point cloud data includes the current frame point cloud data and the at least two frames of point cloud data before the current frame point cloud data, and each initial height is the initial height of the map point determined based on the corresponding frame point cloud data.
12. The step recognition method according to any one of claims 1 to 8, characterized in that: The point cloud data is the current frame point cloud data, and the projection point corresponding to the current frame point cloud data is the current projection point; The step of determining the position of the step in the target map based on the projection point in the target map comprises: Based on the current projection point, determining initial map point information corresponding to the current projection point for each map point in the target map, the initial map point information comprising at least an initial type and an initial height of the map point, the initial type comprising a depression; Based on the initial map point information of at least some of the map points in the target map, determining the edge area of the sunken area, wherein the sunken area is the area where the map points of the initial type belonging to the sunken area are located; It is determined that the edge area belongs to a step area to determine the position of the step in the target map, and the step area is the area where the step is located.
13. The step recognition method according to claim 12, characterized in that: The at least some of the map points are map points that include current projection points when the current frame point cloud data is projected into the target map; The determining of the edge area of the sunken area based on the initial map point information of at least part of the map points in the target map comprises: Based on the initial map point information of at least some of the map points in the target map, determining whether the initial heights corresponding to the map points arranged along a preset direction in the at least some of the map points jump from large to small along the preset direction, and the jump is that the height difference between the initial heights of two adjacent map points is greater than or equal to a fifth height threshold; When the initial heights corresponding to the map points arranged along the preset direction jump from large to small along the preset direction, it is determined that the map points with larger initial heights among the map points that have jumped belong to the edge area.
14. The step recognition method according to claim 1, characterized in that: After determining the position of the step in the target map based on the projection point in the target map, the method further includes: Based on the projection point and the determined position of the step, determining an equal-height area after the mobile robot climbs onto the step, wherein the height of each map point in the equal-height area is within a preset height range; In the target map, an area of equal height after climbing the steps is marked as ground to update the target map.
15. A mobile robot, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the step recognition method according to any one of claims 1 to 14.
16. A cleaning system, characterized in that: It comprises the mobile robot as claimed in claim 15 and a base station used in conjunction with the mobile robot, wherein the mobile robot is a cleaning robot, and the cleaning robot performs maintenance in the base station.
17. A computer-readable storage medium, characterized in that: A computer program / instruction is stored, and when the computer program / instruction is executed by one or more processors, the step recognition method according to any one of claims 1 to 14 is implemented.
