Robot rgbd-based down stair detection and avoidance method
By filtering and clustering RGBD point cloud data, the robot can accurately detect and avoid descending stairs, solving the problems of inaccurate detection and high cost in existing technologies, and achieving more efficient motion planning and safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-02
- Publication Date
- 2026-04-07
AI Technical Summary
When robots encounter descending stairs, existing technologies struggle to accurately detect and avoid them, especially under conditions of light and brightness. Conventional methods can easily lead to delayed braking, and infrared sensors are expensive.
A filtering and clustering method based on RGBD point cloud data is adopted to extract effective point cloud data. Non-planar data is filtered by the Z-value of the normal vector, and regional clustering and plane fitting are performed to select horizontal plane clusters. The avoidance treatment is determined based on the number of ground clusters.
It improves the accuracy of downhill stair detection and the intelligence of the robot, reduces sensor costs, avoids the risk of robot falls, and enhances the accuracy of motion planning.
Smart Images
Figure CN116091407B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent recognition, specifically to a method for detecting and avoiding descending stairs for robots based on RGBD. Background Technology
[0002] Currently, robots can easily detect and avoid obstacles above ground level during autonomous driving using sensors. However, for obstacles below ground level, especially descending stairs, detection typically relies on image and infrared sensors. Regarding image processing, image detection algorithms are often used to detect the edges of the stairs and define the stair area. However, this is difficult to achieve due to variations in lighting and brightness. As for adding sensors, infrared sensors are used for point-to-area detection, which cannot detect the entire staircase, potentially leading to delayed braking. Furthermore, infrared sensors are expensive.
[0003] Therefore, this invention addresses the problem of robots encountering descending stairs by proposing a descending stair detection method based on RGBD point cloud information, and proposes an avoidance strategy for the robot's motion planning when encountering descending stairs. Summary of the Invention
[0004] In view of the above problems, this application provides a robot-based RGBD method for detecting and avoiding descending stairs, which improves the robot's intelligence in dealing with descending stairs by increasing the detection accuracy of descending stairs.
[0005] To achieve the above objectives, the inventors provide a robot-based RGBD method for detecting and avoiding descending stairs, comprising:
[0006] Acquire RGBD point cloud data in the robot coordinate system;
[0007] The RGBD point cloud data is filtered to obtain valid point cloud data that includes the descending staircase.
[0008] For point cloud data in which the normal vector Z value is greater than the preset point cloud Z value, perform clustering processing based on region growing to obtain a region cluster set;
[0009] Filter out the non-planar point cloud data in the region cluster set to obtain a planar cluster set;
[0010] Whether or not avoidance processing needs to be performed is determined based on the number of ground clusters in the planar clustering set.
[0011] Unlike existing technologies, the above solution is based on RGBD point cloud data. It extracts valid point cloud data containing information about descending stairs through filtering. Then, it filters out non-horizontal point cloud data such as railings and walls based on the Z-value of the normal vector. Next, it groups similar point cloud data together to obtain regional clusters. Then, it filters out point cloud data belonging to slopes or stair step shadows, selecting horizontal plane clusters. Finally, it determines the number of ground clusters based on the horizontal plane clusters, thus determining whether avoidance is necessary. Therefore, this solution can avoid the influence of light, improve the accuracy of detecting descending stairs, thereby enhancing the robot's intelligence in dealing with descending stairs and preventing falls; at the same time, it reduces the cost of deploying sensors.
[0012] In some embodiments, determining whether to perform avoidance processing based on the number of ground clusters in the planar clustering set includes:
[0013] Calculate the average point cloud height of each planar cluster in the planar cluster set, and merge the planar clusters according to the average point cloud height to obtain a new cluster set;
[0014] Calculate the average new point cloud height of each new planar cluster in the new cluster set, and determine the number of ground clusters contained in the new cluster set based on the average new point cloud height;
[0015] Based on the number of ground clusters included, determine whether avoidance processing is required.
[0016] In some embodiments, merging planar clusters based on the average point cloud height to obtain a new cluster set includes:
[0017] Based on the average point cloud height, each planar cluster is sorted from highest to lowest.
[0018] Calculate the mean difference in point cloud height between adjacent planar clusters in the sorted planar cluster set;
[0019] Merge adjacent planar clusters whose mean point cloud height difference is less than a preset difference to obtain a new cluster set.
[0020] In some embodiments, determining whether to perform avoidance processing based on the number of included ground clusters includes:
[0021] If no ground clusters exist, publish the guardrail point cloud to the navigation system;
[0022] If a ground cluster exists, the stair cluster in the new cluster set is published to the navigation system.
[0023] In some embodiments, prior to performing the avoidance process, the following is included:
[0024] The point cloud height values of each point cloud within each new planar cluster are normalized to the corresponding new point cloud height mean.
[0025] In some embodiments, after performing region-growing-based clustering processing on the point cloud data whose normal vector Z values are greater than a preset Z value threshold, the method further includes:
[0026] If a regional cluster cannot be obtained, an avoidance process is performed.
[0027] In some embodiments, after filtering out the non-planar point cloud data in the region cluster set, the method further includes:
[0028] If a planar cluster cannot be obtained, an avoidance process is performed.
[0029] In some embodiments, the filtering process on the RGBD point cloud data includes:
[0030] Filter out redundant data in the RGBD point cloud data to obtain valid point cloud data;
[0031] By using pass-through filtering, point cloud data above the ground in the valid point cloud data are filtered out.
[0032] In some embodiments, after filtering out point cloud data above the ground from the valid point cloud data, the method further includes:
[0033] Isolated points in the point cloud data obtained after the pass-through filtering process are filtered out to obtain valid point cloud data that includes the descending staircase.
[0034] In some embodiments, filtering out non-planar point cloud data from the regional cluster set to obtain a planar cluster set includes:
[0035] Perform plane fitting on each region cluster in the region cluster set;
[0036] Calculate the plane normal vector corresponding to each region cluster after the plane fitting;
[0037] Obtain planar clusters whose plane normal vector Z-values are greater than a preset plane Z-value threshold.
[0038] The above embodiments can further improve the detection accuracy of descending stairs, providing more powerful and accurate decision support for motion planning on descending stairs, thereby enhancing the overall intelligence of the robot.
[0039] The above description of the invention is merely an overview of the technical solution of this application. In order to enable those skilled in the art to better understand the technical solution of this application and to implement it based on the description and drawings, and to make the above-mentioned objectives and other objectives, features and advantages of this application easier to understand, the following description is provided in conjunction with the specific embodiments and drawings of this application. Attached Figure Description
[0040] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, features, and effects of specific embodiments of this application and other related content, and should not be considered as limitations on this application.
[0041] In the accompanying drawings of the instruction manual:
[0042] Figure 1 This is a flowchart illustrating an embodiment of a robot-based RGBD method for detecting and avoiding descending stairs;
[0043] Figure 2 This is a schematic diagram illustrating the filtering process described in a specific embodiment;
[0044] Figure 3 This is a schematic diagram illustrating the process of obtaining a planar cluster set according to a specific embodiment;
[0045] Figure 4 This is a schematic diagram illustrating the specific process of step S5 in a specific embodiment;
[0046] Figure 5 This is a flowchart illustrating a robot-based RGBD-based method for detecting and avoiding descending stairs, according to a specific implementation.
[0047] Figure 6 This is a schematic diagram showing the visible range of the point cloud based on the depth camera;
[0048] Figure 7 This is a schematic diagram illustrating the point cloud of the guardrail in a specific implementation. Detailed Implementation
[0049] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described in this document are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.
[0050] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there is no technical contradiction or conflict, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.
[0051] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing specific embodiments of 115 and is not intended to limit this application.
[0052] In the description of this application, the term "and / or" is a statement used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and...
[0053] All three cases, A and B, exist simultaneously. Additionally, in this article, the character " / " generally indicates an "OR" logical relationship between the preceding and following objects.
[0054] 120 In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order between these entities or operations.
[0055] Without further limitations, the use of terms such as “comprising,” “including,” “having,” or other similar expressions in this application is intended to cover non-exclusive inclusion, and these expressions do not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a series of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.
[0056] Similar to the understanding in the Examination Guidelines, in this application, expressions such as "greater than," "less than," "exceeding," and "130" are understood to exclude the number itself; expressions such as "above," "below," and "within" are understood to include the number itself. Furthermore, in the description of the embodiments in this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups," "multiple times," etc., unless otherwise explicitly specified.
[0057] In the description of the embodiments of this application, the space-related expressions used, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "vertical," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," indicate the orientation or positional relationship based on the orientation or positional relationship shown in the specific embodiments or drawings. They are only for the purpose of describing the specific embodiments of this application or for the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.
[0058] Unless otherwise expressly specified or limited, the terms "installation," "connection," "linking," "fixing," and "setting," as used in the description of the embodiments of this application, should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two components or the interaction between two components. For those skilled in the art to which this application pertains, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.
[0059] Please see Figures 1-4 , Figure 1 This is a schematic flowchart illustrating an embodiment of a robot-based RGBD-based method for detecting and avoiding descending stairs. Figure 1 As shown, this embodiment provides a robot-based RGBD-based method for detecting and avoiding descending stairs, including the following steps:
[0060] S1: Obtain RGBD point cloud data in the robot coordinate system;
[0061] Specifically, after periodically receiving RGBD point cloud data from the depth camera, the RGBD point cloud data based on the camera coordinate system will be converted into RGBD point cloud data based on the robot coordinate system.
[0062] S2: Filter the RGBD point cloud data to obtain valid point cloud data that includes the descending staircase;
[0063] Here, filtering will be used to remove invalid data from the RGBD point cloud data that is affected by the on-site environment, such as a large amount of redundant or messy data, thereby improving the efficiency of point cloud data analysis and processing and the accuracy of the results. At the same time, point cloud data that does not contain the descending stairs will also be filtered out, and only the target point cloud data 160 will be processed.
[0064] For some specific implementation methods, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the filtering process described in a specific embodiment, specifically a flowchart of step S2. Figure 2 As shown, the filtering process includes:
[0065] S21: Redundant data in the RGBD point cloud data is filtered out by voxel filtering to obtain effective point cloud data; by removing a large amount of redundant data, the calculation speed of point cloud data can be significantly improved.
[0066] S22: By using a pass-through filter, point cloud data above the ground level is filtered out from the valid point cloud data. This pass-through filter effectively preserves the point cloud data of the descending staircase.
[0067] In some preferred embodiments, the filtering process further includes:
[0068] S23: Filter out isolated points in the point cloud data obtained after the pass-through filtering process to obtain valid point cloud data that includes the descending staircase. Statistical filtering or radius filtering can be used to effectively remove outliers from the point cloud data.
[0069] S3: Perform region-growing-based clustering processing on the point cloud data in which the normal vector Z value is greater than the preset point cloud Z value to obtain the region cluster set;
[0070] Specifically, firstly, among the 175 valid point cloud data points containing the descending staircase obtained after filtering, there may be point cloud data points that are not on the staircase plane, such as those of railings and walls. Therefore, it is necessary to extract the point cloud data of the staircase plane. Here, the characteristics of the normal vector of the horizontal point cloud will be considered, i.e., the more vertically upward the normal vector, the closer the value on the Z-axis is to 1; by calculating the Z-value of the normal vector in the point cloud data, point cloud data points with a normal vector Z-value greater than a preset point cloud Z-value will be retained as points on the horizontal plane.
[0071] Then, a region-growing-based clustering method is applied to the point cloud data on the horizontal plane to group point clouds with 180% similarity (i.e., small differences in point cloud normal vectors) to form a region cluster, thereby obtaining multiple clusters and their corresponding point cloud data within each cluster, i.e., a set of region clusters.
[0072] If at least one region cluster cannot be obtained after clustering, it indicates that there is a high probability that there is no "road surface" in front of the robot. In this emergency, the guardrail point cloud is directly output to prevent the robot from falling.
[0073] S4: Filter out the non-planar point cloud data in the region cluster set to obtain a planar cluster set;
[0074] The resulting clusters may contain point cloud data belonging to slope trails or stair step trails. The purpose of this step is to remove these trail point cloud data and filter out the clusters of the stair horizontal plane.
[0075] In some specific implementations, plane fitting can be used to determine the value of the plane normal vector on the Z-axis to achieve the purpose of removal. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a flowchart illustrating the process of obtaining planar cluster sets according to a specific embodiment, specifically a flowchart of step S4. Figure 3 As shown, step S4 specifically includes the following process:
[0076] S41: Perform plane fitting on each region cluster in the region cluster set;
[0077] S42: Calculate the plane normal vector corresponding to each region cluster after the plane fitting;
[0078] S43: Obtain the plane cluster set whose plane normal vector Z value is greater than the preset plane Z value threshold.
[0079] It should also be noted that if a planar cluster set cannot be obtained in this step, an avoidance process will be performed.
[0080] S5: Determine whether avoidance processing is required based on the number of ground clusters in the planar clustering set.
[0081] Here, by determining the number of ground clusters contained in the planar cluster set, it is possible to determine whether there is ground "in front" of the robot, and the distance between the robot and the descending stairs can be roughly estimated by the number of ground clusters, thus providing decision support for subsequent precise motion planning.
[0082] Therefore, in this step, based on the number of ground clusters, the following steps are performed:
[0083] If no ground clusters exist, publish the guardrail point cloud to the navigation system so that the navigation system can set obstacles based on the guardrail point cloud;
[0084] If ground clusters exist, the stair clusters in the new clusters are published to the navigation system so that the navigation system can set obstacles and perform deceleration and motion planning based on the point cloud data in the stair clusters.
[0085] In some specific implementations, planar clusters are merged based on the average height of the point cloud to obtain new clusters, thereby avoiding the problem of height differences between point cloud data of the same step in the actual scene, thus unifying the point cloud data of each step and improving the accuracy of downhill stair detection.
[0086] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating the specific process of step S5 in a specific implementation.
[0087] like Figure 4 As shown, step S5 specifically includes:
[0088] 215S51: Calculate the average point cloud height of each planar cluster in the planar cluster set, and merge the planar clusters based on the average point cloud height, i.e., the degree of variation in the height difference of the average point cloud, to obtain a new cluster set. This achieves the merging of point cloud data of the same step based on the average point cloud height.
[0089] In some preferred embodiments, this step specifically includes: 220 First, sorting each planar cluster from high to low according to the mean point cloud height; then calculating the difference in mean point cloud height between adjacent planar clusters in the sorted planar cluster set;
[0090] Finally, adjacent planar clusters whose mean point cloud height difference is less than a preset difference are merged to obtain a new cluster set.
[0091] S52: Calculate the average new point cloud height of each new planar cluster in the new cluster set, and determine the number of ground clusters contained in the new cluster set based on the average new point cloud height; that is, determine whether there are ground clusters and the number of ground clusters based on the average new point cloud height.
[0092] S53: Determine whether avoidance processing is required based on the number of ground clusters included.
[0093] In some preferred embodiments, after obtaining the new cluster set in S51, the following will also be performed simultaneously in 230: the point cloud height values of each point cloud in each new planar cluster will be normalized to the corresponding new point cloud height average value, so as to unify the point cloud height values of each point cloud in each new planar cluster, so as to facilitate the subsequent navigation system to perform motion planning based on the new planar cluster.
[0094] Please see Figure 5 and Figure 6 , Figure 5 This is a flowchart illustrating a robot-based RGBD-based method for detecting and avoiding descending stairs, according to a specific implementation. Figure 6 This is a schematic diagram showing the visible range of the point cloud based on the depth camera.
[0095] In this specific embodiment, the RGBD depth point cloud is filtered, extracted from the plane, and clustered. Then, the stair plane is extracted by plane fitting and mean normalization. Based on the stair plane area, the concept of a point cloud "guardrail" is proposed to achieve accurate detection and intelligent avoidance of the robot going down the stairs.
[0096] Specifically, the robot-based RGBD-based method for detecting and avoiding descending stairs in this embodiment includes the following steps:
[0097] 1. Based on the position and angle of the depth camera mounted on the robot, as well as the camera's lateral and longitudinal field of view, calculate the visible range of the point cloud on the ground. The visible range of the point cloud is as follows: Figure 3 As shown. Here, scattered point cloud data that is not within the visible range is not processed.
[0098] 2. Monitor the data topic name ( / camera / depth / points) and periodically receive RGBD point cloud data sent by the depth camera; combine the position translation and rotation relationship between the depth camera and the robot coordinate system to transform the received RGBD point cloud data pose based on the camera coordinate system to the RGBD point cloud data pose based on the robot base coordinate system.
[0099] 3. Due to the influence of the on-site environment, the received RGBD point cloud data will contain a large amount of redundant and messy point cloud data, which will seriously affect the detection and extraction of the stair plane. Therefore, it is necessary to use a voxel filter, a pass-through filter, and an outlier filter in sequence for filtering. The main steps of the filtering process are as follows:
[0100] 3-1: Use a voxel filter to spatially mesh (cube) the point cloud data, selecting the optimal point within each grid as a representative to eliminate a large amount of redundant data and improve computation speed. To preserve the shape characteristics of the point cloud, it is preferable to set the grid size to 0.02m.
[0101] 3-2: Referring to the right-hand rule, establish a spatial coordinate system based on the robot's coordinate system. Set the robot's front as the positive X-axis, the leftward perpendicular to the X-axis as the positive Y-axis, and the upward perpendicular to the ground as the positive Z-axis. Then, use a pass-through filter to filter out point cloud data above the ground along the Z-axis, retaining the remaining point cloud data. This preserves the detected downward staircase point cloud data. Preferably, due to hardware limitations of the depth camera, point cloud data within a certain distance is retained along the X-axis. Preferably, the closest distance parameter along the X-axis is 0.05m, and the farthest distance parameter is 3.5m. Preferably, the highest height along the Z-axis is -0.26m, and the lowest height is -5.0m.
[0102] 3-3: Point cloud data after voxel filtering and pass-through filtering may contain many isolated points. An outlier filter is used to count the number of points within a specified radius to remove noise points that do not meet the filtering criteria. Preferably, the specified radius is set to 0.1m, and the count is 40.
[0103] 4. The filtered point cloud retains the point cloud data for the descending staircase, but it also contains data from non-staircase planes, such as railings and walls. To extract the point cloud data for the staircase plane, the following steps will be performed:
[0104] 4-1: Estimate the normal vector of the point cloud based on the vector difference between the point cloud and other point clouds within a specified radius, and obtain the normalized normal vector.
[0105] 4-2: Considering the characteristics of the normal vector in horizontal point clouds—that is, the more vertically upward the normal vector, the closer its value on the Z-axis is to 1—a threshold will be set on the Z-axis of the normal vector. Point cloud data with values greater than the threshold will be retained as points on the horizontal plane. Preferably, the parameters are set to a specified radius of 0.05m and a Z-axis threshold of 0.9.
[0106] 5. A region-growing-based clustering method is used to group point clouds with similarity (small differences in point cloud normal vectors) into region clusters, obtaining multiple clusters and their corresponding intra-cluster point cloud data. Preferably, the minimum number of intra-cluster point clouds is 40, and the maximum is 100,000; preferably, the number of neighborhood searches is 20; preferably, the normal vector difference threshold is (Pi / 30≈0.10472); preferably, the covariance threshold is 0.1.
[0107] 6. Each clustered point cloud has a certain probability of being a point cloud trail representing a slope or stair step. Therefore, by performing planar fitting on each cluster and determining the value of the plane normal vector on the Z-axis, clusters representing the horizontal plane of the staircase can be selected. Then, the mean point cloud height of the selected clusters is calculated for normalization. Preferably, the parameters are set to 100 clustering iterations, a clustering distance of 0.03, and a Z-axis threshold of 0.8 for the plane normal vector.
[0108] 7. Due to the influence of the actual scene environment, there are certain height differences between the point cloud data of the same step. To obtain the horizontal point cloud of the same step, the average height of the point cloud will be merged and normalized by clustering. The specific steps are as follows:
[0109] 7-1: Sort the clusters according to their mean height, from highest to lowest.
[0110] 7-2: Based on the degree of variation in the height difference of the mean, the two clusters with differences less than the difference threshold are merged to obtain the merged step point cloud data of each layer;
[0111] 2907-3: Calculate the mean height of the merged point cloud clusters and normalize the point cloud height data within the clusters to the mean height;
[0112] 7-4: Based on the merged height data, arrange the step data sequentially, obtain the point cloud data of each step, and publish it through the data topic name ( / stair_rgbd_output) so that the robot can use it for navigation.
[0113] 2958. To effectively address the problem of robots falling when they fail to avoid descending stairs, a "guardrail point cloud" avoidance strategy is proposed based on the detected point cloud data of the ground and the horizontal plane of the stairs. The specific execution steps are as follows:
[0114] 8-1: Combining the field of view of the depth camera on the ground and considering the robot's speed, the guardrail point cloud ( / safe_rgbd_output) is set as follows: Figure 7 As shown, Figure 7 This is a schematic diagram illustrating the point cloud of the 300 guardrail in a specific implementation.
[0115] 8-2: Determine if a regional cluster exists. If no regional cluster can be obtained, immediately stop the robot's movement and output the guardrail point cloud to the navigation system.
[0116] Determine if ground clustering exists. If not, it indicates that the area is too close to the stairs. Issue an emergency guardrail point cloud to the navigation system, and initiate an emergency stop and route replanning.
[0117] If 305 exists and the number is greater than 1, it means that it is on the ground and there is still a certain distance from the stairs. The guardrail point cloud is not published to the navigation system. The navigation system sets obstacles based on the guardrail point cloud data and performs deceleration and operation planning.
[0118] If it exists and the quantity is equal to 1, then it is close to the stairs, and the guardrail point cloud is published to the navigation system.
[0119] In addition, this application also provides a robot 310 based on the aforementioned method for detecting and avoiding descending stairs, which includes a memory and a processor; the memory is configured to store a computer program, and the processor is configured to execute the computer program to implement the RGBD-based method for detecting and avoiding descending stairs for the robot in any of the above embodiments.
[0120] In some specific implementations, the robot can be a service robot, such as a sweeping robot or a butler robot; it can also be an industrial robot, such as a robot responsible for moving things; and of course, it can also be a special purpose robot, such as an exhibition hall robot.
[0121] This application provides a robot-based method for detecting and avoiding descending stairs using RGBD, as well as a robot, which has at least the following advantages:
[0122] (1) To address the shortcomings of conventional RGB images under the influence of light, and improve the success rate and accuracy of the robot in detecting descending stairs;
[0123] (2) Reduce costs and decrease the deployment of sensors;
[0124] (3) Improve robot driving safety, reduce the risk of robots falling when encountering stairs, and improve the overall intelligence of robots;
[0125] (4) It is simple to implement, easy to deploy and promote, and has long-term benefits.
[0126] Finally, it should be noted that although the above embodiments have been described in the text and drawings of this application, this should not limit the scope of patent protection of this application. Any technical solutions that are based on the essential concept of this application and utilize the content described in the text and drawings of this application, resulting in equivalent structural or procedural substitutions or modifications, as well as the direct or indirect application of the technical solutions of the above embodiments to other related technical fields, are all included within the scope of patent protection of this application.
Claims
1. A method for detecting and avoiding descending stairs using RGBD-based algorithms for robots, characterized in that, include: Acquire RGBD point cloud data in the robot coordinate system; The RGBD point cloud data is filtered to obtain valid point cloud data that includes the descending staircase. For point cloud data in which the normal vector Z value is greater than the preset point cloud Z value, perform clustering processing based on region growing to obtain a region cluster set; Filter out the non-planar point cloud data in the region cluster set to obtain a planar cluster set; Whether or not avoidance processing needs to be performed is determined based on the number of ground clusters in the planar clustering set; The step of determining whether to perform avoidance processing based on the number of ground clusters in the planar clustering cluster set includes: Calculate the average point cloud height of each planar cluster in the planar cluster set, and merge the planar clusters according to the average point cloud height to obtain a new cluster set; Calculate the average new point cloud height of each new planar cluster in the new cluster set, and determine the number of ground clusters contained in the new cluster set based on the average new point cloud height; Based on the number of ground clusters included, determine whether avoidance processing is required.
2. The method for robot detection and avoidance of descending stairs based on RGBD as described in claim 1, characterized in that, The step of merging planar clusters based on the average point cloud height to obtain a new cluster set includes: Based on the average point cloud height, each planar cluster is sorted from highest to lowest. Calculate the mean difference in point cloud height between adjacent planar clusters in the sorted planar cluster set; Merge adjacent planar clusters whose mean point cloud height difference is less than a preset difference to obtain a new cluster set.
3. The robot-based RGBD-based method for detecting and avoiding descending stairs as described in claim 1, characterized in that, The determination of whether to perform avoidance processing based on the number of ground clusters included includes: If no ground clusters exist, publish the guardrail point cloud to the navigation system; If a ground cluster exists, the stair cluster in the new cluster set is published to the navigation system.
4. The robot-based RGBD-based method for detecting and avoiding descending stairs as described in claim 3, characterized in that, Prior to performing the avoidance process, the following are included: The point cloud height values of each point cloud within each new planar cluster are normalized to the corresponding new point cloud height mean.
5. The robot-based RGBD-based method for detecting and avoiding descending stairs as described in claim 1, characterized in that, After performing region-growing-based clustering on the point cloud data whose normal vector Z value is greater than a preset Z value threshold, the process further includes: If a regional cluster cannot be obtained, an avoidance process is performed.
6. The method for robot detection and avoidance of descending stairs based on RGBD as described in claim 1, characterized in that, After filtering out the non-planar point cloud data in the clustered regions, the process further includes: If a planar cluster cannot be obtained, an avoidance process is performed.
7. The robot-based RGBD-based method for detecting and avoiding descending stairs as described in claim 1, characterized in that, The filtering process for the RGBD point cloud data includes: Filter out redundant data in the RGBD point cloud data to obtain valid point cloud data; By using pass-through filtering, point cloud data above the ground in the valid point cloud data are filtered out.
8. The robot-based RGBD-based method for detecting and avoiding descending stairs as described in claim 7, characterized in that, After filtering out point cloud data above the ground from the valid point cloud data, the process further includes: Isolated points in the point cloud data obtained after the pass-through filtering process are filtered out to obtain valid point cloud data that includes the descending staircase.
9. A robot-based RGBD-based method for detecting and avoiding descending stairs as described in claim 1, characterized in that, The step of filtering out non-planar point cloud data from the regional cluster sets to obtain planar cluster sets includes: Perform plane fitting on each region cluster in the region cluster set; Calculate the plane normal vector corresponding to each region cluster after the plane fitting; Obtain planar clusters whose plane normal vector Z-values are greater than a preset plane Z-value threshold.
Citation Information
Patent Citations
Stair detection method and device and mobile robot
CN112529963A