Obstacle detection method, device, computer-readable storage medium, and processor

Abstract

The present application provides an obstacle detection method, device, computer-readable storage medium, and processor. The method comprises: obtaining a point cloud to be identified, the point cloud to be identified being a point cloud of objects within the field of view of a depth camera; calculating a ground plane equation in real time based on the point cloud to be identified, the ground plane equation being the equation of the current ground in the robot coordinate system; moving the plane corresponding to the ground plane equation by a first predetermined value along the positive Z-axis of the robot coordinate system to obtain a segmentation plane; determining a preliminary obstacle point cloud based on the segmentation plane, the preliminary obstacle point cloud being the point cloud to be identified located on the side of the segmentation plane close to the positive Z-axis of the robot coordinate system; and identifying the preliminary obstacle point cloud to determine an obstacle point cloud, the obstacle point cloud being a point cloud calibrated as an obstacle. This method solves the problem in the prior art of low reliability of the ground plane equation, which leads to missed and false obstacle detections.

Description

Technical Field

[0001] The present application relates to the field of point cloud processing technology, and more specifically, to an obstacle detection method, device, computer-readable storage medium, and processor. Background Art

[0002] Manufacturing plants are currently moving towards automation and intelligence. In some scenarios, automated guided vehicles (AGVs) have replaced traditional manual handling. AGVs are small, self-driving vehicles designed for specific scenarios. They need to detect the state of their surroundings, making perception essential. In factory workshops or warehouses, environments are rarely static, and unknown obstacles may exist along the AGV's planned path. The perception layer must accurately and promptly identify these obstacles near the AGV. Common AGV environmental perception sensors include lidar, sonar, monocular cameras, binocular cameras, and depth cameras. These sensors act as the eyes and ears of the AGV and play a critical role in ensuring its safe operation.

[0003] Many factories and shopping malls are equipped with automated guided vehicles (AGVs) and other intelligent robots. In recent years, there have been numerous cases of robot damage or personnel injuries caused by a lack of robot perception or low perception accuracy. To reduce this, sensors that can perceive dense environmental information are needed. Depth cameras, as high-perception, low-cost sensors, can capture dense point cloud data from a large area in front of the camera.

[0004] Single-line lidar is often used for indoor scenarios. Its sensing range is limited to a single plane and is largely ineffective against obstacles outside its range. Sonar positioning has low accuracy and a narrow sensing range. Monocular cameras lack scale information and struggle to perceive the true spatial position of obstacles. Binocular cameras have poor detection performance for textureless obstacles, consume high computing resources, and their accuracy decreases significantly with distance. Depth cameras, with their compact size, low power consumption, wide measurement range, and high accuracy, have been widely used in robotics.

[0005] Existing technologies for detecting obstacles using depth cameras also have certain flaws, such as: 1. Only point clouds above a fixed plane are considered obstacles. If the camera shakes during vehicle movement or its external parameters change, there is a risk of missed or false detections. 2. Due to the inherent characteristics of the camera's point cloud imaging, some data may be poor in default scenarios. For example, multipath issues with TOF cameras can cause the ground to be curved, or reflectivity issues can cause the ground label point cloud and the ground point cloud to be off-plane. Existing methods fail to effectively handle data errors caused by the depth camera's unique external environment.

[0006] The above information disclosed in the background technology section is only used to enhance the understanding of the background technology of the technology described in this article. Therefore, the background technology may contain certain information that does not form the prior art known in this country to those skilled in the art. Summary of the Invention

[0007] The main purpose of this application is to provide an obstacle detection method, device, computer-readable storage medium and processor to solve the problem of low reliability of ground plane equations in the prior art leading to missed obstacle detection and false detection.

[0008] According to one aspect of an embodiment of the present invention, a method for detecting an obstacle is provided. A robot includes a depth camera, and the method includes: obtaining a point cloud to be identified, the point cloud to be identified being a point cloud of objects within the field of view of the depth camera; calculating a ground plane equation in real time based on the point cloud to be identified, the ground plane equation being an equation of the current ground in the robot coordinate system; moving a plane corresponding to the ground plane equation by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane; determining a prepared obstacle point cloud based on the segmentation plane, the prepared obstacle point cloud being the point cloud to be identified located on a side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system; and identifying the prepared obstacle point cloud to determine an obstacle point cloud, the obstacle point cloud being a point cloud calibrated as an obstacle.

[0009] Optionally, the ground plane equation is calculated in real time based on the point cloud to be identified, including: obtaining a priori plane equation, the priori plane equation is the equation of the ground in the robot coordinate system when the robot is on flat ground, and the plane corresponding to the priori plane equation is perpendicular to the Z axis of the robot coordinate system; determining a candidate segmentation plane area based on the priori plane equation, the segmentation plane candidate area is the area between a first plane and a second plane in the robot coordinate system, the first plane is obtained by moving the plane corresponding to the priori plane equation along the positive direction of the Z axis of the robot coordinate system by a second predetermined value, and the second plane is obtained by moving the plane corresponding to the priori plane equation along the negative direction of the Z axis of the robot coordinate system by the second predetermined value, and the positive direction of the Z axis of the robot coordinate system is the direction of increasing height; determining a maximum plane equation based on the point cloud to be identified in the segmentation plane candidate area, the maximum plane equation is the equation of the maximum plane formed by the point cloud to be identified in the segmentation plane candidate area in the robot coordinate system; and determining the ground plane equation based on the priori plane equation and the maximum plane equation.

[0010] Optionally, determining the maximum plane equation based on the point cloud to be identified in the candidate area of ​​the segmentation surface includes: dividing the candidate area of ​​the segmentation surface into multiple first cells, where the first cell is a cube with a side length of a first length; sampling the point cloud to be identified in each of the first cells to obtain multiple preliminary sampling points, where the coordinates of the preliminary sampling points are the coordinates of any point in the point cloud to be identified in the first cell; dividing the candidate area of ​​the segmentation surface into multiple second cells, where the second cell is a cube with a side length of a second length, where the second length is greater than the first length; performing secondary sampling on the sampling points in the second cell to obtain multiple sampling points, where the sampling points are the center of gravity of the sampling points in the second cell; and determining the maximum plane equation based on the sampling points, where the maximum plane equation is an equation of the maximum plane formed by the sampling points in the robot coordinate system.

[0011] Optionally, the ground plane equation is determined based on the prior plane equation and the maximum plane equation, including: when at least one predetermined condition is met, determining the prior plane equation to be the ground plane equation, the predetermined condition including that the distance between the first point and the second point is greater than a distance threshold, the angle between the first normal vector and the second normal vector is greater than a predetermined angle, and the number of points in the point cloud within the candidate area of ​​the segmentation surface is less than a point number threshold, the first point is any point on the plane corresponding to the prior plane equation, whose X-axis coordinate and Y-axis coordinate are both less than a third predetermined value, the second point is any point on the plane corresponding to the maximum plane equation, whose X-axis coordinate and Y-axis coordinate are both less than the third predetermined value, the first normal vector is the normal vector of the plane corresponding to the prior plane equation, and the second normal vector is the normal vector of the plane corresponding to the maximum plane equation; when all of the predetermined conditions are not met, determining the maximum plane equation to be the ground plane equation.

[0012] Optionally, the obstacle point cloud includes a suspended obstacle point cloud, a high obstacle point cloud, and a low obstacle point cloud. The prepared obstacle point cloud is identified to determine the obstacle point cloud, including: calibrating the prepared obstacle point cloud with Z-axis coordinates of all points greater than a fourth predetermined value as the suspended obstacle point cloud, where the fourth predetermined value is greater than a second predetermined value; calibrating the prepared obstacle point cloud with Z-axis coordinates of some points greater than the fourth predetermined value as the high obstacle point cloud; and calibrating the prepared obstacle point cloud with Z-axis coordinates of all points less than or equal to the second predetermined value as the low obstacle point cloud.

[0013] Optionally, after identifying the prepared obstacle point cloud and determining the obstacle point cloud, the method further includes: determining the neighboring points of the target point based on the target point, the target point being any point in the high obstacle point cloud or the low obstacle point cloud, and the neighboring points being points in the high obstacle point cloud or the low obstacle point cloud whose distance from the target point is less than a first predetermined distance; clustering the target points according to clustering conditions to obtain multiple point cloud blocks, the clustering conditions including a first clustering condition, a second clustering condition and a third clustering condition, the first clustering condition being that the difference between the Z-axis coordinate of the target point and the Z-axis coordinates of all the neighboring points is greater than a fifth predetermined value, the second clustering condition being that the normal vector angle of the target point is less than or equal to the predetermined angle, the Z-axis coordinate of the target point is less than or equal to the predetermined angle, and the Z-axis coordinate of the target point is less than or equal to the predetermined angle. The difference in the Z-axis coordinates of some of the neighboring points is less than the sixth predetermined value, and the Euclidean distance between the target point and all the neighboring points is less than the second predetermined distance. The normal vector angle is the angle between the local normal vector of the target point and the Z-axis of the robot coordinate system. The local normal vector is the normal vector of the tangent of the surface composed of the target point and the neighboring points at the target point. The second predetermined distance is less than the first predetermined distance. The sixth predetermined value is less than the fifth predetermined value. The third clustering condition is that the target point that does not meet the first clustering condition and the second clustering condition is simultaneously satisfied, the normal vector angle of the target point is greater than the predetermined angle, and the Euclidean distance between the target point and all the neighboring points is less than the third predetermined distance. Multiple point cloud blocks are spliced ​​to obtain the obstacle point cloud.

[0014] Optionally, after identifying the prepared obstacle point cloud and determining the obstacle point cloud, the method further includes: generating a first intensity map based on the point cloud data of the low obstacle point cloud; determining a high-intensity area based on the first intensity map, the high-intensity area being an area where the intensity of the low obstacle point cloud is greater than a first predetermined intensity; determining a shadow area based on the high-intensity area, the shadow area being an area where the point cloud corresponding to the point cloud in the high-intensity area in the first intensity map cannot be collected due to obstacle occlusion; determining whether the low obstacle point cloud is misjudged based on a duty cycle, where the duty cycle is the ratio of the area occupied by the point cloud in the shadow area to the area of ​​the shadow area.

[0015] Optionally, determining whether the low obstacle point cloud is misjudged based on the duty cycle includes: when the duty cycle is greater than a predetermined threshold, determining that the low obstacle point cloud is misjudged; when the duty cycle is less than or equal to the predetermined threshold, determining that the low obstacle point cloud is not misjudged.

[0016] Optionally, after identifying the prepared obstacle point cloud and determining the obstacle point cloud, the method further includes: generating a second intensity map based on the point cloud data of the point cloud to be identified; determining an intensity abnormality area based on the second intensity map, the intensity abnormality area being an area where the intensity of the point cloud to be identified is greater than a second predetermined intensity or less than a third predetermined intensity, and the third predetermined intensity is less than the second predetermined intensity; performing an inverse perspective transformation on the second intensity map to obtain a transformed intensity map; comparing the transformed intensity map and the intensity abnormality area of ​​the second intensity map to determine whether the point cloud to be identified corresponding to the intensity abnormality area is the obstacle point cloud.

[0017] Optionally, the intensity anomaly area of ​​the transformed intensity map and the second intensity map is compared to determine whether the point cloud to be identified corresponding to the intensity anomaly area is the obstacle point cloud, including: when the ratio of the first length to the second length is greater than a predetermined ratio, determining that the point cloud to be identified in the intensity anomaly area is an obstacle point cloud, the first length is the length of the intensity anomaly area in the transformed intensity map in the target direction, the second length is the length of the intensity anomaly area in the second intensity map in the target direction, the target direction is an arbitrary direction, and the predetermined ratio is greater than 1; when the ratio of the first length to the second length is less than or equal to the predetermined ratio, determining that the point cloud to be identified in the intensity anomaly area is not an obstacle point cloud.

[0018] According to another aspect of an embodiment of the present invention, an obstacle detection device is also provided. The robot includes a depth camera, and the device includes: an acquisition unit, configured to acquire a point cloud to be identified, wherein the point cloud to be identified is a point cloud of objects within the field of view of the depth camera; a calculation unit, configured to calculate a ground plane equation in real time based on the point cloud to be identified, wherein the ground plane equation is an equation of the current ground in the robot coordinate system; a processing unit, configured to move a plane corresponding to the ground plane equation by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane; a first determination unit, configured to determine a prepared obstacle point cloud based on the segmentation plane, wherein the prepared obstacle point cloud is the point cloud to be identified located on the side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system; and a second determination unit, configured to identify the prepared obstacle point cloud and determine an obstacle point cloud, wherein the obstacle point cloud is a point cloud calibrated as an obstacle.

[0019] According to yet another aspect of the embodiments of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored program, wherein the program executes the method.

[0020] According to yet another aspect of an embodiment of the present invention, a processor is provided, wherein the processor is configured to run a program, wherein the method is executed when the program is run.

[0021] In an embodiment of the present invention, in the obstacle detection method, first, a point cloud to be identified is obtained, where the point cloud to be identified is a point cloud of objects within the depth camera's field of view; then, a ground plane equation is calculated in real time based on the point cloud to be identified, where the ground plane equation is the equation of the current ground in the robot coordinate system; then, a plane corresponding to the ground plane equation is moved by a first predetermined value along the positive Z-axis of the robot coordinate system to obtain a segmentation plane; then, a preliminary obstacle point cloud is determined based on the segmentation plane, where the preliminary obstacle point cloud is located on the side of the point cloud to be identified that is closer to the positive Z-axis of the robot coordinate system; finally, the preliminary obstacle point cloud is identified to determine an obstacle point cloud, where the obstacle point cloud is calibrated as an obstacle. By calculating the ground plane equation in real time, the problem of unreliable ground plane equations due to a significant difference between the ground plane equation and the ground plane equation of flat ground when the depth camera jitters with the movement of the robot is avoided, the reliability of the ground plane equation is improved, and the problem of low reliability of the ground plane equation in the prior art leading to missed obstacle detections and false detections is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings that constitute part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0023] Figure 1 A flow chart showing a method for detecting an obstacle according to an embodiment of the present application is shown;

[0024] Figure 2 A schematic diagram of an obstacle detection device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0026] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0027] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element or intervening elements may be present. Moreover, in the specification and claims, when it is described that an element is "connected to" another element, the element may be "directly connected to" the other element or "connected to" the other element through a third element.

[0028] As mentioned in the background technology, the low reliability of the ground plane equation in the prior art leads to missed obstacle detection and false detection. In order to solve the above problems, a typical embodiment of the present application provides an obstacle detection method, device, computer-readable storage medium and processor.

[0029] According to an embodiment of the present application, a method for detecting an obstacle is provided.

[0030] Figure 1 FIG. 1 is a flow chart of a method for detecting obstacles according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0031] Step S101, obtaining a point cloud to be identified, where the point cloud to be identified is a point cloud of an object within the field of view of the depth camera;

[0032] Step S102, calculating the ground plane equation in real time based on the point cloud to be identified, where the ground plane equation is the equation of the current ground in the robot coordinate system;

[0033] Step S103, moving the plane corresponding to the ground plane equation along the positive direction of the Z axis of the robot coordinate system by a first predetermined value to obtain a segmentation plane;

[0034] Step S104: determining a prepared obstacle point cloud according to the segmentation plane, wherein the prepared obstacle point cloud is the point cloud to be identified that is located on the side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system;

[0035] Step S105 : Identify the prepared obstacle point cloud to determine the obstacle point cloud, where the obstacle point cloud is the point cloud calibrated as an obstacle.

[0036] In the above-mentioned obstacle detection method, first, a point cloud to be identified is obtained, which is a point cloud of objects within the field of view of the depth camera. Then, a ground plane equation is calculated in real time based on the point cloud to be identified, which is the equation of the current ground in the robot coordinate system. Then, the plane corresponding to the ground plane equation is moved by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane. Then, a preliminary obstacle point cloud is determined based on the segmentation plane, which is the point cloud to be identified located on the side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system. Finally, the preliminary obstacle point cloud is identified to determine an obstacle point cloud, which is a point cloud calibrated as an obstacle. By calculating the ground plane equation in real time, the problem of unreliable ground plane equation due to a large difference between the ground plane equation and the ground plane equation of flat ground when the depth camera shakes with the movement of the robot is avoided, the reliability of the ground plane equation is improved, and the problem of low reliability of the ground plane equation leading to missed obstacle detection and false detection in the prior art is solved.

[0037] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] In one embodiment of the present application, a ground plane equation is calculated in real time based on the above-mentioned point cloud to be identified, including: obtaining a priori plane equation, the above-mentioned priori plane equation is an equation of the ground in the robot coordinate system when the above-mentioned robot is on flat ground, and the plane corresponding to the above-mentioned priori plane equation is perpendicular to the Z-axis of the above-mentioned robot coordinate system; determining a candidate segmentation plane area based on the above-mentioned priori plane equation, the above-mentioned segmentation plane candidate area is an area between a first plane and a second plane in the above-mentioned robot coordinate system, the above-mentioned first plane is obtained by moving the plane corresponding to the above-mentioned priori plane equation along the positive direction of the Z-axis of the above-mentioned robot coordinate system by a second predetermined value, and the above-mentioned second plane is obtained by moving the plane corresponding to the above-mentioned priori plane equation along the negative direction of the Z-axis of the above-mentioned robot coordinate system by the above-mentioned second predetermined value, and the positive direction of the Z-axis of the above-mentioned robot coordinate system is a direction of increasing height; determining a maximum plane equation based on the above-mentioned point cloud to be identified in the above-mentioned segmentation plane candidate area, the above-mentioned maximum plane equation is an equation of the maximum plane formed by the above-mentioned point cloud to be identified in the above-mentioned segmentation plane candidate area in the robot coordinate system; and determining the above-mentioned ground plane equation based on the above-mentioned priori plane equation and the above-mentioned maximum plane equation. Specifically, after the depth camera is installed on the robot, the camera's installation external parameters need to be calibrated. The prior plane equation can be determined based on the external parameters. If the obstacle detection accuracy requirement is low and the ground where the robot is active is relatively flat, then only the point cloud with a fixed height above P1 can be retained, for example, 6 cm above P1. However, in actual scenes, the ground is not very flat, and the robot will bump up and down when moving, which will cause part of the ground point cloud to be identified as an obstacle. The above method obtains the candidate area for the segmented surface, and uses RANSAC to extract the maximum plane in a small area above and below the Z axis of the plane corresponding to the prior plane equation P1, for example, 20 cm above and below, that is, the maximum plane equation P2 corresponding to the maximum plane is obtained. According to the actual situation, the appropriate ground plane equation is selected from the prior plane equation P1 and the maximum plane equation P2.

[0039] It should be noted that a spherical point cloud filter is used to retain only the point cloud within the area that needs to be processed, and the point cloud is converted from the camera coordinate system to the robot coordinate system. The rectangular box filter is used to retain the point cloud data within the specified range of the vehicle on the current point cloud, and the point cloud of the candidate area of ​​the segmentation surface can be obtained. The parameters here are variable. For example, a backpack AGV requires different point cloud ranges in the two states of carrying and no-load.

[0040] In one embodiment of the present application, determining a maximum plane equation based on the point cloud to be identified within the candidate region of the segmented surface includes: dividing the candidate region of the segmented surface into a plurality of first cells, wherein the first cells are cubes having a first side length; sampling the point cloud to be identified within each of the first cells to obtain a plurality of preliminary sampling points, wherein the coordinates of the preliminary sampling points are the coordinates of any point of the point cloud to be identified within the first cells; dividing the candidate region of the segmented surface into a plurality of second cells, wherein the second cells are cubes having a second side length, wherein the second length is greater than the first length; sub-sampling the sampling points within the second cells to obtain a plurality of sampling points, wherein the sampling points are the centroids of the sampling points within the second cells; and determining the maximum plane equation based on the sampling points, wherein the maximum plane equation is the equation of the maximum plane formed by the sampling points in the robot coordinate system. Specifically, the point cloud directly output by the depth camera is too dense. If the subsequent filter directly processes it, it will consume a lot of CPU resources. If the downsampling is too sparse, the detection accuracy will decrease. Therefore, the point cloud is downsampled to an appropriate scale. The mainstream point cloud downsampling methods include voxel filtering and direct downsampling. Voxel filtering calculates the average coordinate values ​​of all points in each cell, which consumes a lot of computing resources, but the overall point cloud distribution is relatively uniform. For example, after voxel filtering, all point clouds on a plane are almost on the same plane. Direct downsampling uses the first point in each cell as the representative of this cell. The downsampling speed is fast and the resource consumption is low, but the point distribution error after sampling is large. The above method combines the advantages and disadvantages of the above two filters, using direct downsampling of small-scale cells and voxel filtering superposition downsampling of large-scale cells. While ensuring the downsampling accuracy, it improves the calculation speed. In other words, it can achieve a better downsampling effect while reducing CPU usage.

[0041] In one embodiment of the present application, the ground plane equation is determined based on the prior plane equation and the maximum plane equation, including: when at least one predetermined condition is met, determining the prior plane equation to be the ground plane equation, the predetermined condition including that the distance between the first point and the second point is greater than a distance threshold, the angle between the first normal vector and the second normal vector is greater than a predetermined angle, and the number of points in the point cloud within the candidate area of ​​the segmentation surface is less than a point number threshold, the first point is any point on the plane corresponding to the prior plane equation, whose X-axis coordinate and Y-axis coordinate are both less than a third predetermined value, the second point is any point on the plane corresponding to the maximum plane equation, whose X-axis coordinate and Y-axis coordinate are both less than the third predetermined value, the first normal vector is the normal vector of the plane corresponding to the prior plane equation, and the second normal vector is the normal vector of the plane corresponding to the maximum plane equation; when all of the above predetermined conditions are not met, determining the maximum plane equation to be the ground plane equation. Specifically, the distance between the first point and the second point is greater than the distance threshold, the plane corresponding to the maximum plane equation is the step plane, the angle between the first normal vector and the second normal vector is greater than the predetermined angle, the plane corresponding to the maximum plane equation is the wall plane, the number of points in the point cloud within the candidate area of ​​the segmentation surface is less than the point number threshold, and the point cloud is a point cloud formed by noise, etc. Therefore, in any of these three cases, the plane corresponding to the ground plane equation is not the ground, and the above-mentioned prior plane equation is determined to be the above-mentioned ground plane equation. Otherwise, the plane corresponding to the ground plane equation is the ground, and the above-mentioned maximum plane equation is determined to be the above-mentioned ground plane equation.

[0042] In one embodiment of the present application, the obstacle point cloud includes a suspended obstacle point cloud, a high obstacle point cloud, and a low obstacle point cloud. Identifying the prepared obstacle point cloud and determining the obstacle point cloud includes: calibrating all points in the prepared obstacle point cloud with a Z-axis coordinate greater than a fourth predetermined value as the suspended obstacle point cloud, where the fourth predetermined value is greater than a second predetermined value; calibrating some points in the prepared obstacle point cloud with a Z-axis coordinate greater than the fourth predetermined value as the high obstacle point cloud; and calibrating all points in the prepared obstacle point cloud with a Z-axis coordinate less than or equal to the second predetermined value as the low obstacle point cloud. Specifically, all point clouds are subjected to Euclidean clustering to remove point cloud noise. The method classifies the multiple point cloud blocks obtained by clustering into suspended obstacle point clouds, high obstacle point clouds, and low obstacle point clouds based on their geometric and statistical features.

[0043] In one embodiment of the present application, after identifying the prepared obstacle point cloud and determining the obstacle point cloud, the method further includes: determining the neighboring points of the target point according to the target point, the target point being any point in the high obstacle point cloud or the low obstacle point cloud, and the neighboring points being points in the high obstacle point cloud or the low obstacle point cloud whose distance from the target point is less than a first predetermined distance; clustering the target points according to clustering conditions to obtain a plurality of point cloud blocks, the clustering conditions including a first clustering condition, a second clustering condition and a third clustering condition, the first clustering condition being that the difference between the Z-axis coordinate of the target point and the Z-axis coordinates of all the neighboring points is greater than a fifth predetermined value, the second clustering condition being that the normal vector angle of the target point is less than or equal to the predetermined angle, the Z-axis of the target point is less than or equal to the predetermined angle, and the Z-axis of the target point is less than or equal to the predetermined angle. The difference between the coordinates and the Z-axis coordinates of all the above-mentioned neighboring points is less than the sixth predetermined value, and the Euclidean distance between the above-mentioned target point and all the above-mentioned neighboring points is less than the second predetermined distance. The above-mentioned normal vector angle is the angle between the local normal vector of the above-mentioned target point and the Z-axis of the above-mentioned robot coordinate system. The above-mentioned local normal vector is the normal vector of the tangent of the surface composed of the above-mentioned target point and the above-mentioned neighboring points at the above-mentioned target point. The above-mentioned second predetermined distance is less than the above-mentioned first predetermined distance. The above-mentioned sixth predetermined value is less than the above-mentioned fifth predetermined value. The above-mentioned third clustering condition is that the above-mentioned target point that does not meet the above-mentioned first clustering condition and the second clustering condition is simultaneously satisfied, the normal vector angle of the above-mentioned target point is greater than the above-mentioned predetermined angle, and the Euclidean distance between the above-mentioned target point and all the above-mentioned neighboring points is less than the third predetermined distance. Multiple point cloud blocks are spliced ​​to obtain the above-mentioned obstacle point cloud. Specifically, due to the multipath factors of the depth camera, the point cloud of a high obstacle will contain some distorted and sunken ground below, and the point cloud of a low obstacle may be a real low obstacle or normal ground. The above method will cluster the points on the same plane together to form point cloud blocks. After the point cloud blocks are spliced, the ground and obstacles can be distinguished, avoiding the identification of normal ground as a low obstacle, and the inclusion of some distorted and sunken ground below the point cloud of a high obstacle, thereby improving the accuracy of obstacle identification.

[0044] In one embodiment of the present application, after identifying the prepared obstacle point cloud and determining the obstacle point cloud, the method further includes: generating a first intensity map based on the point cloud data of the low obstacle point cloud; determining a high-intensity area based on the first intensity map, the high-intensity area being an area where the intensity of the low obstacle point cloud is greater than a first predetermined intensity; determining a shadow area based on the high-intensity area, the shadow area being an area where the point cloud corresponding to the high-intensity area in the first intensity map cannot be collected due to an obstacle occlusion; determining whether the low obstacle point cloud is misjudged based on the duty cycle, the duty cycle being the ratio of the area occupied by the point cloud in the shadow area to the area of ​​the shadow area. Specifically, when a label such as a safety exit is affixed to the ground, the label's reflectivity is higher than the ground's reflectivity, and the point cloud at the label's position is higher than the ground's point cloud, and it will be mistakenly judged as a low obstacle. Therefore, it is necessary to determine the high-intensity area of ​​the first intensity map for false detection screening. There should be no point cloud within a certain range behind the real low obstacle. The duty cycle of the shadow area behind the low obstacle point cloud can be used to determine whether it is a real obstacle, thereby determining whether it is a misjudgment.

[0045] In one embodiment of the present application, determining whether the low obstacle point cloud is misjudged based on the duty cycle includes: determining that the low obstacle point cloud is misjudged when the duty cycle is greater than a predetermined threshold; and determining that the low obstacle point cloud is not misjudged when the duty cycle is less than or equal to the predetermined threshold. Specifically, if the duty cycle is greater than the predetermined threshold, it can be proved that there is a point cloud behind the low obstacle point cloud and the low obstacle point cloud is not a real obstacle. Otherwise, it can be determined that the low obstacle point cloud is not a real obstacle. It should be noted that if the duty cycle is less than or equal to the predetermined threshold, the tag may be far away, and the original point cloud in the area behind the tag is very small, requiring the robot to move closer for identification.

[0046] In one embodiment of the present application, after identifying the above-mentioned prepared obstacle point cloud and determining the obstacle point cloud, the above-mentioned method also includes: generating a second intensity map based on the point cloud data of the above-mentioned point cloud to be identified; determining the intensity abnormality area based on the above-mentioned second intensity map, the above-mentioned intensity abnormality area being the area where the intensity of the above-mentioned point cloud to be identified is greater than the second predetermined intensity or less than the third predetermined intensity, and the above-mentioned third predetermined intensity is less than the above-mentioned second predetermined intensity; performing an inverse perspective transformation on the above-mentioned second intensity map to obtain a transformed intensity map; comparing the above-mentioned transformed intensity map and the above-mentioned intensity abnormality area of ​​the above-mentioned second intensity map to determine whether the above-mentioned point cloud to be identified corresponding to the above-mentioned intensity abnormality area is the above-mentioned obstacle point cloud. Specifically, obstacles on the ground may be materials with high or low reflectivity, such as aluminum profiles or black boxes. Due to overexposure or low intensity, the actual point cloud generated by obstacles in this area is very small, which may lead to missed detection. Therefore, it is necessary to screen for missed detection in areas with abnormal intensity. For areas with abnormal intensity map, perform IPM inverse perspective transformation of the camera and the ground. If the transformed area is stretched abnormally long, it can be determined that the area is a real three-dimensional object, and missed detection is determined. The point cloud below the area is used as the lower edge of the obstacle, and the upper edge of the intensity map is used to obtain the upper edge of the object according to the perspective transformation, so as to estimate the position and size of the weakly reflective obstacle, thereby reducing missed detection.

[0047] In one embodiment of the present application, the intensity abnormality area of ​​the transformed intensity map and the second intensity map is compared to determine whether the point cloud to be identified corresponding to the intensity abnormality area is the obstacle point cloud, including: when the ratio of the first length to the second length is greater than a predetermined ratio, determining that the point cloud to be identified in the intensity abnormality area is an obstacle point cloud, the first length is the length of the intensity abnormality area in the transformed intensity map in the target direction, the second length is the length of the intensity abnormality area in the second intensity map in the target direction, the target direction is an arbitrary direction, and the predetermined ratio is greater than 1; when the ratio of the first length to the second length is less than or equal to the predetermined ratio, determining that the point cloud to be identified in the intensity abnormality area is not an obstacle point cloud. Specifically, the above method can determine whether the area after the transformation of the abnormal area of ​​the intensity map meets the condition of being stretched abnormally long, thereby determining whether it is missed and ensuring the accuracy of recognition.

[0048] The present application also provides an obstacle detection device. It should be noted that the obstacle detection device of the present application can be used to execute the obstacle detection method provided in the present application. The obstacle detection device provided in the present application is introduced below.

[0049] Figure 2 Schematic diagram of an obstacle detection device according to an embodiment of the present application. Figure 2 As shown, the device includes:

[0050] An acquisition unit 10 is configured to acquire a point cloud to be identified, where the point cloud to be identified is a point cloud of an object within the field of view of the depth camera;

[0051] A calculation unit 20 is configured to calculate a ground plane equation in real time based on the point cloud to be identified, where the ground plane equation is an equation of the current ground in the robot coordinate system;

[0052] The processing unit 30 is configured to move the plane corresponding to the ground plane equation by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane;

[0053] A first determining unit 40 is configured to determine a prepared obstacle point cloud according to the segmentation plane, wherein the prepared obstacle point cloud is the point cloud to be identified that is located on a side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system;

[0054] The second determining unit 50 is configured to identify the prepared obstacle point cloud and determine the obstacle point cloud, where the obstacle point cloud is a point cloud calibrated as an obstacle.

[0055] In the obstacle detection device, an acquisition unit acquires a point cloud to be identified, the point cloud being a point cloud of objects within the depth camera's field of view; a calculation unit calculates a ground plane equation in real time based on the point cloud to be identified, the ground plane equation being the equation of the current ground in the robot coordinate system; a processing unit shifts the plane corresponding to the ground plane equation by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane; a first determination unit determines a preliminary obstacle point cloud based on the segmentation plane, the preliminary obstacle point cloud being the point cloud to be identified located on the side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system; and a second determination unit identifies the preliminary obstacle point cloud and determines an obstacle point cloud, the obstacle point cloud being a point cloud calibrated as an obstacle. By calculating the ground plane equation in real time, the device avoids the problem of unreliable ground plane equations due to significant differences between the ground plane equation and the ground plane equation of flat ground when the depth camera jitters as the robot moves, thereby improving the reliability of the ground plane equation and resolving the problem of low reliability of the ground plane equation in the prior art, which can lead to missed and false obstacle detections.

[0056] In one embodiment of the present application, the calculation unit includes an acquisition module, a first determination module, a second determination module and a third determination module, wherein the acquisition module is used to obtain a priori plane equation, the priori plane equation is an equation of the ground in the robot coordinate system when the robot is on flat ground, and the plane corresponding to the priori plane equation is perpendicular to the Z axis of the robot coordinate system; the first determination module is used to determine a candidate segmentation surface area according to the priori plane equation, and the candidate segmentation surface area is an area between the first plane and the second plane in the robot coordinate system, and the first plane is the plane corresponding to the priori plane equation along the Z axis of the robot coordinate system. The second plane is obtained by moving the plane corresponding to the prior plane equation in the positive direction of the Z axis of the robot coordinate system by the second predetermined value, and the positive direction of the Z axis of the robot coordinate system is the direction of height increase; the second determination module is used to determine the maximum plane equation based on the point cloud to be identified in the candidate area of ​​the segmentation surface, and the maximum plane equation is the equation of the maximum plane formed by the point cloud to be identified in the candidate area of ​​the segmentation surface in the robot coordinate system; the third determination module is used to determine the ground plane equation based on the prior plane equation and the maximum plane equation. Specifically, after the depth camera is installed on the robot, the camera's installation external parameters need to be calibrated, and the prior plane equation can be determined based on the external parameters. If the obstacle detection accuracy requirement is low and the ground where the robot is active is relatively flat, then only the point cloud with a fixed height above P1 is retained, for example, 6 cm above P1. However, the ground in real scenes is not very flat, and the robot will bump up and down when moving, which will cause part of the ground point cloud to be identified as obstacles. The above method obtains the candidate area of ​​the segmentation surface, and uses RANSAC to extract the maximum plane in a small area above and below the Z axis in the plane corresponding to the prior plane equation P1, for example, 20 cm above and below, that is, the maximum plane equation P2 corresponding to the maximum plane is obtained. According to the actual situation, the appropriate ground plane equation is selected from the prior plane equation P1 and the maximum plane equation P2.

[0057] It should be noted that a spherical point cloud filter is used to retain only the point cloud within the area that needs to be processed, and the point cloud is converted from the camera coordinate system to the robot coordinate system. The rectangular box filter is used to retain the point cloud data within the specified range of the vehicle on the current point cloud, and the point cloud of the candidate area of ​​the segmentation surface can be obtained. The parameters here are variable. For example, a backpack AGV requires different point cloud ranges in the two states of carrying and no-load.

[0058] In one embodiment of the present application, the second determination module is used to include a first processing submodule, a first sampling submodule, a second processing submodule, a second sampling submodule and a first determination submodule, wherein the first processing submodule is used to divide the segmentation surface candidate area into multiple first cells, where the first cells are cubes with a side length of a first length; the first sampling submodule is used to sample the point cloud to be identified in each of the first cells to obtain multiple preliminary sampling points, where the coordinates of the preliminary sampling points are the coordinates of any point in the point cloud to be identified in the first cell; the second processing submodule is used to divide the segmentation surface candidate area into multiple second cells, where the second cells are cubes with a side length of a second length, where the second length is greater than the first length; the second sampling submodule performs secondary sampling on the sampling points in the second cells to obtain multiple sampling points, where the sampling points are the centers of gravity of the sampling points in the second cells; the first determination submodule is used to determine the maximum plane equation based on the sampling points, where the maximum plane equation is the equation of the maximum plane formed by the sampling points in the robot coordinate system. Specifically, the point cloud directly output by the depth camera is too dense. If the subsequent filter processes it directly, it will take up a lot of CPU resources. If the downsampling is too sparse, the detection accuracy will decrease. Therefore, the point cloud is downsampled at an appropriate scale. The mainstream point cloud downsampling methods include voxel filtering and direct downsampling. Voxel filtering calculates the average coordinate values ​​of all points in each cell, which consumes a lot of computing resources, but the overall point cloud distribution is relatively uniform. For example, after the point cloud on a plane is voxel filtered, all the point clouds are almost on the same plane. Direct downsampling uses the first point in each cell as the representative of this cell. The downsampling speed is fast and the resource consumption is small, but the point distribution error after sampling is large. The above method combines the advantages and disadvantages of the above two filters, using direct downsampling of small-scale cells and voxel filtering of large-scale cells for superimposed downsampling. While ensuring the downsampling accuracy, it improves the calculation speed. In other words, it can achieve a good downsampling effect and reduce CPU usage.

[0059] In one embodiment of the present application, the above-mentioned third determination module includes a second determination submodule and a third determination submodule, wherein the above-mentioned second determination submodule is used to determine that the above-mentioned prior plane equation is the above-mentioned ground plane equation when at least one predetermined condition is met, and the above-mentioned predetermined conditions include that the distance between the first point and the second point is greater than the distance threshold, the angle between the first normal vector and the second normal vector is greater than the predetermined angle, and the number of points in the point cloud within the above-mentioned segmentation surface candidate area is less than the point number threshold, the above-mentioned first point is any point on the plane corresponding to the above-mentioned prior plane equation, and the X-axis coordinate and the Y-axis coordinate are both less than the third predetermined value, the above-mentioned second point is any point on the plane corresponding to the above-mentioned maximum plane equation, and the X-axis coordinate and the Y-axis coordinate are both less than the above-mentioned third predetermined value, the above-mentioned first normal vector is the normal vector of the plane corresponding to the above-mentioned prior plane equation, and the above-mentioned second normal vector is the normal vector of the plane corresponding to the above-mentioned maximum plane equation; the above-mentioned third determination submodule is used to determine that the above-mentioned maximum plane equation is the above-mentioned ground plane equation when all of the above-mentioned predetermined conditions are not met. Specifically, the distance between the first point and the second point is greater than the distance threshold, the plane corresponding to the maximum plane equation is the step plane, the angle between the first normal vector and the second normal vector is greater than the predetermined angle, the plane corresponding to the maximum plane equation is the wall plane, the number of points in the point cloud within the candidate area of ​​the segmentation surface is less than the point number threshold, and the point cloud is a point cloud formed by noise, etc. Therefore, in any of these three cases, the plane corresponding to the ground plane equation is not the ground, and the above-mentioned prior plane equation is determined to be the above-mentioned ground plane equation. Otherwise, the plane corresponding to the ground plane equation is the ground, and the above-mentioned maximum plane equation is determined to be the above-mentioned ground plane equation.

[0060] In one embodiment of the present application, the obstacle point cloud includes a suspended obstacle point cloud, a high obstacle point cloud, and a low obstacle point cloud. The second determination unit includes a first calibration module, a second calibration module, and a third calibration module. The first calibration module is configured to calibrate all points in the prepared obstacle point cloud whose Z-axis coordinates are greater than a fourth predetermined value as the suspended obstacle point cloud, where the fourth predetermined value is greater than a second predetermined value. The second calibration module is configured to calibrate some points in the prepared obstacle point cloud whose Z-axis coordinates are greater than the fourth predetermined value as the high obstacle point cloud. The third calibration module is configured to calibrate all points in the prepared obstacle point cloud whose Z-axis coordinates are less than or equal to the second predetermined value as the low obstacle point cloud. Specifically, all point clouds are subjected to Euclidean clustering to remove point cloud noise. The method then classifies the multiple point cloud blocks obtained by clustering into suspended obstacle point clouds, high obstacle point clouds, and low obstacle point clouds based on their geometric and statistical features.

[0061] In one embodiment of the present application, the above-mentioned device also includes a first re-inspection unit, and the above-mentioned first re-inspection unit includes a fourth determination module, a clustering module and a synthesis module, wherein the above-mentioned fourth determination module is used to identify the above-mentioned prepared obstacle point cloud, determine the obstacle point cloud, and then determine the neighboring points of the above-mentioned target point according to the target point, the above-mentioned target point is any point in the above-mentioned high obstacle point cloud or the above-mentioned low obstacle point cloud, and the above-mentioned neighboring points are points in the above-mentioned high obstacle point cloud or the above-mentioned low obstacle point cloud, whose distance from the above-mentioned target point is less than a first predetermined distance; the above-mentioned clustering module is used to cluster the above-mentioned target points according to clustering conditions to obtain multiple point cloud blocks, and the above-mentioned clustering conditions include a first clustering condition, a second clustering condition and a third clustering condition, the above-mentioned first clustering condition is that the difference between the Z-axis coordinate of the above-mentioned target point and the Z-axis coordinates of all the above-mentioned neighboring points is greater than a fifth predetermined value, and the above-mentioned second clustering condition is that the law of the above-mentioned target point is satisfied at the same time. The vector angle is less than or equal to the predetermined angle, the difference between the Z-axis coordinate of the above-mentioned target point and the Z-axis coordinates of all the above-mentioned neighboring points is less than the sixth predetermined value, and the Euclidean distance between the above-mentioned target point and all the above-mentioned neighboring points is less than the second predetermined distance. The above-mentioned normal vector angle is the angle between the local normal vector of the above-mentioned target point and the Z-axis of the above-mentioned robot coordinate system. The above-mentioned local normal vector is the normal vector of the tangent of the surface composed of the above-mentioned target point and the above-mentioned neighboring points at the above-mentioned target point. The above-mentioned second predetermined distance is less than the above-mentioned first predetermined distance. The above-mentioned sixth predetermined value is less than the above-mentioned fifth predetermined value. The above-mentioned third clustering condition is that the above-mentioned target point that does not meet the above-mentioned first clustering condition and the second clustering condition is simultaneously satisfied, the normal vector angle of the above-mentioned target point is greater than the above-mentioned predetermined angle, and the Euclidean distance between the above-mentioned target point and all the above-mentioned neighboring points is less than the third predetermined distance. The above-mentioned synthesis module is used to splice multiple of the above-mentioned point cloud blocks to obtain the above-mentioned obstacle point cloud. Specifically, due to the multipath factors of the depth camera, the point cloud of a high obstacle will contain some distorted and sunken ground below, and the point cloud of a low obstacle may be a real low obstacle or normal ground. The above method will cluster the points on the same plane together to form point cloud blocks. After the point cloud blocks are spliced, the ground and obstacles can be distinguished, avoiding the identification of normal ground as a low obstacle, and the inclusion of some distorted and sunken ground below the point cloud of a high obstacle, thereby improving the accuracy of obstacle identification.

[0062] In one embodiment of the present application, the above-mentioned device also includes a second re-inspection unit, and the above-mentioned second re-inspection unit includes a first generation module, a fifth determination module, a sixth determination module and a seventh determination module, wherein the above-mentioned first generation module is used to identify the above-mentioned prepared obstacle point cloud and determine the obstacle point cloud, and then generate a first intensity map based on the point cloud data of the above-mentioned low obstacle point cloud; the above-mentioned fifth determination module is used to determine the high-intensity area based on the above-mentioned first intensity map, and the above-mentioned high-intensity area is the area where the intensity of the above-mentioned low obstacle point cloud is greater than the first predetermined intensity; the above-mentioned sixth determination module is used to determine the shadow area based on the above-mentioned high-intensity area, and the above-mentioned shadow area is the area where the point cloud corresponding to the point cloud in the above-mentioned high-intensity area in the above-mentioned first intensity map is blocked by the obstacle, resulting in that the point cloud cannot be collected; the above-mentioned seventh determination module is used to determine whether the above-mentioned low obstacle point cloud is misjudged based on the duty cycle, and the above-mentioned duty cycle is the ratio of the area occupied by the point cloud in the above-mentioned shadow area to the area of ​​the above-mentioned shadow area. Specifically, when a label such as a safety exit is affixed to the ground, the label's reflectivity is higher than the ground's reflectivity, and the point cloud at the label's position is higher than the ground's point cloud, and it will be mistakenly judged as a low obstacle. Therefore, it is necessary to determine the high-intensity area of ​​the first intensity map for false detection screening. There should be no point cloud within a certain range behind the real low obstacle. The duty cycle of the shadow area behind the low obstacle point cloud can be used to determine whether it is a real obstacle, thereby determining whether it is a misjudgment.

[0063] In one embodiment of the present application, the seventh determination module includes a fourth determination submodule and a fifth determination submodule, wherein the fourth determination submodule is used to determine that the low obstacle point cloud is misjudged when the duty cycle is greater than a predetermined threshold; and the fifth determination submodule is used to determine that the low obstacle point cloud is not misjudged when the duty cycle is less than or equal to the predetermined threshold. Specifically, if the duty cycle is greater than the predetermined threshold, it can be proved that there is a point cloud behind the low obstacle point cloud and the low obstacle point cloud is not a real obstacle. Otherwise, it can be determined that the low obstacle point cloud is not a real obstacle. It should be noted that if the duty cycle is less than or equal to the predetermined threshold, the tag may be far away, and the original point cloud in the area behind the tag is very small, requiring the robot to approach for identification.

[0064] In one embodiment of the present application, the above-mentioned device also includes a third re-inspection unit, and the above-mentioned third re-inspection unit includes a second generation module, an eighth determination module, a processing module and a ninth determination module, wherein the above-mentioned second generation module is used to identify the above-mentioned prepared obstacle point cloud and determine the obstacle point cloud, and then generate a second intensity map according to the point cloud data of the above-mentioned point cloud to be identified; the above-mentioned eighth determination module is used to determine the intensity abnormality area according to the above-mentioned second intensity map, and the above-mentioned intensity abnormality area is an area where the intensity of the above-mentioned point cloud to be identified is greater than the second predetermined intensity or less than the third predetermined intensity, and the above-mentioned third predetermined intensity is less than the above-mentioned second predetermined intensity; the above-mentioned processing module is used to perform an inverse perspective transformation on the above-mentioned second intensity map to obtain a transformed intensity map; the above-mentioned ninth determination module is used to compare the above-mentioned transformed intensity map with the above-mentioned intensity abnormality area of ​​the above-mentioned second intensity map to determine whether the above-mentioned point cloud to be identified corresponding to the above-mentioned intensity abnormality area is the above-mentioned obstacle point cloud. Specifically, obstacles on the ground may be materials with high or low reflectivity, such as aluminum profiles or black boxes. Due to overexposure or low intensity, the actual point cloud generated by obstacles in this area is very small, which may lead to missed detection. Therefore, it is necessary to screen for missed detection in areas with abnormal intensity. For areas with abnormal intensity map, perform IPM inverse perspective transformation of the camera and the ground. If the transformed area is stretched abnormally long, it can be determined that the area is a real three-dimensional object, and missed detection is determined. The point cloud below the area is used as the lower edge of the obstacle, and the upper edge of the intensity map is used to obtain the upper edge of the object according to the perspective transformation, so as to estimate the position and size of the weakly reflective obstacle, thereby reducing missed detection.

[0065] In one embodiment of the present application, the ninth determination module includes a sixth determination submodule and a seventh determination submodule, wherein the sixth determination submodule is used to determine that the point cloud to be identified in the intensity abnormality area is an obstacle point cloud when the ratio of the first length to the second length is greater than a predetermined ratio, the first length is the length of the intensity abnormality area in the target direction in the transformed intensity map, the second length is the length of the intensity abnormality area in the target direction in the second intensity map, the target direction is an arbitrary direction, and the predetermined ratio is greater than 1; the seventh determination submodule is used to determine that the point cloud to be identified in the intensity abnormality area is not an obstacle point cloud when the ratio of the first length to the second length is less than or equal to the predetermined ratio. Specifically, the above method can determine whether the area after the transformation of the abnormal area in the intensity map meets the condition of being stretched abnormally long, thereby determining whether it is missed and ensuring the accuracy of recognition.

[0066] The above-mentioned obstacle detection device includes a processor and a memory. The above-mentioned acquisition unit, calculation unit, processing unit, first determination unit and second determination unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0067] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be configured, and kernel parameters can be adjusted to address the existing problem of low reliability of the ground plane equation, which leads to missed and false obstacle detections.

[0068] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0069] An embodiment of the present invention provides a computer-readable storage medium having a program stored thereon, which implements the above method when executed by a processor.

[0070] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method is executed when the program is run.

[0071] An embodiment of the present invention provides a device, comprising a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are performed:

[0072] Step S101, obtaining a point cloud to be identified, where the point cloud to be identified is a point cloud of an object within the field of view of the depth camera;

[0073] Step S102, calculating the ground plane equation in real time based on the point cloud to be identified, where the ground plane equation is the equation of the current ground in the robot coordinate system;

[0074] Step S103, moving the plane corresponding to the ground plane equation along the positive direction of the Z axis of the robot coordinate system by a first predetermined value to obtain a segmentation plane;

[0075] Step S104: determining a prepared obstacle point cloud according to the segmentation plane, wherein the prepared obstacle point cloud is the point cloud to be identified that is located on the side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system;

[0076] Step S105 : Identify the prepared obstacle point cloud to determine the obstacle point cloud, where the obstacle point cloud is the point cloud calibrated as an obstacle.

[0077] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0078] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program for initializing at least the following method steps:

[0079] Step S101, obtaining a point cloud to be identified, where the point cloud to be identified is a point cloud of an object within the field of view of the depth camera;

[0080] Step S102, calculating the ground plane equation in real time based on the point cloud to be identified, where the ground plane equation is the equation of the current ground in the robot coordinate system;

[0081] Step S103, moving the plane corresponding to the ground plane equation along the positive direction of the Z axis of the robot coordinate system by a first predetermined value to obtain a segmentation plane;

[0082] Step S104: determining a prepared obstacle point cloud according to the segmentation plane, wherein the prepared obstacle point cloud is the point cloud to be identified that is located on the side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system;

[0083] Step S105 : Identify the prepared obstacle point cloud to determine the obstacle point cloud, where the obstacle point cloud is the point cloud calibrated as an obstacle.

[0084] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the above-mentioned units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0086] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0087] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0088] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a computer-readable storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present invention. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0089] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0090] 1) In the obstacle detection method of the present application, first, a point cloud to be identified is obtained, which is a point cloud of objects within the field of view of the depth camera; then, a ground plane equation is calculated in real time based on the point cloud to be identified, which is the equation of the current ground in the robot coordinate system; then, the plane corresponding to the ground plane equation is moved by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane; then, a preliminary obstacle point cloud is determined based on the segmentation plane, which is the point cloud to be identified on the side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system; finally, the preliminary obstacle point cloud is identified to determine the obstacle point cloud, which is the point cloud calibrated as an obstacle. By calculating the ground plane equation in real time, the problem of unreliable ground plane equation caused by a large difference between the ground plane equation and the ground plane equation of flat ground when the depth camera shakes as the robot moves is avoided, the reliability of the ground plane equation is improved, and the problem of low reliability of the ground plane equation leading to missed obstacle detection and false detection in the prior art is solved.

[0091] 2) In the obstacle detection device of the present application, an acquisition unit acquires a point cloud to be identified, the point cloud to be identified being a point cloud of objects within the field of view of the depth camera; a calculation unit calculates a ground plane equation in real time based on the point cloud to be identified, the ground plane equation being the equation of the current ground in the robot coordinate system; a processing unit moves the plane corresponding to the ground plane equation by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane; a first determination unit determines a prepared obstacle point cloud based on the segmentation plane, the prepared obstacle point cloud being the point cloud to be identified located on the side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system; a second determination unit identifies the prepared obstacle point cloud and determines an obstacle point cloud, the obstacle point cloud being a point cloud calibrated as an obstacle. By calculating the ground plane equation in real time, the device avoids the problem of unreliable ground plane equations due to a large difference between the ground plane equation and the ground plane equation of flat ground when the depth camera shakes as the robot moves, thereby improving the reliability of the ground plane equation and solving the problem of low reliability of the ground plane equation in the prior art leading to missed and false detection of obstacles.

[0092] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for detecting an obstacle, characterized in that: The robot includes a depth camera, and the method includes: Obtaining a point cloud to be identified, where the point cloud to be identified is a point cloud of an object within the field of view of the depth camera; Calculate the ground plane equation in real time based on the point cloud to be identified, where the ground plane equation is the equation of the current ground in the robot coordinate system; Moving the plane corresponding to the ground plane equation by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane; Determining a prepared obstacle point cloud according to the segmentation plane, the prepared obstacle point cloud being the point cloud to be identified located on a side of the segmentation plane close to the positive direction of the Z axis of the robot coordinate system; Identifying the prepared obstacle point cloud to determine an obstacle point cloud, where the obstacle point cloud is a point cloud calibrated as an obstacle; Calculating a ground plane equation in real time according to the point cloud to be identified, including: obtaining a priori plane equation, the priori plane equation is an equation of the ground in the robot coordinate system when the robot is on flat ground, and the plane corresponding to the priori plane equation is perpendicular to the Z axis of the robot coordinate system; determining a candidate segmentation plane area according to the priori plane equation, the candidate segmentation plane area is an area between a first plane and a second plane in the robot coordinate system, the first plane is obtained by moving the plane corresponding to the priori plane equation along the positive direction of the Z axis of the robot coordinate system by a second predetermined value, and the second plane is obtained by moving the plane corresponding to the priori plane equation along the negative direction of the Z axis of the robot coordinate system by the second predetermined value, and the positive direction of the Z axis of the robot coordinate system is a direction of increasing height; determining a maximum plane equation according to the point cloud to be identified in the candidate segmentation plane area, the maximum plane equation is an equation of the maximum plane formed by the point cloud to be identified in the candidate segmentation plane area in the robot coordinate system; determining the ground plane equation according to the priori plane equation and the maximum plane equation; Determining a maximum plane equation based on the point cloud to be identified in the candidate area of ​​the segmentation surface includes: dividing the candidate area of ​​the segmentation surface into a plurality of first cells, where the first cells are cubes with a side length of a first length; sampling the point cloud to be identified in each of the first cells to obtain a plurality of preliminary sampling points, where the coordinates of the preliminary sampling points are the coordinates of any point in the point cloud to be identified in the first cell; dividing the candidate area of ​​the segmentation surface into a plurality of second cells, where the second cells are cubes with a side length of a second length, where the second length is greater than the first length; performing secondary sampling on the sampling points in the second cells to obtain a plurality of sampling points, where the sampling points are the centers of gravity of the sampling points in the second cells; and determining the maximum plane equation based on the sampling points, where the maximum plane equation is an equation of the maximum plane formed by the sampling points in the robot coordinate system.

2. The method according to claim 1, characterized in that Determining the ground plane equation according to the priori plane equation and the maximum plane equation includes: Determining the a priori plane equation as the ground plane equation when at least one predetermined condition is satisfied, the predetermined conditions including that a distance between a first point and a second point is greater than a distance threshold, an angle between a first normal vector and a second normal vector is greater than a predetermined angle, and the number of points in the point cloud within the candidate segmentation surface area is less than a point number threshold, the first point is any point on the plane corresponding to the a priori plane equation, whose X-axis coordinate and Y-axis coordinate are both less than a third predetermined value, the second point is any point on the plane corresponding to the maximum plane equation, whose X-axis coordinate and Y-axis coordinate are both less than the third predetermined value, the first normal vector is the normal vector of the plane corresponding to the a priori plane equation, and the second normal vector is the normal vector of the plane corresponding to the maximum plane equation; When all the predetermined conditions are not satisfied, the maximum plane equation is determined to be the ground plane equation.

3. The method according to claim 1, characterized in that The obstacle point cloud includes a suspended obstacle point cloud, a high obstacle point cloud, and a low obstacle point cloud. Identifying the prepared obstacle point cloud to determine the obstacle point cloud includes: Marking the prepared obstacle point cloud with all points whose Z-axis coordinates are greater than a fourth predetermined value as the suspended obstacle point cloud, wherein the fourth predetermined value is greater than the second predetermined value; Marking the prepared obstacle point cloud, where the Z-axis coordinates of some points are greater than the fourth predetermined value, as the high obstacle point cloud; The prepared obstacle point cloud with all points whose Z-axis coordinates are less than or equal to the second predetermined value is calibrated as the low obstacle point cloud.

4. The method according to claim 3, characterized in that After identifying the prepared obstacle point cloud and determining the obstacle point cloud, the method further includes: Determining a neighboring point of the target point based on the target point, the target point being any point in the high-obstacle point cloud or the low-obstacle point cloud, and the neighboring point being a point in the high-obstacle point cloud or the low-obstacle point cloud whose distance from the target point is less than a first predetermined distance; Clustering the target points according to clustering conditions to obtain multiple point cloud blocks, the clustering conditions including a first clustering condition, a second clustering condition, and a third clustering condition, wherein the first clustering condition is that a difference between the Z-axis coordinate of the target point and the Z-axis coordinates of all the neighboring points is greater than a fifth predetermined value, the second clustering condition is that a normal vector angle of the target point is less than or equal to a predetermined angle, a difference between the Z-axis coordinate of the target point and the Z-axis coordinates of all the neighboring points is less than a sixth predetermined value, and the Euclidean distance between the target point and all the neighboring points is less than a second predetermined distance. The normal vector angle is the angle between the local normal vector of the target point and the Z-axis of the robot coordinate system, the local normal vector is the normal vector of the tangent of the surface composed of the target point and the neighboring points at the target point, the second predetermined distance is less than the first predetermined distance, and the sixth predetermined value is less than the fifth predetermined value. The third clustering condition is that the target point that does not satisfy the first clustering condition and the second clustering condition is simultaneously satisfied, the normal vector angle of the target point is greater than the predetermined angle, and the Euclidean distance between the target point and all the neighboring points is less than a third predetermined distance. The plurality of point cloud blocks are spliced ​​together to obtain the obstacle point cloud.

5. The method according to claim 3, characterized in that After identifying the prepared obstacle point cloud and determining the obstacle point cloud, the method further includes: generating a first intensity map based on the point cloud data of the low-obstacle point cloud; determining a high-intensity region according to the first intensity map, wherein the high-intensity region is a region where the intensity of the low-obstacle point cloud is greater than a first predetermined intensity; Determine a shadow area based on the high-intensity area, where the shadow area is an area where point cloud corresponding to the high-intensity area in the first intensity map cannot be collected due to obstruction by obstacles; Whether the low obstacle point cloud is misjudged is determined according to a duty cycle, where the duty cycle is a ratio of an area occupied by the point cloud within the shadow region to an area of ​​the shadow region.

6. The method according to claim 5, characterized in that Determining whether the low obstacle point cloud is misjudged according to the duty cycle includes: When the duty cycle is greater than a predetermined threshold, determining that the low obstacle point cloud is misjudged; When the duty cycle is less than or equal to the predetermined threshold, it is determined that the low obstacle point cloud is not misjudged.

7. The method according to claim 3, characterized in that After identifying the prepared obstacle point cloud and determining the obstacle point cloud, the method further includes: generating a second intensity map according to the point cloud data of the point cloud to be identified; determining an intensity abnormal area according to the second intensity map, the intensity abnormal area being an area where the intensity of the to-be-identified point cloud is greater than a second predetermined intensity or less than a third predetermined intensity, and the third predetermined intensity is less than the second predetermined intensity; performing an inverse perspective transformation on the second intensity map to obtain a transformed intensity map; The transformed intensity map and the intensity abnormal region of the second intensity map are compared to determine whether the to-be-identified point cloud corresponding to the intensity abnormal region is the obstacle point cloud.

8. The method according to claim 7, characterized in that Comparing the transformed intensity map and the intensity abnormal region of the second intensity map to determine whether the to-be-identified point cloud corresponding to the intensity abnormal region is the obstacle point cloud includes: Determining the to-be-identified point cloud of the intensity abnormal region as an obstacle point cloud when a ratio of the first length to the second length is greater than a predetermined ratio, wherein the first length is a length of the intensity abnormal region in the transformed intensity map in a target direction, the second length is a length of the intensity abnormal region in the second intensity map in the target direction, the target direction is an arbitrary direction, and the predetermined ratio is greater than 1; When the ratio of the first length to the second length is less than or equal to the predetermined ratio, it is determined that the to-be-identified point cloud in the intensity abnormal area is not an obstacle point cloud.

9. An obstacle detection device, characterized in that: The robot includes a depth camera, and the device includes: An acquisition unit, configured to acquire a point cloud to be identified, wherein the point cloud to be identified is a point cloud of an object within the field of view of the depth camera; a calculation unit, configured to calculate in real time a ground plane equation based on the point cloud to be identified, wherein the ground plane equation is an equation of the current ground in the robot coordinate system; a processing unit, configured to move the plane corresponding to the ground plane equation by a first predetermined value along the positive direction of the Z axis of the robot coordinate system to obtain a segmentation plane; a first determining unit, configured to determine a prepared obstacle point cloud according to the segmentation plane, wherein the prepared obstacle point cloud is the to-be-identified point cloud located on a side of the segmentation plane close to a positive direction of a Z-axis of the robot coordinate system; a second determining unit, configured to identify the prepared obstacle point cloud and determine an obstacle point cloud, wherein the obstacle point cloud is a point cloud calibrated as an obstacle; The calculation unit includes an acquisition module, a first determination module, a second determination module and a third determination module, wherein the acquisition module is used to acquire a priori plane equation, the priori plane equation is an equation of the ground in the robot coordinate system when the robot is on flat ground, and the plane corresponding to the priori plane equation is perpendicular to the Z axis of the robot coordinate system; the first determination module is used to determine a candidate segmentation plane area according to the priori plane equation, the candidate segmentation plane area is an area between a first plane and a second plane in the robot coordinate system, the first plane is obtained by moving the plane corresponding to the priori plane equation along the positive direction of the Z axis of the robot coordinate system by a second predetermined value, and the second plane is obtained by moving the plane corresponding to the priori plane equation along the negative direction of the Z axis of the robot coordinate system by the second predetermined value, and the positive direction of the Z axis of the robot coordinate system is a direction of increasing height; the second determination module is used to determine a maximum plane equation according to the to-be-identified point cloud in the candidate segmentation plane area, the maximum plane equation being an equation of the maximum plane formed by the to-be-identified point cloud in the candidate segmentation plane area in the robot coordinate system; the third determination module is used to determine the ground plane equation according to the priori plane equation and the maximum plane equation; The second determination module is used to include a first processing submodule, a first sampling submodule, a second processing submodule, a second sampling submodule and a first determination submodule, wherein the first processing submodule is used to divide the segmentation surface candidate area into multiple first cells, where the first cell is a cube with a side length of a first length; the first sampling submodule is used to sample the point cloud to be identified in each of the first cells to obtain multiple preliminary sampling points, where the coordinates of the preliminary sampling points are the coordinates of any point in the point cloud to be identified in the first cell; the second processing submodule is used to divide the segmentation surface candidate area into multiple second cells, where the second cell is a cube with a side length of a second length, where the second length is greater than the first length; the second sampling submodule performs secondary sampling on the sampling points in the second cell to obtain multiple sampling points, where the sampling points are the centers of gravity of the sampling points in the second cell; the first determination submodule is used to determine the maximum plane equation based on the sampling points, where the maximum plane equation is the equation of the maximum plane formed by the sampling points in the robot coordinate system.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 8.

11. A processor, characterized in that: The processor is configured to run a program, wherein the program executes the method according to any one of claims 1 to 8 when running.

Images