Obstacle detection method and device and storage medium

Abstract

The invention discloses an obstacle detection method and device and a storage medium, and the method comprises the steps: obtaining point cloud data of a to-be-detected environment, selecting a plurality of seed points from the point cloud data, and selecting to-be-clustered points from other points except the plurality of seed points; dividing the plurality of seed points into ground seed points and obstacle seed points based on the point cloud coordinates of the plurality of seed points; obtaining growth conditions of the ground seed points, and dividing the to-be-clustered points meeting the growth conditions of the ground seed points into clustering clusters of the ground seed points to obtain ground clustering clusters; obtaining the growth conditions of the obstacle seed points, dividing the to-be-clustered points meeting the growth conditions of the obstacle seed points into the clustering clusters of the obstacle seed points, and obtaining obstacle clustering clusters; wherein growth can be carried out through a dynamic step length, and the clustering efficiency is improved; and based on the ground clustering cluster and the obstacle clustering cluster, determining the obstacle pose information of the to-be-detected environment. According to the scheme, the obstacle detection precision can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of point cloud data processing, and in particular, to an obstacle detection method, device, and storage medium. Background Art

[0002] Point cloud detection is one of the very popular detection technologies today and is often used in the fields of robotics, autonomous driving, etc.

[0003] For example, in the field of robotics, obstacle detection based on point clouds is one of the key links to ensure the autonomous operation of robots. A fast and accurate obstacle detection method can provide reliable data support and decision-making information for links such as map construction, path planning, and motion control of robots.

[0004] However, in current obstacle detection methods, when using devices such as lidar and depth cameras to perceive the environment, affected by terrain undulation and data noise, it is often impossible to correctly segment the ground and obstacles, resulting in low accuracy of obstacle detection. Summary of the Invention

[0005] The present application provides at least an obstacle detection method, device, equipment, and computer-readable storage medium.

[0006] In a first aspect of the present application, an obstacle detection method is provided, including:

[0007] Obtaining current point cloud data corresponding to an environment to be detected, selecting a plurality of seed points from the current point cloud data, and selecting points to be clustered from other points except the plurality of seed points; based on the point cloud coordinates of the plurality of seed points, dividing the plurality of seed points into ground seed points and obstacle seed points respectively; obtaining the growth conditions corresponding to the ground seed points, and dividing the points to be clustered that meet the growth conditions corresponding to the ground seed points into the cluster corresponding to the ground seed points to obtain a ground cluster; and obtaining the growth conditions corresponding to the obstacle seed points, and dividing the points to be clustered that meet the growth conditions corresponding to the obstacle seed points into the cluster corresponding to the obstacle seed points to obtain an obstacle cluster; determining the pose information of obstacles in the environment to be detected based on the ground cluster and the obstacle cluster.

[0008] In one embodiment, the selecting the point to be clustered from other points except the multiple seed points includes: obtaining a region growing step length and determining a current growing origin corresponding to the current seed point; wherein the current growing origin includes the current seed point or a growing point corresponding to the current seed point, and the growing point refers to a point belonging to the same clustering cluster as the current seed point; starting from the current growing origin, selecting the point cloud data in the current point cloud data under the region growing step length to obtain the point to be clustered; the method further includes: if the point to be clustered does not meet the growing condition corresponding to the current seed point, reducing the region growing step length and re-selecting the point to be clustered starting from the current growing origin; if the point to be clustered meets the growing condition corresponding to the current seed point, dividing the point cloud data between the current growing origin and the point to be clustered into the clustering cluster corresponding to the current seed point.

[0009] In one embodiment, the number of growing conditions corresponding to the current seed point is multiple; the if the point to be clustered does not meet the growing condition corresponding to the current seed point, reducing the region growing step length and re-selecting the point to be clustered starting from the current growing origin includes: if the point to be clustered does not meet all the growing conditions corresponding to the current seed point, reducing the region growing step length and re-selecting the point to be clustered starting from the current growing origin; if the point to be clustered meets some of the growing conditions corresponding to the current seed point and the number of the growing conditions met is greater than or equal to a preset condition number threshold, using the adjacent point corresponding to the point to be clustered as an auxiliary discrimination point, and if the auxiliary discrimination point meets all the growing conditions corresponding to the current seed point, dividing the point cloud data between the current growing origin and the auxiliary discrimination point into the clustering cluster corresponding to the current seed point.

[0010] In one embodiment, obtaining the growth conditions corresponding to the ground seed points, and dividing the to-be-clustered points that meet the growth conditions corresponding to the ground seed points into the clustering clusters corresponding to the ground seed points to obtain the ground clustering clusters includes: determining the plane represented by the normal vector corresponding to the ground seed points to obtain the ground plane, calculating the distance between the to-be-clustered points and the ground plane to obtain the ground distance; and calculating the distance between the to-be-clustered points and the current growth origin corresponding to the ground seed points to obtain the ground point distance; obtaining the ground distance threshold and the ground point distance threshold corresponding to the ground seed points; dividing the to-be-clustered points with the ground distance less than the ground distance threshold and / or the ground point distance less than the ground point distance threshold into the clustering clusters corresponding to the ground seed points to obtain the ground clustering clusters; and / or, obtaining the growth conditions corresponding to the obstacle seed points, and dividing the to-be-clustered points that meet the growth conditions corresponding to the obstacle seed points into the clustering clusters corresponding to the obstacle seed points to obtain the obstacle clustering clusters includes: calculating the distance between the to-be-clustered points and the current growth origin corresponding to the obstacle seed points to obtain the obstacle point distance; obtaining the obstacle point distance threshold corresponding to the obstacle seed points; dividing the to-be-clustered points with the obstacle distance less than the obstacle point distance threshold into the clustering clusters corresponding to the obstacle seed points to obtain the obstacle clustering clusters.

[0011] In one embodiment, the current point cloud data is sampled from a depth image, and each point in the current point cloud data corresponds to a depth value; calculating the distance between the to-be-clustered points and the current growth origin corresponding to the ground seed points to obtain the ground point distance includes: calculating the distance between the to-be-clustered points and the current growth origin corresponding to the ground seed points based on the point cloud coordinates of the to-be-clustered points and the point cloud coordinates of the current growth origin corresponding to the ground seed points to obtain the first initial distance; obtaining the depth value of the to-be-clustered points; compensating the value of the first initial distance based on the depth value of the to-be-clustered points to obtain the ground point distance; and / or, calculating the distance between the to-be-clustered points and the current growth origin corresponding to the obstacle seed points to obtain the obstacle point distance includes: calculating the distance between the to-be-clustered points and the current growth origin corresponding to the obstacle seed points based on the point cloud coordinates of the to-be-clustered points and the point cloud coordinates of the current growth origin corresponding to the obstacle seed points to obtain the second initial distance; compensating the value of the second initial distance based on the depth value of the to-be-clustered points to obtain the obstacle point distance.

[0012] In one embodiment, the obtaining of the current point cloud data corresponding to the environment to be detected and the selection of a plurality of seed points from the current point cloud data include: obtaining a depth image corresponding to the environment to be detected and converting the depth image into the current point cloud data; determining a downsampling multiple matching the size of the depth image; and selecting a plurality of seed points from the current point cloud data by using the downsampling multiple.

[0013] In one embodiment, the selecting of a plurality of seed points from the current point cloud data by using the downsampling multiple includes: performing point selection from the current point cloud data by using the downsampling multiple to obtain candidate points; obtaining the depth values of adjacent pixels of the candidate points in a preset orientation and counting the number of adjacent pixels with valid depth values; and if the number of the adjacent pixels is greater than or equal to a preset pixel number threshold, taking the candidate points as the seed points.

[0014] In one embodiment, the dividing of the plurality of seed points into ground seed points and obstacle seed points respectively based on the point cloud coordinates of the plurality of seed points includes: obtaining a ground plane model obtained by fitting a ground plane based on previous point cloud data or a pre-calibrated ground plane model, where the previous point cloud data refers to the point cloud data collected before the current point cloud data; calculating the distance between the seed points and the ground plane model based on the point cloud coordinates of the seed points to obtain a first distance; obtaining a first distance threshold, selecting the seed points with the first distance less than the first distance threshold to obtain candidate seed points; performing plane fitting on the candidate seed points to obtain an optimized ground plane; calculating the distance between the candidate seed points and the optimized ground plane based on the point cloud coordinates of the candidate seed points to obtain a second distance; and calculating the angle between the candidate seed points and the optimized ground plane based on the normal vectors of the candidate seed points to obtain a normal vector angle; obtaining a second distance threshold and a normal vector angle threshold, selecting the candidate seed points with the second distance less than the second distance threshold and the normal vector angle less than the normal vector angle threshold to obtain the ground seed points, and taking the other seed points as the obstacle seed points.

[0015] The second aspect of the present application provides an obstacle detection device, including: a point cloud selection module, configured to obtain current point cloud data corresponding to an environment to be detected, select a plurality of seed points from the current point cloud data, and select points to be clustered from other points except the plurality of seed points; a point cloud division module, configured to divide the plurality of seed points into ground seed points and obstacle seed points respectively based on the point cloud coordinates of the plurality of seed points; a point cloud clustering module, configured to obtain the growth conditions corresponding to the ground seed points, divide the points to be clustered that meet the growth conditions corresponding to the ground seed points into the cluster corresponding to the ground seed points to obtain a ground cluster; and obtain the growth conditions corresponding to the obstacle seed points, divide the points to be clustered that meet the growth conditions corresponding to the obstacle seed points into the cluster corresponding to the obstacle seed points to obtain an obstacle cluster; an obstacle determination module, configured to determine the pose information of the obstacle in the environment to be detected based on the ground cluster and the obstacle cluster.

[0016] The third aspect of the present application provides an electronic device, including a memory and a processor, where the processor is configured to execute program instructions stored in the memory to implement the above obstacle detection method.

[0017] The fourth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the above obstacle detection method is implemented.

[0018] In the above solution, by obtaining the current point cloud data corresponding to the environment to be detected, a plurality of seed points can be selected from the current point cloud data, and points to be clustered can be selected from other points except the plurality of seed points; based on the point cloud coordinates of the plurality of seed points, the plurality of seed points are divided into ground seed points and obstacle seed points respectively to realize the classification of the seed points for constructing different region growing channels; the growth conditions corresponding to the ground seed points are obtained, and the points to be clustered that meet the growth conditions corresponding to the ground seed points are divided into the cluster corresponding to the ground seed points to obtain a ground cluster; and the growth conditions corresponding to the obstacle seed points are obtained, and the points to be clustered that meet the growth conditions corresponding to the obstacle seed points are divided into the cluster corresponding to the obstacle seed points to obtain an obstacle cluster. Thus, region growing can be performed by adopting different growth strategies to obtain a ground point cloud cluster and an obstacle point cloud cluster; then, based on the ground cluster and the obstacle cluster, the ground point cloud and the obstacle point cloud can be accurately distinguished, and the pose information of the obstacle in the environment to be detected can be determined, thereby improving the detection accuracy of the obstacle.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application. Description of the Drawings

[0020] The accompanying drawings here are incorporated into the description and form a part of this description. These drawings illustrate embodiments consistent with the present application and, together with the description, are used to explain the technical solutions of the present application.

[0021] Figure 1 is a schematic flowchart of an exemplary embodiment of the obstacle detection method of the present application;

[0022] Figure 2 is a schematic overall flowchart of an exemplary embodiment of the obstacle detection method of the present application;

[0023] Figure 3 is a block diagram of an obstacle detection device shown in an exemplary embodiment of the present application;

[0024] Figure 4 is a schematic structural diagram of an embodiment of an electronic device of the present application;

[0025] Figure 5 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. Detailed Embodiments

[0026] The following will, with reference to the accompanying drawings of the description, elaborate on the solutions of the embodiments of the present application in detail.

[0027] In the following description, specific details such as specific system architectures, interfaces, technologies, etc. are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the present application.

[0028] The term "and / or" in this document is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after. Furthermore, the term "multiple" in this document means two or more. In addition, the term "at least one" in this document represents any one of multiple or any combination of at least two of multiple. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0029] A fast and accurate obstacle detection method can provide reliable data support and decision-making information for links such as map construction, path planning, and motion control of mobile robots, and play a key role in application fields such as driverless, logistics robots, service robots, and home intelligent robots. The present application mainly takes the robot application scenario as an example for illustration, but does not limit the specific available scenarios of the present application.

[0030] Obstacle detection usually involves extracting the features of obstacles in the current driving area of a robot to obtain their pose information and size information, and then making decisions and planning for the robot's next task or driving path. When using devices such as lidar and depth cameras to perceive the environment, affected by terrain undulations and data noise, existing obstacle detection methods often cannot correctly segment the ground and obstacles. In addition, limited by the hardware computing power of mobile robots, obstacle detection may not meet the real-time requirements. Therefore, a fast and accurate obstacle detection algorithm is crucial for the mobile robot platform.

[0031] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an exemplary embodiment of the obstacle detection method of the present application. Specifically, it may include the following steps:

[0032] Step S110: Obtain the current point cloud data corresponding to the environment to be detected, select multiple seed points from the current point cloud data, and select the points to be clustered from the other points except the multiple seed points.

[0033] Among them, the current point cloud data can be collected by a lidar and / or by a depth camera, that is, it is equivalent to that the device applied by the obstacle detection method provided by the present application can be provided with a lidar and / or a depth camera, which is not limited here. The present application mainly takes determining the point cloud data through a depth image as an example for subsequent description and will not elaborate here for the time being. It should be noted that the current point cloud data can be the initially collected point cloud data or the point cloud data after certain preprocessing.

[0034] The seed point refers to the pixel or point (point cloud) that serves as the starting point for growth in the region growing algorithm. The role of the seed point is to start the growth process and expand outward with this as the center, merging adjacent pixels or point clouds with the same or similar attributes into the same region. Regarding the specific principle of the region growing algorithm, reference can be made to existing explanations and will not be elaborated here.

[0035] The points to be clustered refer to the point cloud data that needs to be clustered to the corresponding seed points (with the same or similar attributes).

[0036] Exemplarily, for the method of selecting seed points, one (or more) pixels or points can be randomly selected as seed points, or pixels or points that meet preset specific conditions can be selected as seed points according to certain features of the image or point cloud (such as color, brightness, etc.), which is not limited here. For the obtained current point cloud data, after determining the seed points in the current point cloud data, the other point cloud data (all or part) in the current point cloud data except the seed points can be determined as the points to be clustered.

[0037] Step S120: Based on the point cloud coordinates of multiple seed points, divide the multiple seed points into ground seed points and obstacle seed points respectively.

[0038] It can be understood that the point cloud data has corresponding point cloud coordinates. Therefore, in this application, the seed points can be classified through the point cloud coordinates of the seed points, and the multiple seed points are divided into ground seed points and obstacle seed points respectively. Among them, the seed points can be classified according to the distance between the point cloud coordinates and the specified plane. For example, the seed points close to the ground (the seed points with the distance between the point cloud data and the specified plane less than the preset distance threshold) are determined as ground seed points, and the other seed points except the ground seed points among all the seed points are determined as obstacle seed points.

[0039] Step S130: Obtain the growth conditions corresponding to the ground seed points, and divide the points to be clustered that meet the growth conditions corresponding to the ground seed points into the cluster corresponding to the ground seed points to obtain the ground cluster; and, obtain the growth conditions corresponding to the obstacle seed points, and divide the points to be clustered that meet the growth conditions corresponding to the obstacle seed points into the cluster corresponding to the obstacle seed points to obtain the obstacle cluster.

[0040] Combined with the foregoing steps, in this application, the region growing algorithm can be used to expand with the seed points as the growth starting points, and the points to be clustered that meet the growth conditions of the seed points are divided into the clusters (also called point cloud clusters) corresponding to the seed points, so as to realize the clustering of the point cloud data.

[0041] It should be noted that the seed points in this application include ground seed points and obstacle seed points. Therefore, the growth conditions adopted when performing growth clustering for ground seed points in this application may be the same or different from the growth conditions adopted when performing growth clustering for obstacle seed points, which is not limited here.

[0042] Exemplarily, in the following of this application, it is mainly described by taking the growth conditions corresponding to the ground seed points being different from the growth conditions corresponding to the obstacle seed points as an example, and thus a dual-channel region growing method is proposed. Taking the ground seed points as the growth starting points, similar points to be clustered are clustered into the cluster corresponding to the ground seed points through the growth conditions corresponding to the ground seed points to obtain the ground cluster (ground point cloud cluster); taking the obstacle seed points as the growth starting points, similar points to be clustered are clustered into the cluster corresponding to the obstacle seed points through the growth conditions corresponding to the obstacle seed points to obtain the obstacle cluster (obstacle point cloud cluster). Thereby, the accuracy of region growing can be improved, and the accuracy of obstacle detection can also be improved. Among them, this application does not limit the number of point cloud clusters. For example, the number of ground point cloud clusters can be one or more, and the number of obstacle point cloud clusters can be one or more.

[0043] Step S140: Determine the pose information of obstacles in the environment to be detected based on the ground clustering clusters and the obstacle clustering clusters.

[0044] In combination with the foregoing steps, the ground clustering clusters represent to a certain extent the point cloud data belonging to the ground, and the obstacle clustering clusters represent to a certain extent the point cloud data belonging to the obstacles. In some cases, the pose information of the obstacles can be directly determined from the point cloud data of the obstacle clustering clusters. However, in different application scenarios, the accuracy of obtaining the clustering clusters through the above clustering process does not necessarily meet the requirements. Therefore, after obtaining the ground clustering clusters and the obstacle clustering clusters, the present application can perform combined analysis on the ground clustering clusters and the obstacle clustering clusters, perform misclassification judgment on the point cloud data in the clustering clusters, and then perform optimization methods such as category correction on the misclassified points to improve the quality of the point cloud data in the clustering clusters, and finally obtain the complete ground point cloud and different obstacle point cloud instances, thereby improving the accuracy of point cloud clustering.

[0045] It can be seen that by obtaining the current point cloud data corresponding to the environment to be detected, the present application can select multiple seed points from the current point cloud data, and select points to be clustered from other points except the multiple seed points; based on the point cloud coordinates of the multiple seed points, the multiple seed points are respectively divided into ground seed points and obstacle seed points to classify the seed points so as to construct different region growing channels; obtain the growth conditions corresponding to the ground seed points, and divide the points to be clustered that meet the growth conditions corresponding to the ground seed points into the clustering clusters corresponding to the ground seed points to obtain the ground clustering clusters; and obtain the growth conditions corresponding to the obstacle seed points, and divide the points to be clustered that meet the growth conditions corresponding to the obstacle seed points into the clustering clusters corresponding to the obstacle seed points to obtain the obstacle clustering clusters. Thus, the ground point cloud clusters and the obstacle point cloud clusters can be obtained by performing region growing using different growth strategies; then based on the ground clustering clusters and the obstacle clustering clusters, the ground point cloud and the obstacle point cloud can be accurately distinguished, and the pose information of the obstacles in the environment to be detected can be determined, thereby improving the detection accuracy of the obstacles.

[0046] Based on the above embodiments, the present application embodiment describes the step of selecting points to be clustered from other points except the multiple seed points. Specifically, the method of this embodiment includes the following steps:

[0047] Obtain the region growing step size and determine the current growth origin corresponding to the current seed point; wherein, the current growth origin includes the current seed point or the growth point corresponding to the current seed point, and the growth point refers to the point belonging to the same clustering cluster as the current seed point; starting from the current growth origin, select the point cloud data in the current point cloud data under the region growing step size to obtain the points to be clustered; the method further includes: if the points to be clustered do not meet the growth conditions corresponding to the current seed point, then reduce the region growing step size and re-select the points to be clustered starting from the current growth origin; if the points to be clustered meet the growth conditions corresponding to the current seed point, then divide the point cloud data between the current growth origin and the points to be clustered into the clustering cluster corresponding to the current seed point.

[0048] Combined with the foregoing embodiments for illustration, after determining each seed point, the points to be clustered can be selected from the remaining current point cloud data except the seed points.

[0049] Among them, the region growing step size refers to the parameter representing the degree of each clustering merge in the region growing algorithm, such as the number of steps or distance of pixel or region merge in each iteration process, etc. It can be a fixed step size preset according to empirical values, or it can be dynamically adjusted. This application mainly takes the dynamic step size as an example for illustration.

[0050] Exemplarily, if the point cloud data is determined through a depth image in the specific implementation process of this application (for example, the depth image can be converted into a three-dimensional point cloud through the internal and external parameters of the depth camera), then the initial step size can be set according to the image size of the depth image. For example, taking a depth image of 640×480 as an example, the initial step size can be set to 10. According to the depth value D of the seed point i The step size step corresponding to different seed points can be dynamically adjusted, and its mathematical expression can be:

[0051] step = 10 + λD i

[0052] where i refers to the i-th pixel of the depth image, and λ is an auxiliary parameter, which can take 0.5.

[0053] It should be noted that when using the initial Step for region growing, it may cross the object boundary and cause omission. Therefore, a step size dynamic adjustment mechanism during the growth process is proposed in this application: for the current seed point, when growing for the first time, the initial large step size Step is used. If the point cloud data corresponding to this step size meets the current growth condition (determined by the corresponding seed point type), all the pixel points passed through during the growth along the way can be marked and classified into the same category, and they are divided into the clustering cluster corresponding to the current seed point. If there is point cloud data that does not meet the current growth condition under this step size, the initial step size Step can be reduced (for example, directly halved), and then judged again until the growth condition is met or step is equal to the preset step size threshold (for example, Step = 1). In addition, when the downsampling multiple is not 1, the initial step size needs to be adjusted as Step = Step / α to ensure that the step size and the depth map size are reduced in proportion.

[0054] It should also be noted that when clustering by region growing, the current seed point (which can include ground seed points and / or obstacle seed points) can be determined as the growth origin, or the growth point obtained by growing based on the current seed point (the latest grown growth point can be selected) can be determined as the growth origin. Among them, the growth point needs to belong to the same clustering cluster as the current seed point. Thus, taking the current growth origin as the starting point of growth, the point cloud data within the region growing step size range is selected from the current point cloud data, and the points to be clustered corresponding to the current growth origin (that is, equivalent to the points to be clustered corresponding to the current seed point) can be obtained.

[0055] Optionally, for each current growth origin, the adjacent and unclassified point cloud of the current growth origin can be directly determined as the corresponding points to be clustered.

[0056] In summary, if a certain point to be clustered of the current seed point does not meet its corresponding growth condition (or the number of points to be clustered that do not meet the growth condition is greater than the preset quantity threshold), the region growing step size can be reduced with reference to the foregoing embodiments, and the points to be clustered are selected again. If all the points to be clustered of the current seed point meet their corresponding growth conditions (or the data of the points to be clustered that do not meet the growth condition is less than or equal to the preset quantity threshold), all the point cloud data between the current growth origin and the points to be clustered can be divided into the clustering cluster corresponding to the current seed point.

[0057] Based on the above embodiments, the embodiments of this application illustrate the steps of reducing the region growing step size and re-selecting the points to be clustered with the current growth origin as the starting point if the points to be clustered do not meet the growth condition corresponding to the current seed point. Among them, the number of growth conditions corresponding to the current seed point is multiple. Specifically, the method of this embodiment includes the following steps:

[0058] If the point to be clustered does not meet all the growth conditions corresponding to the current seed point, the region growth step size is decreased, and the selection of the point to be clustered is restarted with the current growth origin as the starting point; if the point to be clustered meets some of the growth conditions corresponding to the current seed point and the number of growth conditions met is greater than or equal to the preset condition number threshold, the adjacent points corresponding to the point to be clustered are used as auxiliary discriminant points. If the auxiliary discriminant points meet all the growth conditions corresponding to the current seed point, the point cloud data between the current growth origin and the auxiliary discriminant points is divided into the cluster corresponding to the current seed point.

[0059] Combined with the foregoing embodiments for illustration, in the present application, the growth conditions corresponding to the ground seed points and the growth conditions corresponding to the obstacle seed points may be the same or different. There may be one or more growth conditions corresponding to the ground seed points, and there may be one or more growth conditions corresponding to the obstacle seed points, which are not limited herein.

[0060] In this embodiment, mainly multiple growth conditions are taken as examples for illustration. The points to be clustered can be classified into strongly related points and weakly related points to the current seed point according to the number of growth conditions met by the points to be clustered, and then different strategies are adopted for clustering processing respectively to improve the clustering accuracy. Among them, the strongly related points to the current seed point refer to the points to be clustered that meet all the growth conditions corresponding to the current seed point, and the weakly related points to the current seed point refer to the points to be clustered that meet some of the growth conditions corresponding to the current seed point.

[0061] It can be understood that for the points to be clustered that meet all the growth conditions corresponding to the current seed point, reference can be made to the foregoing examples for illustration and they can be clustered into the cluster corresponding to the current seed point. For the points to be clustered that do not meet all the growth conditions corresponding to the current seed point, reference can be made to the foregoing examples for illustration, the region growth step size is decreased, and the points to be clustered are reselected according to the adjusted step size.

[0062] If the point to be clustered meets some of the growth conditions corresponding to the current seed point and the number of growth conditions met is greater than or equal to the preset condition number threshold (for example, the total number of growth conditions can be 3, the number of growth conditions met by the current point to be clustered is 1, and the preset condition number threshold is 1), the adjacent points corresponding to the point to be clustered can be used as auxiliary discriminant points. If the auxiliary discriminant points meet all the growth conditions corresponding to the current seed point, all the point cloud data between the current growth origin and the auxiliary discriminant points can be divided into the cluster corresponding to the current seed point.

[0063] Among them, the method for determining the adjacent points of the point to be clustered can be as follows: when growing the ground seed points, for the current point to be clustered, two corresponding points with an interval of Step in the same vertical (column) direction in its depth image are used as adjacent points, and the growth conditions of the adjacent points are judged. If at least one adjacent point meets all the growth conditions, all the point clouds on the path from the current growth origin to this adjacent point in this row are classified into the corresponding clustering cluster S i and the current point (the adjacent point that meets all the growth conditions) can be used as the new growth origin, and the above example steps are repeated until there are no new points to be clustered that can be classified. Subsequently, according to the row (row) and column (col) of the depth image, the column numbers col start ,col end of the first and last pixels of the classified points in this row are recorded, and they are diffused to the adjacent two rows row±1 with the same length. If the first and last col' start ,col' end in the adjacent row also meet the above conditions, the adjacent row is also classified until there are no new candidate points that can be classified.

[0064] Thus, through the region growth mechanism based on dynamic step size proposed in the foregoing embodiments of the present application, the point cloud clustering speed will be greatly improved.

[0065] Based on the above embodiments, the embodiments of the present application illustrate the steps of obtaining the growth conditions corresponding to the ground seed points, and dividing the points to be clustered that meet the growth conditions corresponding to the ground seed points into the clustering clusters corresponding to the ground seed points to obtain the ground clustering clusters. Specifically, the method of this embodiment includes the following steps:

[0066] Determine the plane represented by the normal vector corresponding to the ground seed point to obtain the ground plane, calculate the distance between the point to be clustered and the ground plane to obtain the ground distance; and calculate the distance between the point to be clustered and the current growth origin corresponding to the ground seed point to obtain the ground point distance; obtain the ground distance threshold and the ground point distance threshold corresponding to the ground seed point; divide the points to be clustered with the ground distance less than the ground distance threshold and / or the ground point distance less than the ground point distance threshold into the clustering clusters corresponding to the ground seed point to obtain the ground clustering clusters.

[0067] Among them, there can be one or more growth conditions in the present application, which are not limited here. In this embodiment, three growth conditions corresponding to the ground seed points are exemplarily provided. The main purpose is to ensure that the growth points and the origin set belong to the same plane point set through the Euclidean distance and the distance from a point to a specified plane. Specifically, the following conditions need to be met: A1 the current point to be clustered has not been classified; A2 the Euclidean distance (ground point distance) from the current point to be clustered to the current growth origin corresponding to the ground seed point is less than the preset ground point distance threshold dis point; The distance from the current candidate point of A3 to the plane represented by the ground seed point and its normal vector (ground distance) is less than the preset ground distance threshold dis plane . Among them, the threshold dis point and dis plane are dynamic thresholds, which can be jointly determined by the initial point cloud spacing, the candidate point depth value, the downsampling multiple α of the depth image, and the dynamic step size Step. For example, for the points to be clustered with a larger step size or a larger depth value, a larger distance threshold will be used. Its mathematical expression can be:

[0068] dis point = Step·(α·τ1 + λ1·d i )

[0069] dis plane = α·τ2 + λ2·d i

[0070] Among them, τ1 and τ2 are the initial thresholds of the point-to-point distance and the point-to-plane distance respectively, λ1 and λ2 are auxiliary parameters, and d i is the depth value of the current point to be clustered.

[0071] The thresholds involved in different growth conditions can be the same or different. The growth conditions can be either preset or flexibly adjusted according to the actual scenario (for example, according to the environmental type to which the environment to be detected belongs, the work type of the robot performing the work, etc.).

[0072] Optionally, in the specific implementation process of this embodiment, in addition to clustering according to the ground point distance and the ground distance, it can also be clustering according to the ground point distance or the ground distance. The threshold comparison process in the foregoing example can be referred to similarly, and will not be elaborated here.

[0073] Exemplarily, according to the above 3 growth conditions, with the ground seed point as the origin of region growth, the dynamic step size Step in the foregoing embodiment is used to perform region growth in the current row row, and each point to be clustered is judged for conditions A1 to A3. If there are points to be clustered that cannot meet all conditions, the step size Step is halved and the judgment is performed again. If two conditions are met, the two corresponding points (adjacent points) with an interval of Step in the same vertical (column) direction are judged. For specific details, reference can be made to the description of the foregoing embodiment, and will not be elaborated here.

[0074] This embodiment also describes the steps of obtaining the growth conditions corresponding to the obstacle seed points, and dividing the points to be clustered that meet the growth conditions corresponding to the obstacle seed points into the clustering clusters corresponding to the obstacle seed points to obtain the obstacle clustering clusters. Specifically, the method of this embodiment may further include the following steps:

[0075] Calculate the distance between the point to be clustered and the current growth origin corresponding to the obstacle seed point to obtain the obstacle point distance; obtain the obstacle point distance threshold corresponding to the obstacle seed point; divide the point to be clustered with the obstacle distance less than the obstacle point distance threshold into the cluster corresponding to the obstacle seed point to obtain the obstacle cluster.

[0076] For the growth process of the obstacle seed point, the obstacle seed point can grow based on its connected domain area.

[0077] Combined with the foregoing steps, after the plane area growth of the ground seed point, the ground cluster S with ground point marks is obtained i , i ∈ [1, n], where n is the number of point clouds in the ground cluster. On this basis, the constraint of condition A3 in the foregoing example does not need to be considered, and the connected domain area growth is performed on the remaining unclassified points to be clustered. At this time, the principles such as condition A1 and A2 can be considered (at this time, the distance to be calculated is the distance between the point to be clustered and the current growth origin corresponding to the obstacle seed point, that is, the obstacle point distance, and then the obstacle point distance threshold corresponding to the obstacle seed point is compared with the obstacle point distance of the point to be clustered for division), to ensure that it is not classified as a ground point cloud and is connected in space, and the obstacle cluster is obtained. For example, the obstacle seed point can be used as the current growth origin, and similarly, according to the steps provided in the foregoing embodiment, the unclassified points to be clustered in this row are first grown with a dynamic step length until there are no new unclassified points, and then diffused and grown to the adjacent two rows with the same length according to the first and last column numbers to complete the clustering of the obstacle point cloud of the current category, and the obstacle cluster S is obtained n+i .

[0078] Based on the above embodiments, the embodiments of the present application illustrate the steps of calculating the distance between the point to be clustered and the current growth origin corresponding to the ground seed point to obtain the ground point distance. Among them, the current point cloud data is obtained by sampling a depth image, and each point in the current point cloud data corresponds to a depth value. Specifically, the method of this embodiment includes the following steps:

[0079] Based on the point cloud coordinates of the point to be clustered and the point cloud coordinates of the current growth origin corresponding to the ground seed point, calculate the distance between the point to be clustered and the current growth origin corresponding to the ground seed point to obtain the first initial distance; obtain the depth value of the point to be clustered; compensate the value of the first initial distance based on the depth value of the point to be clustered to obtain the ground point distance.

[0080] It should be noted that in the growth processes of the two types of point cloud regions (ground point cloud and obstacle point cloud) of the present application, the following strategies are also set to improve the accuracy of region growth.

[0081] For the growth process of ground seed points, on the one hand, considering the sensor characteristics, taking a depth camera as an example, affected by the depth map resolution, the point cloud captured by the depth camera will have a layering phenomenon at different depths, resulting in a large distance calculation error in the depth direction. Therefore, when calculating the Euclidean distance between two points, the coordinate difference corresponding to the depth direction can be compensated. Therefore, the method for calculating the ground point distance can be: based on the point cloud coordinates of the point to be clustered and the point cloud coordinates of the current growth origin corresponding to the ground seed point, calculate the distance between the point to be clustered and the current growth origin corresponding to the ground seed point, and obtain the first initial distance. Then obtain the depth value d of the point to be clustered i , and compensate the value of the first initial distance according to the depth value of the point to be clustered to obtain the ground point distance d rec , and its mathematical expression can be

[0082] d rec =k·d i

[0083] where k is a preset adjustment coefficient, which can be selected according to the spacing of the depth camera point cloud layering. Generally, the final result can be set to about 1 / 2 of the layer spacing

[0084] On the other hand, when using a dynamic step size for region growth, there may be a phenomenon of misclassification at the surface boundary of an object where the depth value has a sudden change (for example, it can be judged according to parameters such as the difference or change rate of the depth values of two points, which will not be elaborated here). Therefore, the growth points, seed points, and the intermediate point set between the two points that meet the growth conditions can be re-judged according to the conditions. Generally, the midpoint or trisection point between the two points can be taken for re-judgment to improve the region growth accuracy

[0085] This embodiment also describes the steps of calculating the distance between the point to be clustered and the current growth origin corresponding to the obstacle seed point to obtain the obstacle point distance. Specifically, the method of this embodiment may further include the following steps

[0086] Based on the point cloud coordinates of the point to be clustered and the point cloud coordinates of the current growth origin corresponding to the obstacle seed point, calculate the distance between the point to be clustered and the current growth origin corresponding to the obstacle seed point to obtain the second initial distance; compensate the value of the second initial distance based on the depth value of the point to be clustered to obtain the obstacle point distance

[0087] For the growth process of obstacle seed points, reference can also be made to the foregoing embodiment to compensate the second initial distance between the point to be clustered and the current growth origin corresponding to the obstacle seed point through the depth value. And when using a dynamic step size for region growth, the growth points, seed points, and the intermediate point set between the two points that meet the growth conditions can also be re-conditionally judged

[0088] Based on the above embodiments, the embodiments of the present application will describe the steps of obtaining the current point cloud data corresponding to the environment to be detected and selecting multiple seed points from the current point cloud data. Specifically, the method of this embodiment includes the following steps:

[0089] Obtain the depth image corresponding to the environment to be detected, and convert the depth image into the current point cloud data; determine the downsampling factor that matches the size of the depth image; select multiple seed points from the current point cloud data using the downsampling factor.

[0090] Combined with the foregoing embodiments for description, the point cloud data of the present application can be obtained by converting the depth image.

[0091] Exemplarily, obtain the depth image corresponding to the environment to be detected for detecting obstacles; then the depth image can be converted into a three-dimensional point cloud according to the internal parameters of the depth camera; subsequently, the three-dimensional point cloud can be converted into a specified coordinate system (such as a robot coordinate system or a world coordinate system, etc., which is not limited here) according to the external parameters of the camera; then the depth image and the point cloud data in the specified coordinate system can be downsampled according to the preset downsampling factor α, thereby reducing the subsequent calculation amount; then the seed points can be determined from the downsampled point cloud data.

[0092] Specifically, according to the pixel center (c x , c y ) in the camera internal parameters, the focal length (f x , f y ) and the scale factor scale, calculate the point cloud corresponding to each pixel point according to the following formula, and its mathematical expression is:

[0093]

[0094] Where u i , v i , d i are respectively the horizontal and vertical coordinates and the depth value of the i-th element of the depth image (usually traversed from left to right and from top to bottom), and x i , y i , z i are the corresponding point cloud coordinates.

[0095] According to the external parameter transformation matrix transform the point cloud p i =(x i , y i , z i ) T , i∈[0, n), and obtain the spatial point cloud in the world coordinate system / w =T·p i, where n is the total number of point clouds, and R and t are respectively the rotation matrix and translation vector from the preset current coordinate system to the world coordinate system.

[0096] According to the size of the depth image size=(row, col), select an appropriate downsampling factor α (which should be a common divisor of row and col of the depth image, such as 2, 4, etc.). Subsequently, downsample every α×α pixel block in the depth image and select the z with the smallest depth value among them min as the downsampled depth value z i to obtain a depth image with size=(row / α, col / α). It should be noted that, according to the different settings of the depth camera, pixels with a depth value of 0 (or max) are invalid pixels and need to be excluded and do not participate in the selection of the downsampled depth value.

[0097] Furthermore, calculate the sampling column interval d of the candidate points according to the size of the depth image and the downsampling factor c =col / (10α), and the row interval d r =1, that is, candidate points are sampled in each row, and the interval is 1 / 10 of the number of columns of the downsampled depth image, and its interval ratio can be adjusted according to the point cloud resolution. Subsequently, filter the sampled candidate points. For example, for the candidate point (u i , v i ), judge the validity of the pixel depth values in the 8 directions (such as 8-neighborhood, and can also be set to 4-neighborhood, etc., which will not be elaborated) represented by (u i , v i ±d c / 2), (u i ±d c / 2, v i ±d c / 2), (u i ±d c / 2, v i ). If there are valid depth values in the pixels in the 8 directions with a preset pixel quantity threshold (for example, 6) or more (according to the different cameras, the depth value of 0 or max can be set as an invalid depth value, and the rest are valid), then regard this candidate point as the selected seed point Seed i and calculate its normal vector normal i (for example, by calculating the outer product of the vectors from the center to each direction and taking the average value, the normal vector of this seed point can be obtained (taking the direction above the ground)).

[0098] Based on the above embodiments, the embodiments of the present application illustrate the steps of selecting multiple seed points from the current point cloud data using the downsampling factor. Specifically, the method of this embodiment includes the following steps:

[0099] Points are selected from the current point cloud data using a downsampling factor to obtain candidate points; the depth values of adjacent pixels of the candidate points in a preset orientation are obtained, and the number of adjacent pixels with valid depth values is counted; if the number of adjacent pixels is greater than or equal to a preset pixel number threshold, the candidate points are used as seed points.

[0100] For the specific method, reference can be made to the description of the foregoing embodiments, and details are not elaborated herein.

[0101] For example, for the candidate point (u i , v i ), the pixel depth values of the adjacent pixels in 8 preset orientations represented by (u i , v i ±d c / 2), (u i ±d c / 2, v i ±d c / 2), (u i ±d c / 2, v i ) are judged for validity. It should be noted that the adjacent pixels can be pixels with zero interval from the candidate points, and / or pixels with a preset interval from the candidate points (such as separated by one pixel or multiple pixels, etc.). There can be one or more preset intervals, which are not limited herein. It should also be noted that one or more pixels can be respectively selected as adjacent pixels in each preset orientation, and whether their depth values are valid is judged. For example, the pixel with zero interval directly above the candidate point can be used as an adjacent pixel, or the pixel separated by one pixel directly above the candidate point can be used as an adjacent pixel, or multiple pixels within the preset interval range directly above the candidate point can be used as adjacent pixels, etc., which are not limited herein.

[0102] Based on the above embodiments, the embodiments of the present application describe the steps of dividing multiple seed points into ground seed points and obstacle seed points according to the point cloud coordinates and normal vectors of the multiple seed points. Specifically, the method of this embodiment includes the following steps:

[0103] Obtain a ground plane model obtained by fitting a ground plane based on previous point cloud data, or a pre-calibrated ground plane model; wherein, the previous point cloud data refers to the point cloud data collected before the current point cloud data; calculate the distance between the seed point and the ground plane model based on the point cloud coordinates of the seed point to obtain a first distance; obtain a first distance threshold, and select the seed points with the first distance less than the first distance threshold to obtain candidate seed points; perform plane fitting on the candidate seed points to obtain an optimized ground plane; calculate the distance between the candidate seed points and the optimized ground plane based on the point cloud coordinates of the candidate seed points to obtain a second distance; and calculate the angle between the candidate seed points and the optimized ground plane based on the normal vector of the candidate seed points to obtain a normal vector angle; obtain a second distance threshold and a normal vector angle threshold, and select the candidate seed points with the second distance less than the second distance threshold and the normal vector angle less than the normal vector angle threshold to obtain ground seed points, and regard other seed points as obstacle seed points.

[0104] Combined with the foregoing embodiments for description, if the currently collected point cloud data is the first frame of point cloud data, the seed points can be screened through a pre-calibrated ground plane model to obtain candidate seed points. If the currently collected point cloud data is not the first frame of point cloud data, the seed points can be screened through the ground plane model obtained by fitting the ground plane with the previous point cloud data to obtain candidate seed points. Among them, the acquisition time sequence of the previous point cloud data is earlier than the acquisition time sequence of the current point cloud data.

[0105] Exemplarily, obtain the ground plane model of the previous frame or the calibrated ground plane model plane origin , and according to the preset first distance threshold d thres1 perform the first round of screening on the seed points, and calculate the distance between the seed points and the ground plane model plane origin less than the first distance threshold d thres1 of the seed points are marked as candidate seed points (candidate ground seed points).

[0106] Perform plane fitting (such as using the RANSAC algorithm) on the candidate seed points screened in the previous step to obtain an optimized ground plane model plane optimized , and similarly according to this ground plane model and the preset second distance threshold d thres2 and the normal vector angle threshold angle thres , screen the candidate seed points obtained in the previous step again: the distance between the candidate seed points and plane optimized is less than d thres2 , and the included angle between their normal vectors is less than angle thresThe candidate seed points are marked as ground seed points, and the remaining seed points among the candidate seed points are marked as obstacle seed points, thereby realizing the classification of ground seed points and obstacle seed points.

[0107] Based on the above embodiments, an embodiment of the present application provides a method for correcting misjudged point clouds. Specifically, the steps of this embodiment may include:

[0108] Traverse the depth image to find the clustering cluster Q i , the neighbor points of i∈[1, M] in each direction. This clustering cluster can include all ground clustering clusters and obstacle clustering clusters, and M is the sum of the number of ground clustering clusters and the number of obstacle clustering clusters. First, traverse the point cloud in the row direction of the depth image to find the pixel points with changing class labels and small pixel intervals as boundary points: for example, the current point cloud corresponds to the class Q i , if there are other classes such as S j in the neighborhood range where the current row pixel interval is less than 5 merge , and the distance between the two points is less than the distance threshold d j , then add S i to the neighbor list List i = [S j ...], where d obstacle is the preset minimum obstacle distance, and two obstacle point cloud clusters with a distance less than this value can be classified into the same obstacle point cloud. Subsequently, traverse the point cloud corresponding to the column direction of the depth image according to the above steps to find the neighbors of each point cloud cluster in the column direction and add them to the neighbor list List i .

[0109] In addition, discriminate the boundary points of different-class point cloud clusters that meet the above conditions. If the distance from the boundary point belonging to Q i to its neighborhood points is greater than the distance from this boundary point to the boundary point of S j , then the boundary point of the current Q i is a misclassified boundary point. Perform the same judgment and class correction as in the previous example on such misclassified boundary points and their neighborhood points within the dynamic step Step (for example, modify the class label of this misclassified boundary point to S j ).

[0110] Based on the above embodiments, an embodiment of the present application describes the steps of determining the pose information of obstacles in the environment to be detected based on ground clustering clusters and obstacle clustering clusters. Specifically, the steps of this embodiment may include:

[0111] Merging the ground clustering clusters and the obstacle clustering clusters can obtain the complete ground point cloud and different obstacle point cloud instances. According to the point cloud cluster numbers (clustering cluster numbers) to which the point cloud data belong, the corresponding obstacle point cloud is extracted, and its OBB (Oriented bounding box) or AABB (Axis-aligned bounding box) bounding box in the current coordinate system is calculated, then the obstacle pose information can be obtained.

[0112] Optionally, when merging the ground clustering clusters and the obstacle clustering clusters in this step, the method for determining the neighbor list in the foregoing embodiment can also be referred to by the same token, and then according to the neighbor list List i , i ∈ [1, M], the mergeable neighbors of each point cloud cluster are screened, and the specific steps can be as follows:

[0113] Since the original ground seed point plane obtains the ground point cloud cluster after region growing, at this time, the original seed points and their normal vectors are difficult to accurately represent the grown region. Therefore, for the seed points Seed i and their normal vectors normal i of the ground point cloud cluster, an update is performed. The centroid of all seed points in the current ground point cloud cluster can be used to replace the original seed points, and the mean value of their normal vectors is used to replace the original normal vectors.

[0114] Screen the neighbors of the ground point cloud cluster. According to the preset normal vector angle threshold angle merge and the plane distance threshold dis merge , if the following preset conditions are met, the point cloud cluster with fewer points is merged into the point cloud cluster with more points, where Seed0 and normal0 are the updated seed points and normal vectors of the current ground point cloud cluster, and Seed l , normal l are the seed points and normal vectors of its neighborhood. The mathematical expression of the conditions to be satisfied is:

[0115] arccos(normal0 · normal l ) < angle merge

[0116] max(|(Seed0 - Seed l ) · normal0|, |(Seed0 - Seed l ) · normal l ) < dis merge

[0117] When the local ground point cloud cluster is merged with its obstacle point cloud cluster, the obstacle point cloud cluster is merged into the ground point cloud cluster, and all neighbors of the obstacle point cloud cluster are cleared to prevent its neighbor points from sticking to other obstacle point cloud clusters and causing clustering errors. For two obstacle point cloud clusters that are neighbors of each other and have already undergone distance judgment, they can be directly merged.

[0118] Perform obstacle extraction on the merged point cloud cluster. It can be understood that before the point cloud clusters are merged, S i , i ∈ [1, n] are ground point cloud clusters, and S j , j ∈ (n, M] are obstacle point cloud clusters. According to the merging rules provided in the foregoing embodiments, it can be known that the merged point cloud clusters also conform to this rule. Then, the corresponding obstacle point cloud is extracted according to the serial number of the point cloud cluster to which the spatial point cloud belongs, and its OBB (Oriented bounding box) or AABB (Axis-aligned bounding box) bounding box in the current coordinate system can be calculated to obtain the obstacle pose information.

[0119] Furthermore, it should be noted that the execution subject of the obstacle detection method can be an obstacle detection device. For example, the obstacle detection method can be executed by a terminal device, a server, or other processing devices. Among them, the terminal device can be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the obstacle detection method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0120] In summary, the various embodiments of the present application can be appropriately combined to obtain at least one comprehensively implementable manner. For example Figure 2 as shown Figure 2 is the overall flowchart of an exemplary embodiment in the obstacle detection method of the present application. The method of the present application:

[0121] 1. Proposes a dual-channel region growth process, selects seed points at intervals and classifies them into ground seed points and obstacle seed points respectively, and adopts different strategies for region growth to obtain ground point cloud clusters and obstacle point cloud clusters, which can obtain the pose information of multiple obstacle point cloud instances and can also adapt to ground point clouds with certain undulations or distortions. Among them, the present application does not limit the sequence of dual-channel clustering. For example, obstacle point cloud clustering and ground point cloud clustering can be performed independently of each other; it can also be that ground point cloud clustering is performed first, and after ground plane marking, obstacle point cloud clustering is performed.

[0122] 2. Among them, the region growing mechanism based on dynamic step size will greatly improve the point cloud clustering speed, and the quadratic discrimination and boundary discrimination strategies can also be adopted to improve the clustering accuracy.

[0123] 3. Moreover, this application can downsample the input depth image and adaptively adjust the subsequent distance-related parameters, effectively improving the algorithm processing efficiency.

[0124] 4. Combining with the distribution characteristics of the sensor in different directions, a depth direction distance compensation strategy is proposed, which can adapt to the point cloud stratification phenomenon of the depth camera and optimize the growth effect of the point cloud region.

[0125] 5. During the selection and classification of seed points, the calibrated ground or the ground model of the previous frame is used for selection and classification, and the prior information is combined to improve the algorithm accuracy.

[0126] 6. Through the point cloud cluster merging strategy, after updating the seed points and normal vectors of the ground point cloud cluster, the normal vector angle and plane distance are calculated for merging the ground point cloud, and the obstacle point cloud is merged according to the boundary point distance, and finally the complete ground point cloud and different obstacle point cloud instances are obtained.

[0127] Figure 3 is a block diagram of an obstacle detection device shown in an exemplary embodiment of this application. As Figure 3 shown, the exemplary obstacle detection device 200 includes: a point cloud selection module 210, a point cloud division module 220, a point cloud clustering module 230, and an obstacle determination module 240. Specifically:

[0128] The point cloud selection module 210 is configured to obtain the current point cloud data corresponding to the environment to be detected, select multiple seed points from the current point cloud data, and select the points to be clustered from other points except the multiple seed points.

[0129] The point cloud division module 220 is configured to divide multiple seed points into ground seed points and obstacle seed points respectively based on the point cloud coordinates of the multiple seed points.

[0130] The point cloud clustering module 230 is configured to obtain the growth conditions corresponding to the ground seed points, divide the points to be clustered that meet the growth conditions corresponding to the ground seed points into the clustering clusters corresponding to the ground seed points to obtain the ground clustering clusters; and obtain the growth conditions corresponding to the obstacle seed points, and divide the points to be clustered that meet the growth conditions corresponding to the obstacle seed points into the clustering clusters corresponding to the obstacle seed points to obtain the obstacle clustering clusters.

[0131] The obstacle determination module 240 is configured to determine the obstacle pose information in the environment to be detected based on the ground clustering clusters and the obstacle clustering clusters

[0132] In this exemplary obstacle detection device, by acquiring the current point cloud data corresponding to the environment to be detected, multiple seed points can be selected from the current point cloud data, and points to be clustered can be selected from other points except the multiple seed points; based on the point cloud coordinates of the multiple seed points, the multiple seed points are respectively classified into ground seed points and obstacle seed points to realize the classification of the seed points for constructing different region growing channels; acquire the growth conditions corresponding to the ground seed points, and classify the points to be clustered that meet the growth conditions corresponding to the ground seed points into the cluster corresponding to the ground seed points to obtain the ground cluster; and, acquire the growth conditions corresponding to the obstacle seed points, and classify the points to be clustered that meet the growth conditions corresponding to the obstacle seed points into the cluster corresponding to the obstacle seed points to obtain the obstacle cluster. Thus, different growth strategies can be adopted for region growing to obtain the ground point cloud cluster and the obstacle point cloud cluster; then, based on the ground cluster and the obstacle cluster, the ground point cloud and the obstacle point cloud can be accurately distinguished, and the pose information of the obstacles in the environment to be detected can be determined, thereby improving the detection accuracy of the obstacles.

[0133] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept. The specific ways in which each module and unit perform operations have been described in detail in the method embodiment and will not be repeated here. In practical applications, the device provided in the above embodiment can, according to needs, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not limited here.

[0134] Among them, the functions of each module can be referred to in the embodiment of the obstacle detection method and will not be repeated here.

[0135] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of an embodiment of an electronic device of the present application. The electronic device 100 includes a memory 101 and a processor 102. The processor 102 is used to execute the program instructions stored in the memory 101 to implement the steps in any of the above embodiments of the obstacle detection method. In a specific implementation scenario, the electronic device 100 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 100 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited here.

[0136] Specifically, the processor 102 is used to control itself and the memory 101 to implement the steps in any of the above-described obstacle detection method embodiments. The processor 102 may also be referred to as a CPU (Central Processing Unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 102 may be implemented jointly by integrated circuit chips.

[0137] In this exemplary electronic device, by obtaining the current point cloud data corresponding to the environment to be detected, multiple seed points can be selected from the current point cloud data, and points to be clustered can be selected from other points except the multiple seed points; based on the point cloud coordinates of the multiple seed points, the multiple seed points are respectively classified into ground seed points and obstacle seed points to achieve the classification of the seed points for constructing different region growth channels; the growth conditions corresponding to the ground seed points are obtained, and the points to be clustered that meet the growth conditions corresponding to the ground seed points are divided into the cluster corresponding to the ground seed points to obtain the ground cluster; and, the growth conditions corresponding to the obstacle seed points are obtained, and the points to be clustered that meet the growth conditions corresponding to the obstacle seed points are divided into the cluster corresponding to the obstacle seed points to obtain the obstacle cluster. Thus, different growth strategies can be adopted for region growth to obtain the ground point cloud cluster and the obstacle point cloud cluster; then, based on the ground cluster and the obstacle cluster, the ground point cloud and the obstacle point cloud can be accurately distinguished, and the pose information of the obstacles in the environment to be detected can be determined, thereby improving the detection accuracy of the obstacles.

[0138] Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 110 stores program instructions 111 that can be run by a processor, and the program instructions 111 are used to implement the steps in any of the above-described obstacle detection method embodiments.

[0139] In the exemplary storage medium, by running the program instructions in the storage medium, the current point cloud data corresponding to the environment to be detected can be obtained. Multiple seed points can be selected from the current point cloud data, and points to be clustered can be selected from other points except the multiple seed points. Based on the point cloud coordinates of the multiple seed points, the multiple seed points are respectively classified into ground seed points and obstacle seed points to classify the seed points so as to construct different region growing channels. The growth conditions corresponding to the ground seed points are obtained, and the points to be clustered that meet the growth conditions corresponding to the ground seed points are divided into the cluster corresponding to the ground seed points to obtain the ground cluster. And, the growth conditions corresponding to the obstacle seed points are obtained, and the points to be clustered that meet the growth conditions corresponding to the obstacle seed points are divided into the cluster corresponding to the obstacle seed points to obtain the obstacle cluster. Thus, different growth strategies can be adopted for region growing to obtain the ground point cloud cluster and the obstacle point cloud cluster. Then, based on the ground cluster and the obstacle cluster, the ground point cloud and the obstacle point cloud can be accurately distinguished, and the pose information of the obstacles in the environment to be detected can be determined, thereby improving the detection accuracy of the obstacles.

[0140] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0141] The descriptions of the above embodiments tend to emphasize the differences between the embodiments. The same or similar parts can be referred to each other. For the sake of brevity, they will not be repeated in this article.

[0142] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0143] In addition, each functional unit in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

Claims

1. An obstacle detection method, characterized in that, The method includes: Obtaining current point cloud data corresponding to the environment to be detected, selecting a plurality of seed points from the current point cloud data, and selecting points to be clustered from other points except the plurality of seed points; Based on the point cloud coordinates of the plurality of seed points, dividing the plurality of seed points into ground seed points and obstacle seed points respectively; Obtaining the growth conditions corresponding to the ground seed points, dividing the points to be clustered that meet the growth conditions corresponding to the ground seed points into the clustering clusters corresponding to the ground seed points to obtain ground clustering clusters; and obtaining the growth conditions corresponding to the obstacle seed points, dividing the points to be clustered that meet the growth conditions corresponding to the obstacle seed points into the clustering clusters corresponding to the obstacle seed points to obtain obstacle clustering clusters; Based on the ground clustering clusters and the obstacle clustering clusters, determining the pose information of obstacles in the environment to be detected.

2. The method according to claim 1, wherein The selecting the points to be clustered from other points except the plurality of seed points includes: Obtaining a region growing step length and determining a current growth origin corresponding to the current seed point; wherein the current growth origin includes the current seed point or a growth point corresponding to the current seed point, and the growth point refers to a point belonging to the same clustering cluster as the current seed point; Taking the current growth origin as a starting point, selecting the point cloud data in the current point cloud data under the region growing step length to obtain the points to be clustered; The method further includes: If the points to be clustered do not meet the growth conditions corresponding to the current seed point, reducing the region growing step length and re-selecting the points to be clustered with the current growth origin as the starting point; If the points to be clustered meet the growth conditions corresponding to the current seed point, dividing the point cloud data between the current growth origin and the points to be clustered into the clustering cluster corresponding to the current seed point.

3. The method according to claim 2, wherein The number of the growth conditions corresponding to the current seed point is multiple; the if the points to be clustered do not meet the growth conditions corresponding to the current seed point, reducing the region growing step length and re-selecting the points to be clustered with the current growth origin as the starting point includes: If the points to be clustered do not meet all the growth conditions corresponding to the current seed point, reducing the region growing step length and re-selecting the points to be clustered with the current growth origin as the starting point; If the points to be clustered meet some of the growth conditions corresponding to the current seed point, and the number of the satisfied growth conditions is greater than or equal to a preset condition number threshold, taking the adjacent points corresponding to the points to be clustered as auxiliary discriminant points, and if the auxiliary discriminant points meet all the growth conditions corresponding to the current seed point, dividing the point cloud data between the current growth origin and the auxiliary discriminant points into the clustering cluster corresponding to the current seed point.

4. The method according to claim 1, characterized in that, The obtaining the growth conditions corresponding to the ground seed points, dividing the points to be clustered that meet the growth conditions corresponding to the ground seed points into the clustering clusters corresponding to the ground seed points to obtain ground clustering clusters includes: Determine the plane represented by the normal vector corresponding to the ground seed point to obtain the ground plane, calculate the distance between the point to be clustered and the ground plane to obtain the ground distance; and calculate the distance between the point to be clustered and the current growth origin corresponding to the ground seed point to obtain the ground point distance; Obtain the ground distance threshold and the ground point distance threshold corresponding to the ground seed point; Classify the points to be clustered with a ground distance less than the ground distance threshold and / or a ground point distance less than the ground point distance threshold into the cluster corresponding to the ground seed point to obtain the ground cluster; And / or, obtaining the growth conditions corresponding to the obstacle seed point, and classifying the points to be clustered that meet the growth conditions corresponding to the obstacle seed point into the cluster corresponding to the obstacle seed point to obtain the obstacle cluster, including: Calculate the distance between the point to be clustered and the current growth origin corresponding to the obstacle seed point to obtain the obstacle point distance; Obtain the obstacle point distance threshold corresponding to the obstacle seed point; Classify the points to be clustered with an obstacle distance less than the obstacle point distance threshold into the cluster corresponding to the obstacle seed point to obtain the obstacle cluster.

5. The method according to claim 4, characterized in that The current point cloud data is obtained by sampling a depth image, and each point in the current point cloud data corresponds to a depth value; the calculating the distance between the point to be clustered and the current growth origin corresponding to the ground seed point to obtain the ground point distance includes: Based on the point cloud coordinates of the point to be clustered and the point cloud coordinates of the current growth origin corresponding to the ground seed point, calculate the distance between the point to be clustered and the current growth origin corresponding to the ground seed point to obtain the first initial distance; Obtain the depth value of the point to be clustered; Compensate the value of the first initial distance based on the depth value of the point to be clustered to obtain the ground point distance; And / or, the calculating the distance between the point to be clustered and the current growth origin corresponding to the obstacle seed point to obtain the obstacle point distance includes: Based on the point cloud coordinates of the point to be clustered and the point cloud coordinates of the current growth origin corresponding to the obstacle seed point, calculate the distance between the point to be clustered and the current growth origin corresponding to the obstacle seed point to obtain the second initial distance; Compensate the value of the second initial distance based on the depth value of the point to be clustered to obtain the obstacle point distance.

6. The method according to claim 1, characterized in that, The obtaining the current point cloud data corresponding to the environment to be detected and selecting multiple seed points from the current point cloud data includes: Obtain the depth image corresponding to the environment to be detected and convert the depth image into the current point cloud data; Determine the downsampling multiple matching the size of the depth image; Select multiple seed points from the current point cloud data using the downsampling multiple.

7. The method according to claim 6, wherein The selecting multiple seed points from the current point cloud data using the downsampling multiple includes: Perform point selection from the current point cloud data using the downsampling multiple to obtain candidate points; Obtain the depth values of the adjacent pixels of the candidate points in the preset orientation, and count the number of adjacent pixels with valid depth values; If the number of the adjacent pixels is greater than or equal to a preset pixel number threshold, use the candidate point as a seed point.

8. The method according to claim 1, characterized in that Dividing the multiple seed points into ground seed points and obstacle seed points respectively based on the point cloud coordinates of the multiple seed points includes: Obtain a ground plane model obtained by fitting a ground plane based on previous point cloud data, or a pre-calibrated ground plane model; wherein, the previous point cloud data refers to the point cloud data collected before the current point cloud data; Calculate the distance between the seed point and the ground plane model based on the point cloud coordinates of the seed point to obtain a first distance; Obtain a first distance threshold, and select the seed points with the first distance less than the first distance threshold to obtain candidate seed points; Perform plane fitting on the candidate seed points to obtain an optimized ground plane; Calculate the distance between the candidate seed point and the optimized ground plane based on the point cloud coordinates of the candidate seed point to obtain a second distance; and calculate the angle between the candidate seed point and the optimized ground plane based on the normal vector of the candidate seed point to obtain a normal vector angle; Obtain a second distance threshold and a normal vector angle threshold, select the candidate seed points with the second distance less than the second distance threshold and the normal vector angle less than the normal vector angle threshold to obtain the ground seed points, and use the other seed points as the obstacle seed points.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, and the processor is configured to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor to implement the steps in the method according to any one of claims 1-8.

Images