A Clustering Method Combining Point Cloud Semantic Categories and Distances

Abstract

The present invention relates to the field of autonomous vehicle perception technology, and provides a clustering method that fuses the semantic categories and distances of point clouds. The method includes: segmenting lidar point cloud data based on a semantic segmentation algorithm, and outputting each point with a semantic category label; segmenting the point cloud after semantic segmentation into ground point cloud and non-ground point cloud; based on the non-ground point cloud, calculating the number of grids n corresponding to the seed points, and realizing non-ground point segmentation parameter clustering according to 8-neighborhood search, and sequentially performing 8-neighborhood segmentation parameter clustering on the points that have not been clustered within the grid; comparing the number of candidate category points with the point number threshold corresponding to this category, judging whether there is clustering within the n grids in the 8-neighborhood and making corresponding processing, and making corresponding processing for clustering categories with the same or different categories. The present invention improves the segmentation speed while reducing under-segmentation and over-segmentation, filters out mis-segmented semantic points, makes the clustering categories clearer, and solves the selection of clustering merging parameters for obstacles of different sizes.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous vehicle perception, and particularly to a clustering method that fuses point cloud semantic categories and distances. Background Art

[0002] With the development and application of autonomous driving technology, related technologies such as perception have also made great progress. Point clouds can provide accurate 3D data of the environment. Compared with cameras and millimeter-wave radars, they have better detection accuracy. And as active sensors, they are not affected by light changes and have stronger adaptability. They are generally used as the main sensors in the field of autonomous driving. However, point cloud data has the characteristics of sparsity and disorder. Especially for low-beam lidar or long distances, the data spacing of the point cloud is relatively large. After preprocessing and ground detection, the point cloud obstacle perception algorithm uses clustering algorithms such as Euclidean clustering or DBSCAN to segment the obstacle point cloud into different obstacle point clouds. Due to the sparsity and disorder of the point cloud, when using a clustering algorithm with fixed clustering parameters, there are over-segmentation or under-segmentation problems in the clustered obstacle point cloud.

[0003] In the patent "A Method and Device for Clustering Point Cloud Data of a Lidar and Its Process", the literature publication number is CN110738223A. The point cloud is rasterized, and all the rasters are scanned in turn with a window of a set number of rasters. The rasters with points in the window are segmented into the same category to achieve fast clustering of the point cloud. This method has a simple process, is easy to implement, and has high real-time performance. However, since the point cloud distribution gradually becomes sparse from near to far, scanning with a window of a fixed number of rasters results in the clustering effect being unable to take into account both near and far distances, and the under-segmentation or over-segmentation problems are relatively obvious.

[0004] In the patent "An Adaptive Point Cloud Target Clustering Method Based on an Elliptic Domain", the literature publication number is CN113269889A. Considering the non-uniform distribution of the point cloud, a method for adaptive point cloud target clustering based on DBSCAN clustering and an elliptic neighborhood is proposed, which effectively solves the problems of under-segmentation and over-segmentation. However, this method does not consider the target category information. For a relatively large target, if there is partial occlusion, it may cause the target to be over-segmented into two categories.

[0005] In the patent "A Point Cloud Segmentation Method and System Based on Clustering", the literature publication number is CN105957076B. The normal vector, plane curvature, and compatibility set are calculated for each point, and then point cloud clustering is achieved through patch processing. This method considers the normal vector characteristics of the target and can effectively solve the segmentation problem of adjacent targets that are relatively close. However, since the normal vector of each point needs to be calculated, when the number of point clouds is large, it takes a long time and is difficult to meet the real-time requirement. Summary of the Invention

[0006] In view of this, the present invention provides a clustering method that fuses point cloud semantic categories and distances to solve the problems in the prior art, such as a large number of point clouds, long time consumption, difficulty in meeting real-time requirements, and under-segmentation and over-segmentation.

[0007] The present invention provides a clustering method that fuses point cloud semantic categories and distances, including:

[0008] 1. A clustering method that fuses point cloud semantic categories and distances, characterized by including:

[0009] S1 Segment the lidar point cloud data based on a semantic segmentation algorithm, and output each point with a semantic category label;

[0010] S2 Segment the point cloud after semantic segmentation into ground point clouds and non-ground point clouds;

[0011] S3 Based on the non-ground point cloud, project it into a grid map. According to the distance r of the seed point from the coordinate origin and the grid resolution GMres, calculate the number of grids n corresponding to the seed point. Implement non-ground point segmentation parameter clustering according to 8-neighborhood search, and sequentially perform 8-neighborhood segmentation parameter clustering on the non-ground points that have not been clustered within the grid;

[0012] S4 Count the number of points of each semantic category in each cluster. The category with the largest number of semantic category points is the candidate category. Compare the number of candidate category points with the corresponding point number threshold of this category. If the number of candidate category points exceeds the corresponding point number threshold of this category, then the candidate category is the clustering category where it is currently located; otherwise, the candidate category is an uncertain category;

[0013] S5 Based on each clustering category obtained in S4, set corresponding clustering distance merging parameters according to each clustering category, and calculate the number of grids m corresponding to the distance merging parameters. Traverse each cluster and determine whether there are clustering categories with the same category or clustering of the uncertain category within the range of m grids in the 8-neighborhood. If so, merge the two categories, and the merged category is the corresponding determined category; otherwise, do not merge. Further, the corresponding semantic segmentation algorithm in S1 includes: RangeNet++; the semantic category labels include: truck, car, pedestrian, ground.

[0014] The semantic category labels include: truck, car, pedestrian, ground.

[0015] Further, S3 includes:

[0016] S31 Project the non-ground point cloud onto the grid map GM. Store the point cloud sequence number id within the grid range and the grid belonging clustering sequence number N in each grid. Let the initial clustering sequence number N of all grids be 0, and order the unordered point cloud;

[0017] S32 traverses the grid map GM from left to right and top to bottom. Taking the grid with the initial clustering serial number of the non-ground point cloud being the initial value 0 as the seed point Pseed, mark the grid where the seed point Pseed is located as the Nth class;

[0018] Among them, set the clustering distance parameter Dthr as the distance r of the seed point Pseed from the origin. According to the clustering distance parameter Dthr and the grid map GM resolution GMres, calculate the number n of grids corresponding to the seed point. Taking the seed point Pseed as the center point, search for the adjacent n grids in the 8-neighborhood directions respectively. If there is non-ground point cloud in the adjacent grid and the initial clustering serial number of the grid is the initial value 0, then mark the grid as having the same clustering serial number as the seed point Pseed, otherwise the initial clustering serial number of the grid remains unchanged;

[0019] S33 Take the non-ground points in the 8-neighborhood whose clustering serial number changes from 0 to N as new seed points Pseed in turn, and execute S32 in turn until there is no grid with the initial clustering serial number being the initial value 0 in the 8-neighborhood of all such seed points Pseed;

[0020] S34 Let N = N + 1, and execute S32 and S33 in turn until the clustering serial numbers of all grids containing non-ground points are not 0, and complete the segmented clustering of the grid map GM. Further, in S31, project the non-ground point cloud into the grid map GM, and the grid range and grid resolution are set according to requirements, where the requirements include the horizontal and vertical distances of the grid and the resolution set according to the distance.

[0021] Further, S4 also includes: performing filtering processing on the same semantic category with inconsistencies.

[0022] The beneficial effects of the present invention compared with the prior art are as follows:

[0023] 1. By projecting the non-ground point cloud into the grid map, the present invention orders the disordered point cloud, sets the clustering parameter according to the distance of the point from the origin, improves the segmentation speed and reduces under-segmentation and over-segmentation at the same time;

[0024] 2. By statistically analyzing the clustering results of the grid map according to the category, the present invention filters out the wrong semantic segmentation points to obtain the category of each clustering, and improves the stability of the clustering category;

[0025] 3. According to the clustering category information, set the clustering merging distance, and select the corresponding merging parameter according to the different sizes of the obstacles, further reducing the situations of under-segmentation and over-segmentation. Description of the Drawings

[0026] To more clearly illustrate the technical solutions in the present invention, the following will briefly introduce the drawings required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0027] Figure 1 is a flowchart of a clustering method that fuses the semantic category and distance of point clouds according to the present invention;

[0028] Figure 2 is a flowchart of segmented clustering of a grid map provided by the present invention;

[0029] Figure 3 is a semantic segmentation category filtering process provided by the present invention;

[0030] Figure 4 is a flowchart of clustering merging provided by the present invention. Detailed implementation manners

[0031] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are put forward to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0032] The following will detail a clustering method that fuses the semantic category and distance of point clouds according to the present invention with reference to the drawings.

[0033] Figure 1 is a flowchart of a clustering method that fuses the semantic category and distance of point clouds according to the present invention.

[0034] As Figure 1 shown, the clustering method includes:

[0035] S1. Segment the lidar point cloud data based on a semantic segmentation algorithm, and output each point with a semantic category label;

[0036] The semantic segmentation algorithm in S1 includes: RangeNet++; the semantic category labels include: truck, car, pedestrian, ground.

[0037] S2. Segment the point cloud after semantic segmentation into a ground point cloud and a non-ground point cloud;

[0038] S3. Project the non-ground point cloud onto the grid map. According to the distance r of the seed point from the coordinate origin and the grid resolution GMres, calculate the number of grids n corresponding to the seed point. Implement segment parameter clustering of non-ground points based on 8-neighborhood search, and sequentially perform 8-neighborhood segment parameter clustering on the non-ground points that have not been clustered within the grid.

[0039] S31. Project the non-ground point cloud onto the grid map GM. Store the point cloud serial number id within the grid range and the cluster serial number N to which the grid belongs in each grid. Let the initial cluster serial number N of all grids be 0, and order the unordered point cloud.

[0040] In S31, project the non-ground point cloud onto the grid map GM. The grid range and grid resolution are set according to requirements, where the requirements include the horizontal and vertical distances of the grid and the resolution set according to the distance.

[0041] S32. Traverse the grid map GM from left to right and from top to bottom. Use the grid with the initial cluster serial number of the non-ground point cloud being the initial value 0 as the seed point Pseed, and mark the grid where the seed point Pseed is located as the Nth class.

[0042] Among them, set the distance r of the seed point Pseed from the origin as the clustering distance parameter Dthr. According to the clustering distance parameter Dthr and the grid map GM resolution GMres, calculate the number of grids n corresponding to the seed point. Use the seed point Pseed as the center point and search for the adjacent n grids in 8-neighborhood directions. If there is non-ground point cloud in the adjacent grid and the initial cluster serial number of the grid is the initial value 0, then mark the grid as having the same cluster serial number N as the seed point Pseed; otherwise, keep the initial cluster serial number of the grid unchanged.

[0043] S33. Sequentially use the non-ground points in the 8-neighborhood whose cluster serial number changes from 0 to N as new seed points Pseed, and sequentially execute S32 until there is no grid with the initial cluster serial number being the initial value 0 within the 8-neighborhood of all such seed points Pseed.

[0044] S34. Let N = N + 1, and sequentially execute S32 and S33 until the cluster serial numbers of all grids containing non-ground points are not 0, completing the segment clustering of the grid map GM.

[0045] Project the non-ground point cloud data onto the grid map. The grid range and grid resolution are set according to requirements (for example, the grid range is set to -30 to 30 m horizontally and -15 to 100 m vertically, and the resolution is set to 0.2 m). The initial cluster serial number in each grid is 0, that is, the unclustered state.

[0046] Traverse each of the above grids to find the grids in the unclustered state and having non-ground point cloud, and use this grid as the seed point Pseed (such as Figure 2Grid A), set the grid category to 1 (increasing sequentially), set the clustering distance threshold according to the grid position where Pseed is located, and convert the clustering distance threshold into the number of grids. For example, Figure 2 The corresponding number of grids for the clustering distance parameter shown), if the grid distance from the valid non-ground point to the seed grid is less than the set threshold, then set the grid category to the same category as the seed point grid. For example, Figure 2 In the figure, the distance from the seed point A to the grid B is 1 grid, and there is a valid non-ground point in the grid B. The clustering threshold for the grid A and the grid B is 1 grid, then add the grid B to the category 1 of the seed point A;

[0047] Take the newly added grid as the search starting point in turn, and according to the number of grids corresponding to the clustering threshold at the corresponding position, search for adjacent grids that meet the conditions of having non-ground point clouds and not being clustered (the grid category is 0). For example, taking B as the search starting point, add C to the clustering result, and set the grid category of C to 1. Similarly, add the grids DEFGHI to the clustering with the clustering category of 1.

[0048] In the embodiment of the present invention, the non-ground point cloud is projected onto the grid map to order the disordered point cloud, and the clustering parameter is set according to the distance of the point from the origin, which improves the segmentation speed and reduces under-segmentation and over-segmentation at the same time.

[0049] S4: Count the number of points of each semantic category in each clustering. The category with the largest number of semantic category points is the candidate category. Compare the number of candidate category points with the corresponding point number threshold of this category. If the number of candidate category points exceeds the corresponding point number threshold of this category, then the candidate category is the clustering category where it is currently located; otherwise, the candidate category is the uncertain category;

[0050] S4 also includes: performing filtering processing on the inconsistent same semantic category.

[0051] In the embodiment of the present invention, the grid map clustering result is filtered according to the category statistical value to remove the wrong semantic segmentation points and obtain the category of each clustering.

[0052] Figure 3 It is the semantic segmentation category filtering process provided by the embodiment of the present invention.

[0053] Statistics are carried out according to Figure 2 the number of points of different categories in the result of clustering according to the process, Figure 3 which represents the bar chart of the number of points of different categories. It represents the number of truck type points, the number of car type points, the number of pedestrian type points, the number of ground type points, and the number of other type points in turn. As shown in the figure, the number of points in the truck category is 200, the number of points in the car category is 20, the number of points in the pedestrian category is 5, the number of points in the ground category is 10, and the number of points in the other category is 20;

[0054] The truck category has the most points, and the truck category points of 200 are greater than the truck category minimum point threshold. If the threshold is 100, then the clustering category is a truck;

[0055] Set all the point categories in the clustering result to the truck category.

[0056] In the embodiment of the present invention, the grid map clustering result is filtered according to the category statistical value to remove the wrong semantic segmentation points, and the category of each clustering is obtained.

[0057] S5 Based on each clustering category obtained in S4, set the corresponding clustering distance merging parameter according to each clustering category, and calculate the number of grid cells m corresponding to the distance merging parameter. Traverse each clustering, and judge whether there are clustering categories with the same category or clustering categories with uncertain categories within the range of m grid cells in the 8-neighborhood. If so, merge the two categories, and the merged category is the corresponding determined category; otherwise, do not merge.

[0058] Among them, the clustering categories that are not merged are the uncertain categories obtained in S4.

[0059] According to the clustering category information, set the clustering merging distance to solve the selection of the clustering merging parameter for obstacles with different sizes, and further reduce under-segmentation and over-segmentation.

[0060] Figure 4 It is the clustering merging flow chart provided by the present invention.

[0061] After the previous several processes, the point cloud is clustered into two categories according to the grid segmentation clustering and category filtering method. One category is category 1, and the clustering type is a truck, shown as black dots in the figure. The other category is category 2, and the clustering type is a car, shown as black rectangles in the figure;

[0062] The merging clustering threshold parameters for different category type objects are different. For example, since the truck has a larger volume, the merging clustering distance parameter is larger. For example, the merging clustering distance parameter for the truck is 4 grid cell distances, then merge category 2 into category 1;

[0063] According to Figure 3 In the category filtering process, since the number of points in the truck category is much more than that in the car category, and the number of points in the truck category exceeds the truck category minimum point threshold, then set the merged categories of category 1 and category 2 to the truck category.

[0064] According to the clustering category information, set the clustering merging distance to solve the selection of the clustering merging parameter for obstacles with different sizes, and further reduce under-segmentation and over-segmentation.

[0065] The method of the present invention determines whether adjacent points belong to the same clustering category, with a small amount of calculation, and can meet the real-time requirements of perception for autonomous driving. The disordered point cloud is ordered, and clustering parameters are set according to the distance of the points from the origin, improving the segmentation speed while reducing under-segmentation and over-segmentation; by setting the clustering merging distance, the selection of clustering merging parameters for obstacles of different sizes is solved, further reducing under-segmentation and over-segmentation.

[0066] Any combination of the above optional technical solutions can form an optional embodiment of the present application, which will not be elaborated herein one by one.

[0067] The following is an embodiment of the device of the present invention, which can be used to execute the embodiment of the method of the present invention. For the details not disclosed in the embodiment of the device of the present invention, please refer to the embodiment of the method of the present invention.

[0068] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A clustering method that fuses the semantic categories and distances of point clouds, characterized in that, it includes: S1 Segment the lidar point cloud data based on a semantic segmentation algorithm, and output each point with a semantic category label; S2 Segment the point cloud after semantic segmentation into ground point clouds and non-ground point clouds; S3 Based on the non-ground point cloud, project it into a grid map. According to the distance r of the seed point from the coordinate origin and the grid resolution GMres, calculate the number of grids n corresponding to the seed point. Implement non-ground point segment parameter clustering according to 8-neighborhood search, and sequentially perform 8-neighborhood segment parameter clustering on the non-ground points that have not been clustered within the grid; S4 Count the number of points of each semantic category in each cluster. The category with the largest number of semantic category points is the candidate category. Compare the number of candidate category points with the corresponding point number threshold of this category. If the number of candidate category points exceeds the corresponding point number threshold of this category, then the candidate category is the current cluster category; otherwise, the candidate category is an uncertain category; S5 Based on each cluster category obtained in S4, set corresponding cluster distance merging parameters according to each cluster category, and calculate the number of grids m corresponding to the distance merging parameters. Traverse each cluster and determine whether there are cluster categories with the same category or clusters of the uncertain category within the range of m grids in the 8-neighborhood. If so, merge the two categories, and the merged category is the corresponding determined category; otherwise, do not merge; The S3 includes: S31 Project the non-ground point cloud onto the grid map GM. Each grid stores the point cloud serial number id within the grid range and the grid belonging cluster serial number N. Let the initial cluster serial number N of all grids be 0, and order the unordered point cloud; S32 Traverse the grid map GM from left to right and from top to bottom. Take the grid with the initial cluster serial number of the non-ground point cloud as the initial value 0 as the seed point Pseed, and mark the grid where the seed point Pseed is located as the Nth category; Among them, set the clustering distance parameter Dthr as the distance r of the seed point Pseed from the origin. According to the clustering distance parameter Dthr and the grid map GM resolution GMres, calculate the number of grids n corresponding to the seed point. Take the seed point Pseed as the center point and search for the adjacent n grids in the 8-neighborhood directions respectively. If there is non-ground point cloud in the adjacent grid and the initial cluster serial number of the grid is the initial value 0, then mark the grid as the same cluster serial number as the seed point Pseed; otherwise, the initial cluster serial number of the grid remains unchanged; S33 Take the non-ground points in the 8-neighborhood whose cluster serial numbers change from 0 to N as new seed points Pseed in turn, and execute S32 in turn until there is no grid with the initial cluster serial number of the initial value 0 in the 8-neighborhood of all such seed points Pseed; S34 Let N = N + 1, and execute S32 and S33 in turn until the cluster serial numbers of all grids containing non-ground points are not 0, and complete the segmented clustering of the grid map GM.

2. The clustering method according to claim 1, It is characterized in that The corresponding semantic segmentation algorithm in S1 includes: RangeNet++; the semantic class labels include: truck, car, pedestrian, ground.

3. The clustering method according to claim 1, It is characterized in that In S31, the non-ground point cloud is projected onto the grid map GM, and the grid range and grid resolution are set according to requirements, where the requirements include the horizontal and vertical distances of the grid and the resolution set according to the distance.

4. The clustering method according to claim 1, It is characterized in that S4 further includes: performing filtering processing on the same semantic class that is inconsistent.

Images