LiDAR obstacle detection method based on Euclidean clustering and deep learning

Abstract

The present invention discloses a laser radar obstacle detection method based on Euclidean clustering and deep learning. The method uses a deep learning method to process laser radar point cloud data to obtain deep learning detection results. Then, Euclidean clustering is added to perform deep learning false detection processing to eliminate deep learning false detection situations and obtain a list of deep learning detection results that exclude false detections. The list of deep learning detection results that exclude false detections is traversed and merged clustering and over-segmentation is performed to obtain three groups of detection results: a deep learning detection list containing clustered point clouds; a deep learning detection list without clustered point clouds; and clusters without deep learning semantic attributes. The deep learning detection list without clustered point clouds and the clusters without deep learning semantic attributes are used as input to perform split clustering and incomplete segmentation to obtain the final detection results. The present invention fuses the detection results of Euclidean clustering and deep learning to improve the perception accuracy of the laser radar.

Description

Technical Field

[0001] The present invention relates to vehicle obstacle detection technology, and in particular to a lidar obstacle detection method based on Euclidean clustering and deep learning. Background Art

[0002] Common LiDAR obstacle detection methods are mainly divided into traditional Euclidean clustering and advanced deep learning detection, both of which have their advantages and disadvantages. The detection method based on Euclidean clustering forms clusters of non-ground point clouds in the environment and considers the clusters as the targets to be detected. However, it has a high dependence on clustering parameters and is prone to clustering multiple adjacent small targets into one large target (under-segmentation) or identifying a large target as multiple small targets (over-segmentation). In addition, it lacks semantic information about obstacles. The detection method based on deep learning has high requirements for environmental sample data and is prone to missed detections, such as the inability to detect children, ice cream cones, and some non-motor vehicles in the environment. In addition, it has false detections, such as identifying flower beds and green plants as moving vehicles, which can easily lead to vehicle braking errors.

[0003] Therefore, the present invention proposes a lidar obstacle detection method based on Euclidean clustering and deep learning, which fuses the detection results of Euclidean clustering and deep learning to improve the perception accuracy of lidar. Summary of the Invention

[0004] In order to address the deficiencies in the prior art, the present invention aims to provide a lidar obstacle detection method based on Euclidean clustering and deep learning.

[0005] To achieve the purpose of the present invention, the technical solution adopted by the present invention is:

[0006] A lidar obstacle detection method based on Euclidean clustering and deep learning, comprising the following steps:

[0007] (1) Using deep learning methods to process the point cloud data obtained by lidar obstacle detection, the deep learning detection results are obtained; then Euclidean clustering is added to perform deep learning false detection processing, eliminate the false detection of deep learning, and obtain a list of deep learning detection results that exclude false detection;

[0008] (2) Traverse the list of deep learning detection results that exclude false detections, perform merging and clustering over-segmentation, and obtain three groups of detection results: a deep learning detection list containing clustered point clouds; a deep learning detection list without clustered point clouds; and a cluster without deep learning semantic attributes;

[0009] (3) The above deep learning detection list without clustered point cloud and the cluster without deep learning semantic attributes are used as input to perform split clustering incomplete segmentation to obtain the final detection result.

[0010] Furthermore, step (1) specifically includes:

[0011] (1.1) Use deep learning methods to detect LiDAR point cloud data, obtain deep learning detection results, and form an obstacle list; each obstacle is surrounded by a minimum rectangular box;

[0012] (1.2) The Euclidean clustering method is used to detect the lidar point cloud data and obtain clusters; the deep learning detection results are associated with the corresponding clusters. If the intersection ratio of the associated Euclidean cluster and the deep learning detection area is less than the threshold, it is considered that the current situation is a deep learning error detection and it is removed from the detection list.

[0013] Furthermore, in step (1.2), the distance between the center position of the deep learning minimum rectangular box and the center position of the cluster is calculated. If the distance is less than the set threshold, it is considered to be associated.

[0014] Furthermore, step (2) specifically includes:

[0015] (2.1) dilating the area of ​​obstacles detected by deep learning to eliminate false detections;

[0016] (2.2) Based on the results of deep learning detection that excludes false detections, the obstacles detected by the clustering module are traversed separately to perform target association. The distance between the clustering results and the deep learning detection results is determined. If the distance is less than a certain threshold, the two are considered to be correlated.

[0017] (2.3) Determine one by one whether multiple clusters belong to the same deep learning detection target. If the ratio of the intersection of the Euclidean clustering and the deep learning detection area to the cluster is greater than the threshold, the cluster is considered to belong to the deep learning detection result, and the point cloud of the cluster is spliced ​​into the deep learning detection result; if it is less than the threshold, it is detected as a cluster without deep learning semantic attributes;

[0018] (2.4) Repeat the above steps to traverse all deep learning and clustering detection results, and obtain three groups of detection results: a deep learning detection list containing clustered point clouds; a deep learning detection list without clustered point clouds; and a cluster without deep learning semantic attributes.

[0019] Furthermore, in step (2.1), its shape is changed to 1.1-1.2 times of its original size. Expansion is to enlarge the minimum rectangular box of deep learning.

[0020] Furthermore, in step (2.3), the splicing method is to assign the point cloud data in the cluster to the minimum rectangular box detected by deep learning, and the minimum rectangular box contains the point cloud data of the outer box and the obstacles in the box.

[0021] Furthermore, step (3) specifically includes:

[0022] (3.1) Taking the above deep learning detection list without clustered point clouds and the clusters without deep learning semantic attributes as input, we traverse the obstacles detected by the clustering module and determine the distance between the clustering result and the deep learning detection result. If the distance is less than a certain threshold, we consider that there is a correlation between the two.

[0023] (3.2) Perform under-segmentation clustering judgment. If the judgment conditions are met, the cluster is considered to be under-fitting. According to the current situation, the current cluster is further disassembled into multiple small clusters;

[0024] (3.3) Continuously adjust the clustering parameters, reduce the clustering parameters, split the current cluster into multiple small clusters, and then interpret the intersection ratio between them and the deep learning detection results. The cluster with the largest intersection ratio is the current detection result.

[0025] (3.4) The above detection results are continuously cycled, and finally the clustering work is completed, and the detection results are obtained: clusters not related to deep learning, deep learning lists without clustered point clouds, and deep learning lists with clustered point clouds.

[0026] Furthermore, in step (3.2), the judgment conditions for under-segmentation clustering are: judgment condition 1: the ratio of the intersection of the cluster and the CNN to the cluster area is greater than the threshold; condition 2: the ratio of the intersection of the cluster and the CNN to the CNN area is less than the threshold.

[0027] Furthermore, in step (3.4), the confidence of the clusters not related to deep learning is 0.5, the confidence of the deep learning list without clustered point clouds is 0, and the confidence of the deep learning list with clustered point clouds is 1; the confidence is set artificially.

[0028] The beneficial effect of the present invention is that, compared with the existing technology, the present invention integrates the detection results of Euclidean clustering and deep learning, thereby improving the perception accuracy of the lidar.

[0029] The present invention integrates the clustering results with the deep learning detection results to eliminate the deep learning error detection situation; and automatically adjusts the parameters of the cluster cluster to obtain the optimal clustering parameters for the current area.

[0030] The fusion result of the present invention includes confidence, and the confidence information can provide effective information for multi-sensor fusion modules such as radar and camera, and can improve the robustness of the autonomous driving perception module. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of the obstacle detection method for laser radar described in the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are not intended to limit the scope of protection of this application.

[0033] like Figure 1 As shown, the lidar obstacle detection method based on Euclidean clustering and deep learning described in the present invention integrates the detection results of Euclidean clustering and deep learning, and mainly includes the following three steps:

[0034] The laser radar is used to detect vehicle obstacles, and the real-time point cloud is obtained by the laser radar. The point cloud data is collected as input and analyzed using the algorithm of the present invention to obtain the detection results.

[0035] (1) First, the deep learning method is used to process the point cloud data obtained by the lidar obstacle detection to obtain the deep learning detection results; then Euclidean clustering is added to perform deep learning false detection processing to eliminate the false detection of deep learning and obtain a list of deep learning detection results that exclude false detection;

[0036] (1.1) Use deep learning methods to detect lidar point cloud data and obtain deep learning detection results;

[0037] Using common deep learning methods such as pointpillar and pointnet, we detect obstacles such as trucks, buses, two-wheeled bicycles, motorcycles, and pedestrians from LiDAR point cloud data and generate an obstacle list. Each obstacle is enclosed by a minimum rectangular box (bounding box).

[0038] (1.2) The Euclidean clustering method is used to detect the lidar point cloud data and obtain clusters; the deep learning detection results are associated with the corresponding clusters. If the intersection over union (IOU) of the associated Euclidean cluster and the deep learning detection area is less than the threshold (0.2), it is considered that the current situation is a deep learning error detection and it is removed from the detection list.

[0039] The clusters obtained by clustering, that is, the positions of obstacles in the environment, are highly robust and will not cause missed detection or false detection.

[0040] The distance between the center position of the deep learning minimum rectangular box (bounding box) and the center position of the cluster is calculated. If the distance is less than the set threshold, it is considered to be associated.

[0041] Calculate the intersection ratio of the associated Euclidean clustering and deep learning detection areas. The larger the value, the higher the probability of the same object; the smaller the value, the lower the probability of the same object. When the threshold is less than 0.2, the current situation is considered a deep learning false detection and is removed from the detection list.

[0042] (2) Perform merged clustering over-segmentation and traverse all deep learning and clustering detection results to obtain three groups of detection results: a deep learning detection list containing clustered point clouds; a deep learning detection list without clustered point clouds; and a cluster without deep learning semantic attributes.

[0043] (2.1) The area (bird's-eye-view projection area) of the deep learning obstacle detected in the above step that has been excluded from false detection is expanded to 1.1-1.2 times its original size;

[0044] Because actual point clouds contain noise and jitter, clustering parameters may not be optimal, leading to over-segmentation. Therefore, we dilate the deep learning detection results. Dilation is the process of enlarging the deep learning bounding box for subsequent reassociation with the clusters.

[0045] (2.2) Based on the results of deep learning detection that excludes false detections, the obstacles detected by the clustering module are traversed separately to perform target association. The distance between the clustering results and the deep learning detection results is determined. If the distance is less than a certain threshold, the two are considered to be correlated.

[0046] Based on the center position of the deep learning minimum bounding box (bbox) and the center position of the cluster, the distance between the two is calculated. If the distance is less than the set threshold, it is considered to be associated. The center distances of all the deep learning detection lists and cluster lists are calculated separately to exclude false positives.

[0047] (2.3) Determine one by one whether multiple clusters belong to the same deep learning detection target. If the ratio of the intersection of the Euclidean clustering and the deep learning detection area to the cluster is greater than the threshold, the cluster is considered to belong to the deep learning detection result, and the point cloud of the cluster is spliced ​​into the deep learning detection result; if it is less than the threshold, it is detected as a cluster without deep learning semantic attributes;

[0048] The splicing method is to assign the point cloud data in the cluster to the bbox detected by deep learning. Therefore, the bbox contains the point cloud data of the outer box and the obstacles in the box.

[0049] (2.4) Repeat the above steps to traverse all the deep learning and clustering detection results to obtain three sets of detection results:

[0050] A list of deep learning detections containing clustered point clouds (point clouds are assigned to deep learning clusters with a confidence level of 1);

[0051] List of deep learning detections without clustered point clouds;

[0052] Clustering without deep learning semantic attributes (clustering without semantic information).

[0053] (3) Taking the above deep learning detection list without clustered point cloud and the cluster without deep learning semantic attributes as input, perform split clustering incomplete segmentation to obtain the final detection result;

[0054] (3.1) Taking the above deep learning detection list without clustered point clouds and the clusters without deep learning semantic attributes as input, we traverse the obstacles detected by the clustering module and determine the distance between the clustering result and the deep learning detection result. If the distance is less than a certain threshold, we consider that there is a correlation between the two.

[0055] The thresholds are divided according to the attributes of obstacles detected by deep learning: 4m for trucks, 3m for buses, 2m for cars, 1m for motorcycles and bicycles, and 0.5m for pedestrians.

[0056] (3.2) Perform under-segmentation clustering processing: Judgment condition 1: the ratio of (cluster and CNN intersection) to cluster area is greater than the threshold (note: the value is 0.6), condition 2: the ratio of (cluster and CNN intersection) to CNN area is less than the threshold (note: the value is 1); CNN can represent the deep learning detection result;

[0057] If the above conditions are met, the clustering is considered to be underfitting, and the current cluster needs to be re-decomposed into multiple small clusters based on the current situation.

[0058] Under-segmentation means that two close vehicles and pedestrians are clustered into one vehicle or pedestrian, and a parked vehicle and a roadside branch are clustered into one vehicle, etc.

[0059] (3.3) Continuously adjust the clustering parameters, reduce the clustering parameters, split the current cluster into multiple small clusters, and then interpret the intersection over union (IOU) between the clusters and the deep learning detection results. The cluster with the maximum IOU is the current detection result.

[0060] In this embodiment, clustering is performed according to DBSCAN (conventional algorithm), and adjusting the clustering parameters is to reduce the search radius.

[0061] (3.4) The above detection results are continuously cycled to finally complete the re-clustering of the clusters. At the same time, the clustering parameters near the range are obtained and recorded for subsequent use.

[0062] The final output includes clusters not related to deep learning (confidence level 0.5), a list of deep learning point clouds without clusters (confidence level 0), and a list of deep learning point clouds with clusters (confidence level 1). The confidence information can provide effective information for multi-sensor fusion modules such as radar and camera, improving the robustness of autonomous driving perception modules.

[0063] The confidence level is artificially set: clusters not related to deep learning (confidence level is 0.5), deep learning lists without clustered point clouds (confidence level is 0), and deep learning lists with clustered point clouds (confidence level is 1).

[0064] The beneficial effect of the present invention is that, compared with the existing technology, the present invention integrates the detection results of Euclidean clustering and deep learning, thereby improving the perception accuracy of the lidar.

[0065] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation plans of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, and is not a limitation on the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A lidar obstacle detection method based on Euclidean clustering and deep learning, characterized in that: Including steps: (1) The deep learning method is used to process the point cloud data obtained by the lidar obstacle detection to obtain the deep learning detection results; then Euclidean clustering is added to perform deep learning false detection processing to eliminate the false detection of deep learning and obtain a list of deep learning detection results that exclude false detection; (2) Traverse the list of deep learning detection results that exclude false detections, perform merging and clustering over-segmentation, and obtain three sets of detection results: a deep learning detection list containing clustered point clouds; List of deep learning detections without clustered point clouds; Clustering without deep learning semantic attributes; (3) Taking the above deep learning detection list without clustered point cloud and the cluster without deep learning semantic attributes as input, perform split clustering incomplete segmentation to obtain the final detection result; Specifically include: (3.1) Using the deep learning detection list without clustered point clouds and the clusters without deep learning semantic attributes as input, we traverse the obstacles detected by the clustering module and determine the distance between the clustering result and the deep learning detection result. If the distance is less than a certain threshold, we consider the two to be correlated. (3.2) Perform under-segmentation clustering judgment. If the judgment conditions are met, the cluster is considered to be under-fitting. The current cluster is further decomposed into multiple small clusters according to the current situation. (3.3) Continuously adjust the clustering parameters, reduce the clustering parameters, split the current cluster into multiple small clusters, and then interpret the intersection ratio between them and the deep learning detection results. The cluster with the largest intersection ratio is the current detection result. (3.4) The above detection results are continuously cycled, and finally the clustering work is completed, and the detection results are obtained: clusters unrelated to deep learning, deep learning lists without clustered point clouds, and deep learning lists with clustered point clouds.

2. The laser radar obstacle detection method based on Euclidean clustering and deep learning according to claim 1, characterized in that: Step (1) specifically includes: (1.1) Use deep learning methods to detect LiDAR point cloud data, obtain deep learning detection results, and form an obstacle list; each obstacle is surrounded by a minimum rectangular box; (1.2) Use the Euclidean clustering method to detect the lidar point cloud data and obtain clusters; construct association properties between the deep learning detection results and the corresponding clusters. If the intersection ratio of the associated Euclidean cluster and the deep learning detection area is less than a threshold, it is considered that the current situation is a deep learning error detection and it is removed from the detection list.

3. The laser radar obstacle detection method based on Euclidean clustering and deep learning according to claim 2, characterized in that: In step (1.2), the distance between the center position of the deep learning minimum rectangular box and the center position of the cluster is calculated. If the distance is less than the set threshold, it is considered to be associated.

4. The laser radar obstacle detection method based on Euclidean clustering and deep learning according to claim 1, characterized in that: Step (2) specifically includes: (2.1) dilating the area of ​​obstacles detected by deep learning to eliminate false detections; (2.2) Using the deep learning detection results that exclude false detections as a benchmark, traverse the obstacles detected by the clustering module and perform target association. Determine the distance between the clustering results and the deep learning detection results. If the distance is less than a certain threshold, the two are considered to be correlated. (2.3) Determine one by one whether multiple clusters belong to the same deep learning detection target. If the ratio of the intersection of the Euclidean clustering and the deep learning detection area to the cluster is greater than the threshold, the cluster is considered to belong to the deep learning detection result, and the point cloud of the cluster is spliced ​​into the deep learning detection result. If it is less than the threshold, it is detected as a cluster without deep learning semantic attributes. (2.4) Repeat the above steps to traverse all deep learning and clustering detection results, and obtain three groups of detection results: a deep learning detection list with clustered point clouds; a deep learning detection list without clustered point clouds; and a cluster without deep learning semantic attributes.

5. The laser radar obstacle detection method based on Euclidean clustering and deep learning according to claim 4, characterized in that: In step (2.1), its shape is changed to 1.1-1.2 times of its original size. Expansion is to enlarge the minimum rectangular box of deep learning.

6. The laser radar obstacle detection method based on Euclidean clustering and deep learning according to claim 4, characterized in that: In step (2.3), the stitching method is to assign the point cloud data in the cluster to the minimum rectangular box detected by deep learning. The minimum rectangular box contains the point cloud data of the outline box and the obstacles in the box.

7. The laser radar obstacle detection method based on Euclidean clustering and deep learning according to claim 1, characterized in that: In step (3.2), the judgment conditions for under-segmentation clustering are: judgment condition 1: the ratio of the intersection of the cluster and CNN to the cluster area is greater than the threshold; condition 2: the ratio of the intersection of the cluster and CNN to the CNN area is less than the threshold.

8. The laser radar obstacle detection method based on Euclidean clustering and deep learning according to claim 1, characterized in that: In step (3.4), the confidence of the clusters not related to deep learning is 0.5, the confidence of the deep learning list without clustered point clouds is 0, and the confidence of the deep learning list with clustered point clouds is 1; the confidence is set manually.

Images