Lidar object detection method based on edge detection optimization

Abstract

The application discloses a laser radar object detection method based on edge detection optimization, and comprises an initial coarse prediction module, an edge detection module and an optimized prediction module.The initial coarse prediction module comprises a downsampling function, a feature encoding function, a feature extraction function, a regression function and a classification function.The edge detection module comprises a region point cloud extraction function, a dense filling and projection function and an edge detection function.The optimized prediction module comprises an edge detection discrimination function, a feature extraction function, a regression function and a classification function.The application can solve the problem that one object detection bounding box contains multiple objects in the scenario where there are many objects and the distance between adjacent objects is short during driving of the current autonomous vehicle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving perception, and particularly relates to a laser radar object detection method based on edge detection optimization. BACKGROUND

[0002] In the process of driving an automatic driving vehicle, accurate perception of the surrounding environment of the vehicle is crucial, and object detection is a very important perception technology in the task of environment perception. With the development of deep learning, object detection methods based on deep learning have been widely used due to their excellent performance. The existing laser radar object detection methods are divided into single-stage and multi-stage according to the number of feature extraction. Among them, the single-stage laser radar object detection method (PointPillars, SASSD, SESSD, etc.) directly uses the learned features for object classification and regression prediction after feature encoding and feature extraction operations on the input laser radar point cloud data. The multi-stage laser radar object detection method (PointRCNN, PV-RCNN, Voxel-RCNN, etc.) performs a rough binary classification (foreground and background) and regression prediction after feature encoding and feature extraction operations on the input laser radar point cloud data in the first stage, and performs further feature extraction on the ROI obtained in the first stage in the second stage, and finally obtains accurate classification and regression prediction results combined with the features in the first stage.

[0003] The existing single-stage laser radar object detection method directly uses the original point cloud data representing the front of the object to regress the length, width, height and center point position of the entire object through a large amount of data and the powerful representation ability of the convolutional neural network. Although this method can obtain a bounding box that fits the shape of the object, it will only obtain one bounding box and classification (actually two) of the object in the situation where multiple objects are closely attached (such as a person opening the car door to get in / out of the car). Such prediction results will have a certain adverse effect on the subsequent decision planning of the automatic driving vehicle and will interfere with the safe driving of the vehicle. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the present application provides a laser radar object detection method based on edge detection optimization, which can solve the problem that one object detection bounding box contains multiple objects in the scenario where there are many objects and the distance between adjacent objects is close in the process of driving the current automatic driving vehicle.

[0005] To solve the above technical problems, the application provides the following technical scheme: a laser radar object detection method based on edge detection optimization, which comprises an initial coarse prediction module and an edge detection module.

[0006] As a preferred technical scheme of the application, the downsampling function in the initial coarse prediction module is used for downsampling original point cloud data, the feature encoding function in the initial coarse prediction module is used for encoding information of the downsampled point cloud data, the feature extraction function in the initial coarse prediction module uses a convolutional neural network to process input voxels to obtain high-level semantic information of the input point cloud data, the regression function in the initial coarse prediction module uses the high-level semantic information obtained by the feature extraction function to regress the prediction result of an obstacle such as a center coordinate, a length-width-height, an orientation and the like, and these information is represented by a cuboid bounding box with an orientation identifier, and the classification function in the initial coarse prediction module classifies the point cloud (i.e. the obstacle) in the bounding box into categories such as pedestrians, vehicles and the like, and outputs the confidence of the classification result, and finally, the M bounding boxes with the highest confidence are selected by the NMS algorithm according to the confidence.

[0007] As a preferred technical scheme of the application, the region point cloud extraction function in the edge detection module uses the three-dimensional bounding box obtained by the initial coarse prediction module, the dense filling function in the edge detection module is used for filling the extracted region point cloud data, and at the same time, the data needs to be processed to remove the height value (directly zeroize the Z coordinate of all points) for the next edge detection function, and a bird's eye view of the BEV projection can be obtained, and the edge detection function in the edge detection module is used for judging the included angle of each point P and the K surrounding points in the BEV bird's eye view, and when the included angle is greater than a certain value, the point P is determined as an edge point.

[0008] As a preferred technical scheme of the application, the edge detection function comprises the following specific steps: step one: for each point P, project all points (N, not including point P) in the neighborhood (spherical grouping or K-nearest neighbor search) of point P to the tangent plane of point P, at this time, the position of point P does not change after projection; step two: in the projected plane, all points are connected with P, and a set of included angles θ={θ1, θ2, … θN-1} can be obtained according to a certain order (counterclockwise or clockwise) of each adjacent two connecting lines, and the included angle is calculated by the formula θi=|θi-1-θi|, i=1, 2, … N-1, and when θi>θ0, point P is determined as an edge point. NStep three: find the maximum included angle θ N in the included angle set θ = {θ1, θ2, … θ max = max(θ);Step four: when θ max is larger, the point P is more likely to be a boundary point, at this time, a threshold is set, when θ max is larger than the set threshold, the point P is determined as a boundary point;Step five: repeat the above steps to realize edge detection, and obtain an edge point set of the point cloud in the bounding box.

[0009] As a preferred technical solution of the present application, the edge detection discrimination function in the optimization prediction module utilizes the fact that objects in three-dimensional space exist independently and are at a certain distance from each other, and then can judge whether the result of the edge detection module is continuous, if continuous, it proves that the box contains an object, and there is no need for optimization, the initial rough prediction result can be output, if not continuous, it means that the box contains multiple objects, and optimization is needed, the feature extraction function in the optimization prediction module utilizes the convolutional neural network to process the input points to obtain high-level semantic information of the input point cloud data, the regression function in the optimization prediction module utilizes the high-level semantic information obtained by the feature extraction function to optimize the prediction regression result of the obstacle, and the classification function in the optimization prediction module is used for classifying the point cloud (i.e. obstacle) in the bounding box and outputting the confidence of the classification result, and finally the bounding box with the highest confidence number is selected through the NMS algorithm according to the confidence.

[0010] As a preferred technical solution of the present application, the region point cloud extraction function in the edge detection module takes into account that the initial box is not accurate enough, and the box may be smaller relative to the real object, at this time, the bounding box is enlarged by a certain multiple, and then all the point cloud data in the three-dimensional bounding box are extracted through the coordinate information.

[0011] Compared with the prior art, the present application has the following beneficial effects:

[0012] The present application can fill the points of the point cloud data in the bounding box obtained in the first stage, densify the sparse original point cloud data, then project the densified point cloud data (in the bounding box) in BEV, perform edge detection based on the bird's eye view obtained by BEV projection, judge whether the bounding box contains multiple objects through the edge detection result, if the bounding box contains multiple objects, perform second detection on the small range of point cloud in the bounding box, optimize the result, and obtain more accurate detection result, solving the problem that one object detection bounding box contains multiple objects in the scene where there are more objects and the distance between adjacent objects is close in the current automatic driving vehicle driving. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1This is a schematic diagram of the overall method flow of the present invention;

[0014] Figure 2 This is a schematic diagram of the edge detection algorithm for lidar point cloud data in this invention. Detailed Implementation

[0015] To make the technical means, creative features, and achieved objectives and effects of this invention easier to understand, the invention is further described below with reference to specific embodiments. However, the following embodiments are merely preferred embodiments of this invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments described herein without creative effort are all within the protection scope of this invention. Unless otherwise specified, the experimental methods in the following embodiments are conventional methods, and the materials and reagents used in the following embodiments are commercially available unless otherwise specified.

[0016] Example:

[0017] Example 1:

[0018] like Figure 1 , Figure 2 As shown, this invention provides a lidar object detection method based on edge detection optimization. The lidar object detection method includes: an initial coarse prediction module, which includes downsampling, feature encoding, feature extraction, regression, and classification functions; an edge detection module, which includes region point cloud extraction, dense filling and projection, and edge detection functions; and an optimized prediction module, which includes edge detection discrimination, feature extraction, regression, and classification functions.

[0019] The downsampling function in the initial coarse prediction module is used to downsample the original point cloud data, the feature encoding function in the initial coarse prediction module is used to encode the information of the downsampled point cloud data, the feature extraction function in the initial coarse prediction module uses a convolutional neural network to process the input voxels to obtain high-level semantic information of the input point cloud data, the regression function in the initial coarse prediction module uses the high-level semantic information obtained by the feature extraction function to regress the prediction results of the obstacles such as the center coordinates, length, width, height, orientation and the like, which are represented by a cuboid bounding box with an orientation identifier, the classification function in the initial coarse prediction module classifies the point cloud (i.e. the obstacles) in the bounding box by category, such as pedestrians, vehicles and the like, and outputs the confidence of the classification result, and finally the M bounding boxes with the highest confidence are selected by the NMS algorithm according to the confidence, the region point cloud extraction function in the edge detection module uses the three-dimensional bounding box obtained by the initial coarse prediction module, the dense filling function in the edge detection module is used to fill and densify the region point cloud data extracted above, and at the same time, the data needs to be processed to remove the height value (directly zeroize the Z coordinate of all points) for the next edge detection function, and the bird's eye view of the BEV projection can be obtained, the edge detection function in the edge detection module is used to judge the included angle of the adjacent two lines formed by each point P and the surrounding K points in the BEV bird's eye view, and when the included angle is greater than a certain value, it is determined that the point P is an edge point, the region point cloud extraction function in the edge detection module considers that the initial bounding box is not accurate enough and may be smaller relative to the real object, so the bounding box is expanded by a certain multiple, and then all the point cloud data in the three-dimensional bounding box is extracted through the coordinate information, the edge detection discrimination function in the optimization prediction module uses the fact that objects in a three-dimensional space exist independently and there is a certain distance between them to judge whether the result of the edge detection module is continuous, if it is continuous, it proves that the box contains an object and optimization is not needed, and the initial coarse prediction result is output, if it is not continuous, it means that the box contains multiple objects and needs to be optimized, the feature extraction function in the optimization prediction module uses a convolutional neural network to process the input points to obtain high-level semantic information of the input point cloud data, the regression function in the optimization prediction module uses the high-level semantic information obtained by the feature extraction function to optimize the prediction regression result of the obstacles, and the classification function in the optimization prediction module is used to classify the point cloud (i.e. the obstacles) in the bounding box by category and output the confidence of the classification result, and finally the bounding box with the highest confidence is selected by the NMS algorithm according to the confidence.

[0020] In the initial coarse prediction module, the point cloud data needs to be downsampled to reduce the amount of calculation and the cost of calculation.

[0021] In the edge detection module, considering that the region point cloud extracted in the region point cloud extraction function is relatively sparse and cannot be used for edge detection continuity judgment, the region point cloud data needs to be filled and densified first.

[0022] The input point cloud data enters the initial prediction module, and the point cloud data is sequentially down-sampled, feature encoded, and feature extracted, and then classified and detected for regression, so as to obtain the coordinate position, length-width-height size, pose orientation, category and confidence of the initial prediction obstacle. Then, the data enters the edge detection module, and the region point cloud data is extracted, filled and projected, and then edge detection is performed. Then, the data is judged. If it is continuous, the initial prediction obstacle result is output. If it is not continuous, the point cloud data feature encoding, re-extraction, classification and detection regression are performed again. Finally, the coordinate position, length-width-height size, pose orientation, category and confidence of the predicted obstacle are optimized, and the final prediction obstacle result is output.

[0023] Embodiment 2

[0024] As shown in Figure 2 , further elaboration is made on the basis of embodiment 1. The specific steps of the edge detection function include:

[0025] Step 1: For each point P, project all points (N, not including point P) in the neighborhood of point P (spherical grouping or K-nearest neighbor search) to the tangent plane of point P. At this time, the position of point P does not change after projection.

[0026] Step 2: In the projected plane, all points are connected with P, and a set of angles θ = {θ1, θ2, … θ N} can be obtained according to a certain order (counterclockwise or clockwise) by connecting each adjacent two lines.

[0027] Step 3: Find the maximum angle θ N = max(θ) in the angle set θ = {θ1, θ2, … θ max}.

[0028] Step 4: The larger θ max is, the more likely point P is a boundary point. At this time, a threshold is set. When θ max is greater than the set threshold, point P is determined to be a boundary point.

[0029] The application fills the points of the point cloud data in the bounding box obtained in the first stage, densifies the sparse original point cloud data, then performs BEV projection on the densified point cloud data (in the bounding box), performs edge detection based on the overhead view obtained by the BEV projection, and judges whether the bounding box contains multiple objects through the edge detection result. If the bounding box contains multiple objects, the point cloud in a small range in the bounding box is detected for a second time, the result is optimized, and a more accurate detection result is obtained, solving the problem that one object detection bounding box contains multiple objects in the scene where the current autonomous vehicle is running, the number of objects is large, and the distance between adjacent objects is close.

[0030] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for object detection of a laser radar based on edge detection optimization, characterized in that: The laser radar object detection method comprises: An initial coarse prediction module, which comprises a downsampling function, a feature coding function, a feature extraction function, a regression function and a classification function; An edge detection module, which comprises a region point cloud extraction function, a dense filling and projection function and an edge detection function; An optimized prediction module, which comprises an edge detection discrimination function, a feature extraction function, a regression function and a classification function; The downsampling function in the initial coarse prediction module is used for downsampling the original point cloud data, the feature coding function in the initial coarse prediction module is used for coding the information of the point cloud data after downsampling, the feature extraction function in the initial coarse prediction module is used for processing the input voxels by using a convolutional neural network to obtain high-level semantic information of the input point cloud data, the regression function in the initial coarse prediction module is used for regressing the prediction result center coordinates, length, width, height and orientation information of the obstacle by using the high-level semantic information obtained by the feature extraction function, and the classification function in the initial coarse prediction module is used for classifying the point cloud in the bounding box and outputting the confidence of the classification result, and finally, the M bounding boxes with the highest confidence are selected by using the NMS algorithm according to the confidence. The region point cloud extraction function in the edge detection module uses the three-dimensional bounding box obtained by the initial coarse prediction module, the dense filling function in the edge detection module is used for densely filling the region point cloud data extracted above, and at the same time, the data is processed in height to obtain the bird's eye view of the BEV projection, and the edge detection function in the edge detection module is used for judging the angle between the connecting lines of each point P and the surrounding K points in the BEV bird's eye view, and when the angle between the adjacent two lines is greater than a certain value, the point P is determined as an edge point. The edge detection discrimination function in the optimized prediction module is used for judging whether the results of the edge detection module are continuous according to the fact that the objects in the three-dimensional space exist independently and there is a certain distance between the objects, and if the results are continuous, it is proved that the bounding box contains an object, and optimization is not needed, and the initial coarse prediction result can be output, and if the results are not continuous, it is proved that the bounding box contains multiple objects, and optimization is needed, the feature extraction function in the optimized prediction module is used for processing the input points by using a convolutional neural network to obtain high-level semantic information of the input point cloud data, the regression function in the optimized prediction module is used for optimizing the prediction regression result of the obstacle by using the high-level semantic information obtained by the feature extraction function, and the classification function in the optimized prediction module is used for classifying the point cloud in the bounding box and outputting the confidence of the classification result, and finally, the bounding box with the highest confidence is selected by using the NMS algorithm according to the confidence.

2. The edge detection optimization based lidar object detection method of claim 1, wherein: The edge detection function comprises the following specific steps: S1: For each point P, project all points in the neighborhood of point P that do not contain point P into the tangent plane of point P by spherical grouping or K-nearest neighbor search. At this time, the position of point P does not change after projection. S2: In the plane of the projection, all points are connected with P, and according to the clockwise or counterclockwise order, a set of angles can be obtained, θ = {θ1, θ2, … θn}, where each adjacent two lines form an angle. N} S3: In the set of included angles θ = {θ1, θ2, ... θ} N Find the largest included angle θ max =max(θ); S4: When θ max is larger, the point P is more likely to be a boundary point, at which a threshold is set, and when θ max is larger than the set threshold, the point P is determined as a boundary point. S5: Repeat the above steps to perform edge detection and obtain a set of edge points in the point cloud within the bounding box.

3. The edge detection optimization based lidar object detection method of claim 1, wherein: The information of the downsampled point cloud data in the feature encoding function of the initial coarse prediction module includes the three-dimensional coordinates, reflection intensity, and time series information of each point. The above multiple sets of information are encoded to obtain the information contained in a voxel.

4. The edge detection optimization based lidar object detection method of claim 1, wherein: The region point cloud extraction function in the edge detection module takes into account that the initial bounding box may not be accurate enough and may be smaller than the real object. Therefore, the bounding box is enlarged by a certain factor, and then all point cloud data within the 3D bounding box is extracted using coordinate information.

Images