Intelligent driving automobile safety target detection method

Abstract

The invention discloses an intelligent driving automobile safety target detection method, and relates to the technical field of automatic driving automobiles. The method includes: acquiring an original point cloud of a vehicle driving environment by using a laser radar; preprocessing the original point cloud to remove noise in the original point cloud; performing ground segmentation processing on the preprocessed point cloud to obtain a ground point cloud and an obstacle point cloud; inputting the obstacle point cloud into a 3D point cloud target detection algorithm to obtain a first target detection result, and inputting the obstacle point cloud into a clustering algorithm to obtain a second target detection result; wherein the 3D point cloud target detection algorithm is a Point Pillars target detection algorithm, and the clustering algorithm is a DBSCAN algorithm; and performing combination processing on the first target detection result and the second target detection result, and outputting a target detection result after combination processing. According to the invention, the obstacle can be effectively identified in the vehicle driving process, and the detection accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving vehicles, and specifically is a method for detecting safety targets of intelligent driving vehicles. Background Art

[0002] With the rapid progress of artificial intelligence and other related professions, new changes have taken place in the field of intelligent transportation, and self-driving cars have emerged. If the self-driving technology can be successfully put into use, it will effectively solve problems such as road congestion and environmental pollution. As an important part of self-driving technology, environmental perception has the core task of target detection. The current mainstream is the 3D point cloud target detection algorithm, which can be divided into two categories: non-deep learning algorithms and deep learning algorithms. The 3D point cloud target detection algorithm based on non-deep learning can cluster according to various attributes of the point cloud, and then identify any point cloud cluster. However, this type of algorithm has the defects of being very sensitive to noise and taking a long time to detect. The 3D point cloud target detection algorithm based on deep learning has the advantage of fast detection speed, but its detection ability is relatively weak for objects outside the label range of the data set.

[0003] The prior art (invention patent application with publication number CN114155511A) discloses a method for collecting environmental information for self-driving cars on public roads. The method uses a drone to accompany the test vehicle for a long time and steadily, uses the drone's onboard environmental information collection equipment to collect road information ahead, pre-processes the collected video frames, and performs lane line inspection, pedestrian target detection and tracking, and vehicle target detection and tracking on the pre-processed video frames. However, existing self-driving car technology can usually detect pedestrians and vehicles, but in actual driving, it may not be able to identify small obstacles. If it cannot be identified in time and effective obstacle avoidance operations cannot be implemented, it is very likely to cause serious traffic accidents.

[0004] The present invention provides a method for detecting safe targets of intelligent driving vehicles to solve the above technical problems. Summary of the invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art; to this end, the present invention proposes a safe target detection method for an intelligent driving car, which is used to solve the technical problems such as missed detection and wrong detection existing in the perception system of the autonomous driving car.

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for detecting a safety target of an intelligent driving vehicle, comprising:

[0007] Step 1: Use LiDAR to obtain the original point cloud of the vehicle's driving environment;

[0008] Step 2: Preprocess the original point cloud to remove the noise;

[0009] Step 3: Perform ground segmentation processing on the point cloud obtained after preprocessing to obtain ground point cloud and obstacle point cloud;

[0010] Step 4: Input the obstacle point cloud into the 3D point cloud target detection algorithm to obtain the first target detection result, and input the obstacle point cloud into the clustering algorithm to obtain the second target detection result; wherein the 3D point cloud target detection algorithm is the PointPillars target detection algorithm, and the clustering algorithm is the DBSCAN algorithm;

[0011] Step 5: Combine the first target detection result and the second target detection result, and output the combined target detection result.

[0012] Preferably, the original point cloud is preprocessed, including:

[0013] Step 2.1: Establish the topological relationship between the original point clouds through the octree;

[0014] Step 2.2: Use principal component analysis to estimate the normal vector of the point cloud, and make the normal vector direction consistent through the minimum spanning tree;

[0015] Step 2.3: De-noise the point cloud obtained after step 2.2 using a bilateral filter.

[0016] The present invention establishes the topological relationship between the original point clouds through the octree, which helps to quickly locate and retrieve specific areas or points in the point cloud data and improve the efficiency of data processing; through the principal component analysis method, the normal vector can be accurately calculated based on the distribution of the point cloud, providing a reliable basis for subsequent processing; after the point cloud is denoised by the bilateral filter, the point cloud quality is improved, providing data support for subsequent point cloud segmentation, target detection and other tasks.

[0017] Preferably, estimating the normal vector of the point cloud using the principal component analysis method includes:

[0018] The normal vector directions of two adjacent point clouds are determined by principal component analysis;

[0019] When the dot product result of the normal vector is greater than 0, the normal vectors have the same direction; when the dot product result of the normal vector is less than 0, the normal vectors have opposite directions and are adjusted.

[0020] Preferably, adjusting the normal vector includes:

[0021] Select the normal vector of any point through the Riemann diagram of the point cloud and propagate along the direction of the normal vector;

[0022] When the dot product result between the normal vectors of the point cloud is less than 0, the normal vector is flipped through the minimum spanning tree, and the above operation is repeated to make the normal vectors of all points have the same direction.

[0023] Preferably, inputting the obstacle point cloud into a 3D point cloud target detection algorithm to obtain a first target detection result includes:

[0024] The obstacle point cloud is divided into h×w grids on the XY plane; each grid generates a point column in the z-axis direction, and the points in each point column are represented by a 9-dimensional vector D (x, y, z, r, x c ,y c ,z c ,x p ,y p ), where x, y, and z represent the position of the point cloud in three-dimensional space, r represents the laser radar reflection intensity, and x c ,y c , z c Indicates the distance from the point to the arithmetic mean of all points in the bar, x p ,y p Indicates the offset value from the point to the center of the point column;

[0025] Set the number of point columns to P. The number of points in each point column is less than or equal to N. When the number of points in the point column is greater than N, use random sampling to select N points from the point column; otherwise, use 0 to fill the number of points in the point column to N.

[0026] The points in the point cloud data are encoded into a tensor of (D, P, N); each point is processed through a linear layer, BatchNorm and ReLu to generate a tensor of dimension (C, P, N); a maximum pooling operation is performed on each point column to obtain a tensor of dimension (C, P), and a pseudo image is generated through a scatter operator; the generated pseudo image is input into 16 layers of stacked convolution for feature extraction, and three sets of different feature maps are output; finally, the three sets of different feature maps are sent to the detection head composed of three deconvolutions and two 1×1 convolutions, and the first target detection result is output.

[0027] The present invention is based on the PointPillars target detection algorithm, through the above series of processing and operations, and can accurately detect the target object in the obstacle point cloud, thereby improving the accuracy and precision of target detection.

[0028] Preferably, inputting the obstacle point cloud into a clustering algorithm to obtain a second target detection result includes:

[0029] The obstacle point cloud is input into the DBSCAN algorithm for target detection. An arbitrary point g is selected and statistical judgment is performed on the points within the point g area ε. When the point density is greater than or equal to the threshold Minps, point g is recorded as a core point and a new category is created. Otherwise, point g is marked as a noise point. Other arbitrary points in the point cloud are selected and the above steps are repeated until all points are marked as processed. The category information point output in the cluster is the second target detection result.

[0030] After the obstacle point cloud in the present invention is clustered, the point cloud data is divided into different categories, each category corresponds to a relatively independent object or area, which can locate and analyze the category to which the target object belongs more quickly, reduce computing time and resource consumption, and improve detection performance.

[0031] Preferably, the first target detection result and the second target detection result are combined for processing, including: setting a target confidence threshold, when neither the 3D point cloud target detection algorithm nor the clustering algorithm detects the target and the confidence does not exceed the target confidence threshold, rejecting the detection result; when both the 3D point cloud target detection algorithm and the clustering algorithm detect the target and exceed the target confidence threshold, accepting the detection result; wherein the detection result includes the first target detection result or the second target detection result; when either the 3D point cloud target detection algorithm or the clustering algorithm detects the target and exceeds the target confidence threshold, receiving the detection result of the algorithm.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The method includes point cloud preprocessing, ground point cloud segmentation, and point cloud target detection.

[0034] (1) The collected original point cloud is processed through point cloud preprocessing to remove noise points in the point cloud to prevent the noise points from affecting subsequent point cloud processing.

[0035] (2) Perform ground segmentation on the preprocessed point cloud to obtain ground point cloud and obstacle point cloud; use the obstacle point cloud as the input of target detection and discard the ground point cloud, thereby reducing the computational complexity of target detection.

[0036] (3) The obstacle point cloud is input into the 3D point cloud target detection algorithm to obtain the first target detection result, and the obstacle point cloud is input into the clustering algorithm to obtain the second target detection result. The first target detection result and the second target detection result are combined and processed, and the combined point cloud target detection result is output. The detection of obstacles in the point cloud is realized, and the effective identification of obstacles during vehicle driving is effectively solved, thereby improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a system block diagram of the present invention;

[0039] Figure 2 is a point cloud preprocessing flow chart of the present invention;

[0040] Figure 3 It is a flow chart of the method for fitting a piecewise plane according to the present invention;

[0041] Figure 4 It is a flow chart of the target detection algorithm of the present invention. DETAILED DESCRIPTION

[0042] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] See also Figure 1-Figure 4 , an embodiment of the present invention provides a method for detecting a safe target of an intelligent driving vehicle, comprising:

[0044] Step 1: Use LiDAR to obtain the original point cloud of the vehicle’s driving environment.

[0045] Equip the car with a laser radar, drive the car on the road, and collect raw laser point cloud data.

[0046] Step 2: Preprocess the original point cloud information to remove the noise points.

[0047] The collected original point cloud is preprocessed. The original point cloud first establishes the topological relationship between point clouds through the octree, and then uses PCA to estimate the normal vector. At the same time, the normal vectors of two adjacent points are judged. If the normal vector dot product result is greater than 0, the normal vectors have the same direction; if the normal vector dot product result is less than 0, the normal vectors have opposite directions. When the directions of the two normal vectors are opposite, their directions are adjusted so that the directions of the two normal vectors in opposite directions are consistent. The adjustment process is as follows: select the normal vector of any point through the Riemann diagram of the point cloud and propagate along the direction of the normal vector; when the dot product result between the point cloud normal vectors is less than 0, flip the normal vector through the minimum spanning tree, repeat the above operation, so that the normal vector directions of all points are consistent, and the processed point cloud is denoised through the bilateral filter.

[0048] Step 3: Perform ground segmentation on the point cloud obtained after preprocessing to obtain ground point cloud and obstacle point cloud.

[0049] The ground segmentation in step 3 is based on the segmented plane fitting method. The specific steps are as follows: divide each frame of point cloud into several segments based on distance, and perform the following operations on each segment of point cloud:

[0050] (1) The preprocessed point cloud M i Sort by z-axis value from large to small to get point cloud N i ;

[0051] (2) Calculate point cloud N i The average value H of the z-axis values ​​of the last N point clouds is used as M i minimum height;

[0052] (3) According to the formula, determine whether the point belongs to the initial seed point and traverse the point cloud M i , find all seed points that meet the conditions and use them as the ground point cloud for the initial plane fitting.

[0053] bz <H+Δh

[0054] Where b represents a point in the point cloud, and each point b can be represented by its coordinates x, y, z, where x and y represent the position of the horizontal plane, and z represents the height value relative to the reference plane. bz represents the z coordinate value of point b. Δh is a set threshold that defines the height tolerance of a point compared to the reference height H for it to be considered a ground point.

[0055] (4) Perform plane fitting on the ground point cloud. The specific steps are as follows: First, construct the covariance matrix CM and mean vector M for each point in the ground point cloud, then perform singular value decomposition on CM, and obtain the normal vector d(x, y, z) of the fitting plane through the singular vector corresponding to the minimum singular value; and obtain the plane equation parameters through the mean vector M to determine whether a certain point belongs to the distance threshold of the fitting plane; then use the point cloud M i The coordinate values ​​of each point in the matrix T (m is M i The number of midpoints), T×dT, obtains the distance vector V from each point to the fitting plane, traverses all elements in V, if the element value is less than the distance threshold, the point belongs to the fitting plane, and a new ground point cloud is obtained.

[0056] (5) Repeat the above step (4) to obtain the ground point cloud and obstacle point cloud.

[0057] Step 4: Input the obstacle point cloud into the 3D point cloud target detection algorithm to obtain the first target detection result, and at the same time input the obstacle point cloud into the clustering algorithm to obtain the second target detection result; wherein, the 3D point cloud target detection algorithm is the PointPillars target detection algorithm, and the clustering algorithm is the DBSCAN algorithm.

[0058] The PointPillars target detection algorithm includes linear layers, BatchNorm, ReLu, scatter operators, and 16 layers of stacked convolutions. The obstacle point cloud is divided into multiple point pillar units, each of which is divided into h×w grids on the XY plane;

[0059] Determine the h×w range based on the number of points on the x-axis and y-axis of the point cloud obtained in step 3 and the length and width of the expected Pillar. Assuming that the number of points on the x-axis and y-axis of the point cloud are X and Y respectively, and the length and width of the pillar are H and W, the calculation formula for h is h=X / H, w=Y / W. The number of points in the specific point cloud in the x and y directions is related to the lidar equipment.

[0060] Each grid generates a Pillar in the z-axis direction. The points in each Pillar are represented by a 9-dimensional vector D(x, y, z, r, x c ,y c ,z c ,x p ,y p ), where x, y, and z represent the position of the point cloud in three-dimensional space, r represents the laser radar reflection intensity, and x c ,y c , z c Indicates the distance to the arithmetic mean point of all points in the Pillar, x p ,yp Represents the offset value from the point to the x, y center of the Pillar. Set the number of point columns to P and each point column contains at most N points. When the number of points in the point column is greater than N, use random sampling to select N points from the point column; otherwise, use 0 to fill the points in the point column to N; through the above process, encode the points in the point cloud data into a tensor of (D, P, N); perform a convolution operation on each point through a 1×1 convolution kernel, and then generate a tensor of dimension (C, P, N) after BatchNorm and ReLu processing; perform a maximum pooling operation on each Pillar unit to obtain a tensor of dimension (C, P), and generate a pseudo image through a scatter operator; input the generated pseudo image into 16 layers of stacked convolution for feature extraction, and output three sets of different feature maps; finally, send the three sets of different feature maps to the detection head composed of three deconvolutions and two 1×1 convolutions, and output the target detection results.

[0061] The DBSCAN clustering algorithm clusters points with a density exceeding a preset threshold into classes based on the density of the point cloud, and classifies points with a density below the density threshold as noise points. The specific steps are as follows: (1) Under the preset threshold Minps, select a domain ε that increases adaptively with distance for the collected point cloud, and mark all data points as unprocessed. (2) Select an arbitrary point g and count the points within the domain ε of point g. The domain ε refers to the spatial range with point g as the center and ε as the radius. If the point density is greater than or equal to Minps, point g is recorded as a core point and a new category is created. Otherwise, point g is marked as a noise point. (3) Select any other point in the point cloud and repeat step 2 until all points are marked as processed. The category information points output in the cluster are the second target detection results, where the category information points are represented as core points and their corresponding categories.

[0062] Step 5: Combine the first target detection result and the second target detection result, and output the combined target detection result.

[0063] The combined processing process is: setting a target confidence threshold, when neither the 3D point cloud target detection algorithm nor the clustering algorithm detects the target and the confidence does not exceed the target confidence threshold, the detection result is rejected; when both the 3D point cloud target detection algorithm and the clustering algorithm detect the target and exceed the target confidence threshold, the detection result is accepted; wherein the detection result includes the first target detection result or the second target detection result; when either the 3D point cloud target detection algorithm or the clustering algorithm detects the target and exceeds the target confidence threshold, the detection result of the algorithm is received.

[0064] The working principle of the present invention is as follows: First, the original point cloud is obtained by using a laser radar. Then, the original point cloud is preprocessed, including establishing a topological relationship through an octree, estimating and adjusting the normal vector by using the principal component analysis method, and bilateral filtering for denoising. Then, the ground is segmented to obtain an obstacle point cloud. After that, two methods are used to perform target detection on the obstacle point cloud. One is to output the first target detection result based on the PointPillars target detection algorithm; the other is to output the second target detection result based on the clustering processing of the DBSCAN algorithm. Finally, the two target detection results are combined by setting a confidence threshold, and the final target detection result is output to ensure the safety of the smart driving car.

[0065] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for detecting safe targets of intelligent driving vehicles, characterized in that: include: Step 1: Use LiDAR to obtain the original point cloud of the vehicle's driving environment; Step 2: Preprocess the original point cloud to remove the noise; Step 3: Perform ground segmentation processing on the point cloud obtained after preprocessing to obtain ground point cloud and obstacle point cloud; Step 4: Input the obstacle point cloud into the 3D point cloud target detection algorithm to obtain the first target detection result, and input the obstacle point cloud into the clustering algorithm to obtain the second target detection result; wherein the 3D point cloud target detection algorithm is the PointPillars target detection algorithm, and the clustering algorithm is the DBSCAN algorithm; Step 5: Combine the first target detection result and the second target detection result, and output the combined target detection result.

2. The method for detecting safe targets of intelligent driving vehicles according to claim 1, characterized in that: The original point cloud is preprocessed, including: Step 2.1: Establish the topological relationship between the original point clouds through the octree; Step 2.2: Use principal component analysis to estimate the normal vector of the point cloud, and make the direction of the normal vector consistent through the minimum spanning tree; Step 2.3: De-noise the point cloud obtained after step 2.2 using a bilateral filter.

3. The method for detecting safe targets of intelligent driving vehicles according to claim 2, characterized in that: The principal component analysis method is used to estimate the normal vector of the point cloud, including: The normal vector directions of two adjacent point clouds are determined by principal component analysis; When the dot product result of the normal vector is greater than 0, the normal vectors have the same direction; when the dot product result of the normal vector is less than 0, the normal vectors have opposite directions and are adjusted.

4. The method for detecting a safe target for an intelligent driving vehicle according to claim 3, characterized in that: The normal vector is adjusted, including: Select the normal vector of any point through the Riemann diagram of the point cloud and propagate along the direction of the normal vector; When the dot product result between the normal vectors of the point cloud is less than 0, the normal vector is flipped through the minimum spanning tree, and the above operation is repeated to make the normal vectors of all points have the same direction.

5. The method for detecting safe targets of intelligent driving vehicles according to claim 1, characterized in that: Inputting the obstacle point cloud into a 3D point cloud target detection algorithm to obtain a first target detection result, including: The obstacle point cloud is divided into h×w grids on the XY plane; each grid generates a point column in the z-axis direction, and the points in each point column are represented by a 9-dimensional vector D (x, y, z, r, x c ,y c ,z c ,x p ,y p ), where x, y, and z represent the position of the point cloud in three-dimensional space, r represents the laser radar reflection intensity, and x c ,y c , z c Indicates the distance from the point to the arithmetic mean of all points in the bar, x p ,y p Indicates the offset value from the point to the center of the point column; Set the number of point columns to P. The number of points in each point column is less than or equal to N. When the number of points in the point column is greater than N, use random sampling to select N points from the point column; otherwise, use 0 to fill the number of points in the point column to N. The points in the point cloud data are encoded into a tensor of (D, P, N); each point is processed through a linear layer, BatchNorm and ReLu to generate a tensor of dimension (C, P, N); a maximum pooling operation is performed on each point column to obtain a tensor of dimension (C, P), and a pseudo image is generated through a scatter operator; the generated pseudo image is input into 16 layers of stacked convolution for feature extraction, and three sets of different feature maps are output; finally, the three sets of different feature maps are sent to the detection head composed of three deconvolutions and two 1×1 convolutions, and the first target detection result is output.

6. The method for detecting safe targets of intelligent driving vehicles according to claim 1, characterized in that: Inputting the obstacle point cloud into a clustering algorithm to obtain a second target detection result includes: The obstacle point cloud is input into the DBSCAN algorithm for target detection. An arbitrary point g is selected and statistical judgment is performed on the points within the point g area ε. When the point density is greater than or equal to the threshold Minps, point g is recorded as a core point and a new category is created. Otherwise, point g is marked as a noise point. Other arbitrary points in the point cloud are selected and the above steps are repeated until all points are marked as processed. The category information point output in the cluster is the second target detection result.

7. The method for detecting safe targets of intelligent driving vehicles according to claim 1, characterized in that: The first target detection result and the second target detection result are combined for processing, including: setting a target confidence threshold, when neither the 3D point cloud target detection algorithm nor the clustering algorithm detects the target and the confidence does not exceed the target confidence threshold, rejecting the detection result; when both the 3D point cloud target detection algorithm and the clustering algorithm detect the target and exceed the target confidence threshold, accepting the detection result; wherein the detection result includes the first target detection result or the second target detection result; when either the 3D point cloud target detection algorithm or the clustering algorithm detects the target and exceeds the target confidence threshold, receiving the detection result of the algorithm.

Images