A method for unstructured road detection based on three-dimensional laser radar

Abstract

The application discloses a kind of unstructured road detection methods based on three-dimensional laser radar, it is characterized in that, including the following steps: taking original point cloud data generated by vehicle-mounted three-dimensional laser radar sensor, and using PCL library to remove invalid point in point cloud;According to the size of the detection area set by user, screen point cloud, remove the point outside the region, remove the point with elevation feature greater than a certain value in the detection area to extract ground point cloud;According to azimuth angle order, utilize the fusion detection boundary candidate point of space and intensity feature;According to radar scanning line order, utilize the fusion detection boundary candidate point of space and intensity feature;Using second order Bezier curve fitting drivable area boundary and output, form efficient, accurate detection result.

Description

Technical Field

[0001] This invention relates to the field of road detection technology, and in particular to an unstructured road detection method based on three-dimensional lidar. Background Technology

[0002] With the rapid development of artificial intelligence technology, autonomous driving technology is also being rapidly updated. To ensure the driving safety of intelligent vehicles, the perception system needs to classify everything around the vehicle into two categories for identification: static identification and dynamic identification. Static identification mainly includes static objects such as lane lines, curbs, and obstacles; dynamic identification mainly includes dynamic objects such as pedestrians and moving vehicles. The drivable area is one of the important research points in static identification, and it is widely used in advanced driver assistance systems as an important reference indicator to ensure driving safety. Unstructured roads are an important scenario that intelligent vehicles need to face during their popularization, such as rural dirt roads, park paths, and hilly fields. Due to their uneven surface, lack of clearly structured curbs, and irregular road boundaries, intelligent vehicles face significant safety hazards when driving on them, making the detection of drivable areas on unstructured roads particularly important.

[0003] 3D LiDAR, as a crucial sensor for autonomous driving, offers significant advantages over cameras in acquiring 3D information. Unlike passive sensors like cameras, 3D LiDAR actively emits and receives laser beams, exhibiting strong anti-interference capabilities and providing 3D spatial information about the vehicle's surroundings. It boasts advantages such as long range, wide detection angle, and high data accuracy. In recent years, with the continuous development of 3D LiDAR technology, its cost has decreased year by year, and 3D LiDAR sensors are now standard equipment in intelligent connected vehicles. Drivable area detection methods based on 3D LiDAR focus on maximizing the utilization of 3D point cloud data. By combining the missing 3D spatial information from a monocular camera with the unique laser reflection intensity information of the point cloud, the function of detecting drivable areas using only 3D LiDAR can be achieved. This type of method is less affected by environmental factors and does not require consideration of time synchronization issues between multiple sensor data.

[0004] While current drivable area detection methods based on 3D LiDAR can achieve detection functions in most structured road scenarios, they still have significant limitations in some unstructured road scenarios where spatial information about the boundaries is lacking. This poses a great threat to the driving safety of intelligent vehicles. Summary of the Invention

[0005] This application provides a method for detecting unstructured roads based on three-dimensional lidar, which solves the problem of inaccurate detection of unstructured roads in the prior art.

[0006] This application provides an embodiment of an unstructured road detection method based on three-dimensional lidar, characterized by the following steps:

[0007] Step 1: Obtain the raw point cloud data generated by the vehicle-mounted 3D LiDAR sensor and use the PCL library to remove invalid points in the point cloud;

[0008] Step 2: Filter the point cloud according to the user-set detection area size, remove points outside the area, and remove points in the detection area with elevation features greater than a certain value to extract the ground point cloud.

[0009] Step 3: Detect candidate boundary points by fusing spatial and intensity features in azimuth order;

[0010] Step 4: Detect candidate boundary points by fusing spatial and intensity features according to the radar scan line sequence;

[0011] Step 5: Fit the drivable region boundary using a second-order Bézier curve and output the result.

[0012] Preferably, the intensity feature fusion specifically involves fusing reflection intensity feature factors in a spatial feature detection algorithm to expand the candidate point detection dimension, thereby extracting various unstructured road drivable area edge candidate points.

[0013] Preferably, step 3 specifically includes: setting up a container for adjacent point sequences to clarify the radial adjacent point relationships; adding point cloud reflection intensity feature detection; and using simplified spatial feature detection.

[0014] Preferably, the step of setting the adjacent point sequence container to clarify the radial adjacent point relationship specifically includes:

[0015] Adjacent point containers cover all directions of the vehicle in a sliding window pattern. Let the initial container be R. 0 Then the remaining containers are calculated according to the following formula:

[0016] R k =T z (αk).D x (Ik)R 0

[0017] Where: R 0 Let T be the initial cuboid vertex matrix. z Let D be the homogeneous rotation transformation matrix, α be the rotation increment, and D be the rotation matrix. x Let R be a homogeneous translation matrix, l be the translation increment, k be the number of iterations, and R be the number of iterations. k Here is the vertex matrix of the cuboid after k iterations:

[0018]

[0019]

[0020] Fill the point cloud data P into R. k Obtain the set of interior points C′ k ={P|F(P,R)} k The algorithm formula is as follows: )=1}

[0021]

[0022]

[0023] in: For cuboid R k Vertex coordinates, P is the coordinates of a point, and v is the coordinates of a cuboid R. 0 The volume of the cuboid R, where w, l, and h are respectively... 0 Length, width, and height;

[0024] Let the set of interior points C′ k Transform into a set of cylindrical coordinate points C k ={p|p=E(p), p∈C′ k The transformation function E() is as follows:

[0025]

[0026] Set the cylindrical coordinate points C k The adjacent point sequence can be obtained by sorting by radial distance.

[0027] Preferably, the addition of point cloud reflection intensity feature detection specifically includes adding an intensity dimension to each point in the adjacent point sequence, using a sliding window for detection, calculating the average intensity within the sliding window, setting an intensity difference threshold, and if the intensity of a detected point exceeds the threshold range, it is judged as a candidate point and added to the candidate point set.

[0028] Preferably, the simplified spatial feature detection specifically includes the following steps: calculating the gradient value of each point in the adjacent point sequence, setting a gradient difference threshold, and if the gradient difference is greater than the threshold, it is judged as a candidate point and added to the candidate point set.

[0029] Preferably, step 4 specifically includes: fusing reflection intensity feature detection; and calculating dimensionality reduction space features.

[0030] Preferably, the fused reflection intensity feature detection specifically includes the following: Based on the characteristic that radar scan lines are circular, two sliding windows are set, placed before and after the detection point, respectively. The average intensity within each sliding window is calculated. If the difference between the intensity of the detection point and the average intensity of the preceding and following windows is greater than a threshold, it is judged as a candidate point and added to the candidate point set. Let the set of intensity circular points be P.r ′={p|p=[x,y,z,σ,intensity]T},preceding sliding window strength is w f The rear sliding window is w b Given a sliding window size n and an intensity difference threshold of τ², the candidate point judgment formula is as follows:

[0031] n r =card(P r ′)

[0032] W f ={(its r，i+1 its r，i+2 , ...,its r，i+n )|i+n<n r its r，i ∈P r}

[0033] W b ={(its r，i-n its r，i-n+1 , ...,its r，i-1 )|i>n, its r，i ∈P r}

[0034]

[0035]

[0036]

[0037] Where: n r The number of items in the circular point set. The average intensity of the preceding window. This represents the average intensity of the back window.

[0038] Preferably, the dimensionality reduction spatial feature calculation specifically includes sequentially selecting three adjacent points from the annular point set, reducing the dimensionality by setting x=0, so that they form a triangle in the yz plane, calculating the angle of the midpoint, and judging it as a candidate point if the angle is less than a threshold. Let the set of points for the triangle be N. r The angle of the midpoint is α r，i If the angle threshold is τ1, then the set of points to be selected is:

[0039] N r ={(q r，i-1 q r，i q r，i+1 )|q r，i =[y, z] T , i > 2, q r，i ∈P r}

[0040] The formula for calculating the midpoint angle and determining candidate edge points is as follows:

[0041]

[0042] S={p r，i |α r，i ≤τ1}

[0043] in: d r Let r be the scanning radius of the radar scan line, and γ be the horizontal angular resolution of the lidar.

[0044] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0045] 1. In this embodiment of the application, the method of fusion detection of spatial features and reflection intensity features is adopted to realize the detection and extraction of drivable areas in unstructured road scenes lacking obvious spatial features, thereby improving accuracy.

[0046] 2. This application uses elevation data to directly filter ground point clouds, takes cylindrical coordinate system azimuth and scan line direction to fuse spatial and intensity features and simultaneously retrieves boundary candidate points. There is no need to perform Hough transform to fit straight lines or filter. The extracted candidate points are fitted with Bézier curves, which better matches the tortuous and varied road morphology of unstructured roads and has stronger robustness. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the detection of drivable areas in this invention.

[0048] Figure 2 This is a schematic diagram of the azimuth angle detection method of the present invention.

[0049] Figure 3 This is a schematic diagram illustrating the azimuth angle detection data processing of the present invention.

[0050] Figure 4 This is a schematic diagram of the radar scan line detection method of the present invention.

[0051] Figure 5 This is a schematic diagram illustrating the detection based on the reflection intensity characteristics of radar scan lines according to the present invention.

[0052] Figure 6 This is a diagram showing the results of applying the present invention to unstructured roads in a park.

[0053] Figure 7 This is a diagram showing the results of applying the present invention to unstructured roads with curves.

[0054] Figure 8This is a diagram showing the results of applying the present invention to unstructured roads in parking lots.

[0055] Figure 9 This is a diagram showing the results of applying the present invention to a partially obscured structured road.

[0056] Figure 10 This invention compares the accuracy of four methods for detecting drivable areas. Detailed Implementation

[0057] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0058] Example 1

[0059] like Figure 1 As shown, the present invention relates to an unstructured road detection method based on three-dimensional lidar, which mainly includes the following steps:

[0060] S1: Acquire raw point cloud data from the vehicle-mounted 3D LiDAR sensor and remove invalid points. Acquire the raw point cloud data generated by the vehicle-mounted 3D LiDAR sensor using a network cable interface or wireless device, and use the PCL library to remove invalid points from the point cloud.

[0061] S2: Filter the point cloud according to the user-defined detection area size, removing points outside the area. Remove points in the detection area with elevation features greater than a certain value to extract the ground point cloud.

[0062] S3: Detecting candidate boundary points by fusing spatial and intensity features in azimuth order. A boundary feature point retrieval method using azimuth-ordered spatial and intensity feature fusion.

[0063] like Figure 2 As shown, the algorithm is initialized as follows: a cylindrical coordinate system is constructed with the vehicle-mounted radar as the origin and the vehicle's driving direction as the positive direction. A cuboid is set in the detection area as the initial container for the adjacent point sequence, and this cuboid is translated and rotated around the z-axis to cover all directions of the vehicle in a sliding window mode; as shown... Figure 3 As shown, ground point cloud data is filled into each divided cuboid and converted into a set of cylindrical coordinate points. The points are sorted by radial distance, and the sorted sequence of points within the same cuboid is set as an adjacent point sequence.

[0064] Let the initial cuboid be R. 0 After k iterations, a cuboid R is obtained. k R k Calculate using the following formula:

[0065] R k =T z ·D x R0

[0066] Wherein: T z Let D be the homogeneous rotation transformation matrix. x It is a homogeneous translation matrix.

[0067] Let the point cloud dataset be P = {p | p = [x, y, z]}. T}, fill them into each cuboid R k The interior point set C′ is formed. k ={P|F(P,R)} k )=1}. The function F() is the interior point judgment function, using the principle of conservation of the volume of a cuboid:

[0068]

[0069]

[0070] Where v = wlh; w, l, and h are the length, width, and height of the cuboid, respectively.

[0071] Let the set of interior points C′ k Transform into a set of cylindrical coordinate points C k ={p|p=E(o), p∈C′ k}, and sort them to obtain the sequence of adjacent points. Function E() is a cylindrical coordinate point transformation function:

[0072]

[0073]

[0074] S4: Detect boundary candidate points by fusing spatial and intensity features according to the radar scan line order.

[0075] Spatial and intensity features are fused for boundary candidate point detection. Spatial feature detection: The gradient value of each point in the adjacent point sequence is calculated, and a gradient difference threshold is set. If the gradient difference is greater than the threshold, it is judged as a candidate point and added to the candidate point set. Let R be a cuboid. k The gradient neighbor sequence in G is k If the set of candidate points is S and the gradient difference threshold is ε1, then the candidate point judgment steps are as follows:

[0076]

[0077]

[0078]

[0079] Intensity feature detection: An intensity dimension is added to each point in the adjacent point sequence. A sliding window detection is used to calculate the average intensity within the sliding window. An intensity difference threshold is set. If the intensity of a detected point exceeds the threshold range, it is judged as a candidate point and added to the candidate point set. Let R be a cuboid. k The intensity neighbor sequence is The sliding window is w k，i Given a window size of n, a candidate point set of S, and an intensity difference threshold of ε2, the candidate point judgment steps are as follows:

[0080]

[0081]

[0082]

[0083]

[0084] The specific steps in S4 are as follows: A boundary feature point retrieval method that fuses spatial and intensity features according to radar beam order is used. Point cloud data is classified according to radar scan lines, and candidate points are detected based on spatial and intensity features using a beam-based dimensionality reduction approach. Figure 4 As shown.

[0085] First, calculate the elevation angle of each point. Points with elevation angles within the same range form a ring-shaped set of points. Let the point cloud dataset be P = {p | p = [x, y, z]}. T The point cloud set on radar beam r is P. r If the vertical angular resolution of the lidar is τ0, then the conversion method is:

[0086]

[0087] Then, spatial and intensity features are fused for boundary candidate point detection. Spatial feature detection: Using dimensionality reduction, three adjacent points are selected sequentially from the set of bundle points, and x = 0 is set to form a triangle in the yz plane. The angle of the midpoint is calculated; if the angle is less than a threshold, it is considered a candidate point. Let N be the set of points selected for the triangle. r The angle of the midpoint is α r，i If the angle threshold is τ1, the candidate point judgment steps are as follows:

[0088] N r ={(q r，i-1 q r，i q r，i+1 )|q r，i =[y, z] T , i > 2, q r，i ∈P r}

[0089]

[0090] S={p r，i |α r，i ≤τ1}

[0091] in: d r Let r be the scanning radius of the radar scan line, and γ be the horizontal angular resolution of the lidar.

[0092] Intensity feature detection: Based on the characteristic that the detection set is circular, two sliding windows are set, one before and one after the detection point, as shown below. Figure 5 As shown. The average intensity within each sliding window is calculated. If the difference between the intensity of a detection point and the average intensity of the preceding and following windows is greater than a threshold, it is identified as a candidate point and added to the candidate point set. Let P be the set of intensity ring points. r ′={p|p=[x, y, z, σ, intensity] T The strength of the preceding sliding window is w. f The rear sliding window is w b Given a sliding window size n and an intensity difference threshold of τ², the candidate point judgment formula is as follows:

[0093] n r =card(P r ′)

[0094] W f ={(its r，i+1 its r，i+2 its r，i+n )|i+n<n r its r，i ∈P r}

[0095] W b ={(its r，i-n its r，i-n+1 , ...,its r，i-1 )|i>n, its r，i ∈P r}

[0096]

[0097]

[0098]

[0099] Where: n r Number of items in a circular point set The average intensity of the preceding window. This represents the average intensity of the post-window.

[0100] S5: Fit the drivable region boundary using a second-order Bézier curve and output the result.

[0101] Dividing the area into left and right sides, a second-order Bézier curve is used to select three adjacent points from the candidate point set as a group of control points to determine the boundary of a drivable area. Let the Bézier curve control point set be C, and the generated curve be B. The curve is then calculated using the following formula:

[0102] C>{p i |p i >[x, y] T p i ∈S}

[0103] B(t)=(1-t)p i-1 +2t(1-t)p i +t 2 p i+1 t∈[0,1],p i ∈C

[0104] Where t is Increasing upwards, This is a set of control points.

[0105] By stitching together the curves generated by each group of control points in the point set, the final drivable area boundary can be obtained.

[0106] This invention's method is primarily applied to the real-time drivable area perception of unstructured roads by intelligent driving vehicles. The experimental platform is a low-speed unmanned vehicle equipped with a RoboSense RS-LiDAR-16 lidar, four RGB cameras, and a GNSS integrated navigation system. The RS-LiDAR-16 lidar used in the experimental test is horizontally mounted on the top of the unmanned vehicle at a height of 1.4 meters, with a rotation frequency of 10 Hz. The radar coordinate system has its origin at the radar center, with the y-axis representing the vehicle's forward direction, the x-axis parallel to the ground to the right, and the z-axis perpendicular to the ground upwards.

[0107] To verify the method of this invention, experimental analysis was conducted on three datasets. Dataset 1 is a dataset of 2395 frames of park scenes collected using an unmanned vehicle and recorded by a 16-line LiDAR. The scene consists of sidewalks and lanes, and is mainly unstructured roads. Dataset 2 is a dataset of 2395 frames of scenes from the China Intelligent Vehicle R&D and Testing Center campus collected using an unmanned vehicle and recorded by a 16-line LiDAR. The recorded scene is relatively enclosed and consists of unstructured roads. Dataset 3 is a dataset of 2541 frames from the KITTI autonomous driving public dataset, recorded by a 64-line LiDAR. The scene mainly includes parking lots and highways, and consists of most structured roads and a small portion of unstructured roads.

[0108] The method of this invention was developed using a Linux and ROS architecture and implemented in C++. The computer used an AMD Ryzen 7 5800H processor with a 3.2GHz clock speed and 16GB of RAM. The method achieved an average frame time of 18.47ms on the 64-line KITTI public dataset, and average frame times of 8.37ms and 10.31ms on two 16-line real-world road datasets, respectively, fully meeting the 100ms scanning frequency requirement of 3D LiDAR and demonstrating excellent real-time performance.

[0109] like Figure 6 , Figure 7 , Figure 8 , Figure 9 As shown, the method of this invention performs well on three datasets. The left side represents actual road images; the right side represents the detection results of the method of this invention, where white dots indicate the detected boundary candidate points, and lines indicate the drivable area detection results. Figure 6 As shown, this is a typical scene in the unstructured road data set of a park. As can be seen from the actual road images, the carriageway and sidewalk are separated only by smooth-surfaced bricks, and the segmentation boundary lacks spatial features. The method of this invention can accurately identify the lanes. Figure 7 As shown, this is a scene of road turns in a park dataset. The method of this invention uses dual detection directions to accurately retrieve candidate points and has good robustness to various road shapes. Figure 8 As shown, this is a parking lot entrance / exit in a park dataset. The road width narrows, and the left side of the scene is defined by low grass, while the right side boundary contracts inwards without obvious spatial features. The method of this invention can extract candidate points using intensity features and has good robustness to widened road sections. Figure 9 As shown, this is a partially occluded structured road scene in the KITTI autonomous driving public dataset. This scene has two characteristics: the boundary has obvious spatial features and some boundaries are blocked by vehicles. The method of this invention can effectively utilize spatial features to extract candidate points and use Bézier curves to fit the blocked boundaries.

[0110] This invention selects three commonly used drivable area detection algorithms as control algorithms and conducts comparative experiments on three datasets: For each frame in the three datasets, correctly drivable areas are manually labeled. A correctly drivable area refers to the area around the intelligent vehicle where driving is permitted, i.e., a region where a driving route can be planned. If the overlap rate between the drivable area generated by the algorithm and the labeled area reaches 90%, the frame is considered to have been correctly detected. The accuracy of each algorithm is reflected by statistically analyzing the ratio of frames with correctly detected drivable areas to the number of labeled frames. The recognition accuracy of each algorithm is shown below. Figure 10 As shown.

[0111] The accuracy comparison in the table shows that the method of the present invention has a significantly higher detection accuracy than other detection methods, meeting the current demand for detection of drivable areas in unstructured road scenarios.

[0112] The embodiments described herein are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

[0113] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for detecting unstructured roads based on three-dimensional lidar, characterized in that, Includes the following steps: Step 1: Obtain the raw point cloud data generated by the vehicle-mounted 3D LiDAR sensor and use the PCL library to remove invalid points in the point cloud; Step 2: Filter the point cloud according to the user-set detection area size, remove points outside the area, and remove points in the detection area with elevation features greater than a certain value to extract the ground point cloud. Step 3: Set a cuboid as the initial adjacent point sequence container in the detection area, and translate and rotate the cuboid around the z-axis to cover all directions of the vehicle in a sliding window mode; detect boundary candidate points by fusing spatial and intensity features according to azimuth angle; set an adjacent point sequence container to clarify the radial adjacent point relationship; add point cloud reflection intensity feature detection; use simplified spatial feature detection. Step 4: Detect candidate boundary points by fusing spatial and intensity features according to the radar scan line sequence; Step 5: Fit the drivable region boundary using a second-order Bézier curve and output the result.

2. The unstructured road detection method based on three-dimensional lidar as described in claim 1 is characterized in that, The intensity feature fusion specifically refers to fusing reflection intensity feature factors in a spatial feature detection algorithm to expand the detection dimension of candidate points, thereby extracting various unstructured road drivable area edge candidate points.

3. The unstructured road detection method based on three-dimensional lidar as described in claim 1 is characterized in that... The step of setting up an adjacent point sequence container to clarify the radial adjacent point relationship specifically includes: Adjacent point containers cover all directions of the vehicle in a sliding window pattern. Let the initial container be... Then the remaining containers are calculated according to the following formula: in: Let the initial cuboid vertex matrix be... Let be the homogeneous rotation transformation matrix. For rotation increments, It is a homogeneous translation matrix. For translation increment, For the number of iterations, For the The cuboid vertex matrix after the next iteration transformation: ， ， ； Point cloud data Fill in Obtain the set of interior points The algorithm formula is as follows: ； in: cuboid Vertex coordinates, P is the coordinates of a point. cuboid volume, They are cuboids Length, width, and height; Set of interior points Convert to a set of cylindrical coordinate points Transformation function as follows: ； Set of cylindrical coordinate points The adjacent point sequence can be obtained by sorting by radial distance. .

4. The unstructured road detection method based on three-dimensional lidar as described in claim 1 is characterized in that, The addition of point cloud reflection intensity feature detection specifically includes adding an intensity dimension to each point in the adjacent point sequence, using a sliding window for detection, calculating the average intensity within the sliding window, setting an intensity difference threshold, and if the intensity of a detected point exceeds the threshold range, it is judged as a candidate point and added to the candidate point set.

5. The unstructured road detection method based on three-dimensional lidar as described in claim 1 is characterized in that, The simplified spatial feature detection method specifically includes the following steps: calculating the gradient value of each point in the adjacent point sequence, setting a gradient difference threshold, and if the gradient difference is greater than the threshold, it is judged as a candidate point and added to the candidate point set.

6. The unstructured road detection method based on three-dimensional lidar as described in claim 1 is characterized in that, Step 4 specifically includes, Fusion reflection intensity feature detection; dimensionality reduction space feature calculation.

7. The unstructured road detection method based on three-dimensional lidar as described in claim 6 is characterized in that, The fusion reflection intensity feature detection specifically includes the following feature: based on the characteristic that the radar scan line is a ring, two sliding windows are set and placed before and after the detection point, respectively. Calculate the average intensity within each sliding window. If the difference between the intensity of a detection point and the average intensity of the preceding and following windows is greater than a threshold, it is identified as a candidate point and added to the candidate point set. Let the intensity ring point set be... The strength of the preceding sliding window is The rear sliding window is Sliding window size The intensity difference threshold is The candidate point judgment formula is as follows: in: The number of items in the circular point set. The average intensity of the preceding window. This represents the average intensity of the back window.

8. The unstructured road detection method based on three-dimensional lidar as described in claim 7 is characterized in that, The dimensionality reduction spatial feature calculation specifically includes selecting three adjacent points sequentially from the set of annular points, and reducing the dimensionality by... So that it can be in The plane forms a triangle. The angle of the midpoint is calculated; if the angle is less than a threshold, it is considered a candidate point. Let the set of points for the triangle be... The angle of the midpoint is The angle threshold is Then the set of points is: The formula for calculating the midpoint angle and determining candidate edge points is as follows: in: , For radar scan lines The scanning radius, This refers to the horizontal angular resolution of the lidar.

Images