Lane detection methods, devices, vehicles and storage media

Abstract

This application relates to the field of vehicle technology, and in particular to a lane line detection method, apparatus, vehicle, and storage medium. The method includes: acquiring three-dimensional point cloud data around the vehicle; identifying a region of interest (ROI) in the three-dimensional point cloud data; segmenting the ROI along the vehicle's driving direction to obtain multiple sub-regions; fitting the point cloud data of each sub-region to a ground plane to obtain ground point cloud data; and identifying the ground point cloud data to obtain lane lines for the road surrounding the vehicle. This solves the problems in related technologies where lane line detection is based on an ideal flat ground surface, making it impossible to guarantee the accuracy and robustness of lane line detection in various complex road environments.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a lane line detection method, device, vehicle, and storage medium. Background Technology

[0002] Environmental perception systems are a crucial component of autonomous driving technology, ensuring the safety and intelligence of autonomous vehicles. Because 3D LiDAR can provide rich 3D road data in real time, with a wider scanning range and higher detection accuracy, it is gradually being applied to lane detection.

[0003] In related technologies, after extracting ground points, the SVM (Support Vector Machine) algorithm is usually used to classify lane lines and ground point data, and finally lane line information is obtained by fitting the lane line points.

[0004] However, the relevant technologies are designed for ideal flat surfaces, and the height threshold method and normal vector method cannot cope with ground conditions such as slopes and drainage protrusions, resulting in poor subsequent lane line detection performance. Summary of the Invention

[0005] This application provides a lane line detection method, device, vehicle, and storage medium to solve the problems in related technologies where lane line detection based on an ideal flat ground cannot guarantee the accuracy and robustness of lane line detection in various complex road environments.

[0006] The first aspect of this application provides a lane line detection method, comprising the following steps: acquiring three-dimensional point cloud data around a vehicle; identifying a region of interest in the three-dimensional point cloud data, segmenting the region of interest along the driving direction of the vehicle to obtain multiple sub-regions, and fitting the point cloud data of each sub-region to a ground plane to obtain ground point cloud data; and identifying the ground point cloud data to obtain lane lines of the road around the vehicle.

[0007] Optionally, in one embodiment of this application, the step of fitting the point cloud data of each sub-region to a ground plane to obtain ground point cloud data includes: extracting a seed point set of the point cloud data of each sub-region; establishing a plane model based on the seed point set; fitting the plane model to obtain a ground plane; calculating the orthogonal projection distance of the point cloud data of each sub-region to the ground plane; if the orthogonal projection distance is less than a distance threshold, determining that the corresponding point cloud data belongs to the ground plane; otherwise, determining that the corresponding point cloud data does not belong to the ground plane; and filtering the point cloud data belonging to the ground plane to obtain the ground point cloud data.

[0008] Optionally, in one embodiment of this application, the step of extracting the seed point set of the point cloud data of each sub-region includes: sorting the point cloud data of each sub-region according to the point cloud height to obtain sorted point cloud data; identifying the lowest point representative and multiple target height points in the sorted point cloud data, and determining an extraction threshold based on the average of the lowest point representative and the multiple target height points; taking the point cloud data in the sorted point cloud data that is less than or equal to the extraction threshold as seed points, and generating the seed point set based on the extracted seed points.

[0009] Optionally, in one embodiment of this application, identifying the ground point cloud data to obtain the lane lines of the road surrounding the vehicle includes: classifying the ground point cloud data into lane line point data and non-lane line point data according to the scan lines; converting the reflection intensity values ​​of the data within the scan lines into grayscale values; calculating an optimal segmentation threshold based on the respective probabilities of the lane line point data and the non-lane line point data, and the grayscale values; filtering data whose grayscale values ​​within the scan lines are greater than the optimal segmentation threshold as lane line candidate point cloud data; and using the lane line candidate point cloud data to identify the lane lines of the road surrounding the vehicle.

[0010] Optionally, in one embodiment of this application, the step of calculating the optimal segmentation threshold based on the probabilities of the lane line point data and the non-lane line point data, and the grayscale value, includes: calculating a secondary grayscale mean of the grayscale value based on grayscale values ​​greater than the mean of the grayscale value, using the secondary grayscale mean as an initial threshold to determine a threshold selection interval; calculating a first mean and a second mean corresponding to the probabilities of the lane line point data and the non-lane line point data respectively, calculating the inter-class variance based on the first mean, the second mean, and the mean of grayscale values ​​greater than any threshold in the threshold selection interval; and using the threshold that makes the inter-class variance reach its maximum value as the optimal segmentation threshold.

[0011] Optionally, in one embodiment of this application, the step of identifying lane lines of the road surrounding the vehicle using the lane line candidate point cloud data includes: identifying the lane line direction of the vehicle; performing density clustering on the lane line candidate point cloud data based on the lane line direction to obtain clustering results; and using the clustering results to perform lane line fitting to obtain the lane lines of the road surrounding the vehicle.

[0012] Optionally, in one embodiment of this application, identifying the region of interest in the three-dimensional point cloud data includes: filtering out the background data of the three-dimensional point cloud data to obtain the region of interest.

[0013] A second aspect of this application provides a lane line detection device, comprising: an acquisition module for acquiring three-dimensional point cloud data around a vehicle; a first identification module for identifying a region of interest in the three-dimensional point cloud data, segmenting the region of interest along the driving direction of the vehicle to obtain multiple sub-regions, and fitting the point cloud data of each sub-region to a ground plane to obtain ground point cloud data; and a second identification module for identifying the ground point cloud data to obtain lane lines of the road around the vehicle.

[0014] Optionally, in one embodiment of this application, the first identification module is further configured to extract a seed point set of the point cloud data of each sub-region; establish a planar model based on the seed point set; fit the planar model to obtain a ground plane; calculate the orthogonal projection distance from the point cloud data of each sub-region to the ground plane; if the orthogonal projection distance is less than a distance threshold, determine that the corresponding point cloud data belongs to the ground plane; otherwise, determine that the corresponding point cloud data does not belong to the ground plane; and filter the point cloud data belonging to the ground plane to obtain the ground point cloud data.

[0015] Optionally, in one embodiment of this application, the first identification module is further configured to sort the point cloud data of each sub-region according to the point cloud height to obtain sorted point cloud data; identify the lowest point representative and multiple target height points in the sorted point cloud data, determine an extraction threshold based on the average of the lowest point representative and the multiple target height points; take the point cloud data in the sorted point cloud data that is less than or equal to the extraction threshold as seed points, and generate the seed point set based on the extracted seed points.

[0016] Optionally, in one embodiment of this application, the second identification module is further configured to classify the ground point cloud data into lane line point data and non-lane line point data according to the scan lines; convert the reflection intensity values ​​of the data within the scan lines into grayscale values; calculate the optimal segmentation threshold based on the respective probabilities of the lane line point data and the non-lane line point data, and the grayscale values; filter the data within the scan lines whose grayscale values ​​are greater than the optimal segmentation threshold as lane line candidate point cloud data; and use the lane line candidate point cloud data to identify the lane lines of the road surrounding the vehicle.

[0017] Optionally, in one embodiment of this application, the second identification module is further configured to calculate a secondary grayscale mean of the grayscale value based on the grayscale value greater than the average grayscale value, and use the secondary grayscale mean as an initial threshold to determine a threshold selection interval; calculate the first mean and the second mean corresponding to the probabilities of the lane line point data and the non-lane line point data respectively, calculate the inter-class variance based on the first mean, the second mean, and the mean of grayscale values ​​greater than any threshold in the threshold selection interval; and use the threshold that makes the inter-class variance reach its maximum value as the optimal segmentation threshold.

[0018] Optionally, in one embodiment of this application, the second identification module is further configured to identify the lane line direction of the vehicle; perform density clustering on the candidate point cloud data of the lane line based on the lane line direction to obtain the clustering result, and use the clustering result to perform lane line fitting to obtain the lane lines of the road surrounding the vehicle.

[0019] Optionally, in one embodiment of this application, the first identification module is further configured to filter out background data of the three-dimensional point cloud data to obtain the region of interest.

[0020] A third aspect of this application provides a vehicle, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the lane line detection method as described in the above embodiments.

[0021] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the lane line detection method as described in the above embodiments.

[0022] Therefore, this application has at least the following beneficial effects:

[0023] By identifying the region of interest (ROI) in the 3D point cloud data surrounding the vehicle, and segmenting the ROI along the vehicle's direction of travel to obtain multiple sub-regions, the point cloud data of each sub-region is fitted to the ground plane to ensure that as much ground point cloud data as possible is acquired. This approach can also handle ground conditions with varying slopes. Furthermore, by identifying the ground point cloud data, lane lines around the vehicle are obtained, further improving lane line detection accuracy and efficiency. This solves the problem in related technologies where lane line detection is based on an ideal flat ground surface, which cannot guarantee accuracy and robustness in various complex road environments.

[0024] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. Attached Figure Description

[0025] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0026] Figure 1 This is a flowchart of a lane line detection method provided according to an embodiment of this application;

[0027] Figure 2 This is a flowchart of a ground plane fitting algorithm provided according to an embodiment of this application;

[0028] Figure 3 This is a flowchart illustrating the process of filtering lane line candidate point data according to an embodiment of this application.

[0029] Figure 4 This is a schematic diagram of a lane detection method based on three-dimensional LiDAR according to an embodiment of this application;

[0030] Figure 5 This is a block diagram of a lane line detection device according to an embodiment of this application;

[0031] Figure 6 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0032] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0033] The lane line detection method, apparatus, vehicle, and storage medium of this application are described below with reference to the accompanying drawings. Addressing the problems mentioned in the background art, this application provides a lane line detection method. In this method, by identifying the region of interest (ROI) in the three-dimensional point cloud data surrounding the vehicle, the ROI is segmented along the vehicle's driving direction to obtain multiple sub-regions. The point cloud data of each sub-region is fitted to a ground plane to ensure that as much ground point cloud data as possible is obtained. Furthermore, this method can handle ground conditions with certain slope variations. By identifying the ground point cloud data, the lane lines of the road surrounding the vehicle are obtained, further improving the lane line detection accuracy and efficiency. Therefore, this solves the problems in related technologies where lane line detection is based on an ideal flat ground, which cannot guarantee the accuracy and robustness of lane line detection in various complex road environments.

[0034] Specifically, Figure 1 This is a schematic flowchart of a lane line detection method provided in an embodiment of this application.

[0035] like Figure 1 As shown, the lane line detection method includes the following steps:

[0036] In step S101, three-dimensional point cloud data around the vehicle is acquired.

[0037] This embodiment of the application utilizes an onboard 3D LiDAR to perform laser scanning of the road environment, acquiring 3D point cloud data of the vehicle's surroundings. This data encompasses all environmental information within the surrounding traffic scene, including buildings, trees, vehicles, pedestrians, streetlights, vegetation, and the ground. The data for each laser point acquired by the LiDAR includes: Cartesian coordinates (x, y, z), laser reflection intensity, timestamp, and distance between the point and the LiDAR. In practical implementation, this embodiment uses a 16-line LiDAR sensor to acquire a 3D laser point cloud dataset of the surrounding road environment, mounted on the roof of the vehicle at a height of 1.7 meters.

[0038] In step S102, the region of interest in the three-dimensional point cloud data is identified, the region of interest is segmented along the vehicle's driving direction to obtain multiple sub-regions, and the point cloud data of each sub-region is fitted to the ground plane to obtain ground point cloud data.

[0039] It is understandable that, due to the massive amount of 3D point cloud data and the presence of numerous interference points, which severely impacts detection efficiency and accuracy, this application embodiment can utilize a pass-through filter to filter out background data. This can more effectively separate normal ground points from other foreground and background elements, segmenting the region of interest and ensuring that the lidar acquires as much ground point cloud data as possible. Furthermore, it can handle ground conditions with varying slopes. The X-axis range can be set to [-80m, 80m], and the Y-axis range to [-10m, 10m].

[0040] Furthermore, embodiments of this application can extract ground point cloud data from the segmented region of interest. Specifically, the real ground is not a perfect plane. Due to the presence of slope, the lidar will introduce a large amount of measurement noise at large distances, easily mistaking slopes for non-ground surfaces, thus affecting the subsequent detection of lane lines. In order to better detect and effectively extract ground point cloud data from the region of interest, embodiments of this application can employ methods such as... Figure 2 The ground detection method shown improves the ground detection steps by dividing the space into several sub-planes along the x-direction (the direction of the vehicle's front), and then using a ground plane fitting algorithm on each sub-plane to finally detect the ground cloud data.

[0041] In one embodiment of this application, the point cloud data of each sub-region is fitted to a ground plane to obtain ground point cloud data, including: extracting a seed point set of the point cloud data of each sub-region; establishing a plane model based on the seed point set; fitting the plane model to obtain a ground plane; calculating the orthogonal projection distance from the point cloud data of each sub-region to the ground plane; if the orthogonal projection distance is less than a distance threshold, it is determined that the corresponding point cloud data belongs to the ground plane; otherwise, it is determined that the corresponding point cloud data does not belong to the ground plane; and filtering the point cloud data that belong to the ground plane to obtain ground point cloud data.

[0042] Specifically, such as Figure 2 As shown, in this embodiment of the application, three-dimensional point cloud data P is input, and outliers are filtered out according to the density distribution to obtain P*. Seed point set P is then extracted for each sub-region's point cloud data. seeds The planar model is built based on the seed point set to fit the ground plane. The formula is as follows:

[0043] ax + by + cz + d = 0

[0044] Right now:

[0045] n T X = -d

[0046] Where n = [a, b, c] T X = [x, y, z] T Using the seed point set S∈R 3 As an initial point set, its covariance matrix C is calculated using the following formula:

[0047]

[0048] in, Let represent the mean of all points, and C represent the distribution of the seed point set. Since it is a planar model, the embodiments of this application can obtain the normal vector n (i.e., a, b, c) perpendicular to the plane by calculating the singular vector with the minimum singular value. By substituting... We directly obtain d to get the planar model estimate, and calculate the orthogonal projection distance from each point in the point cloud to the ground plane. When the orthogonal projection distance is less than the planar distance threshold Th... dist If the condition is met, the point is considered to belong to the ground; otherwise, it is considered to be non-ground. Finally, the ground plane is detected and extracted to obtain ground point cloud data. Among these, Th... dist The plane distance threshold is used to determine whether a point belongs to the ground.

[0049] In one embodiment of this application, extracting a seed point set for the point cloud data of each sub-region includes: sorting the point cloud data of each sub-region according to the point cloud height to obtain sorted point cloud data; identifying the lowest point representative and multiple target height points in the sorted point cloud data, and determining an extraction threshold based on the average of the lowest point representative and multiple target height points; using the point cloud data in the sorted point cloud data that is less than or equal to the extraction threshold as seed points, and generating a seed point set based on the extracted seed points.

[0050] It is understood that, in the embodiments of this application, the point cloud data of each sub-region can be sorted according to the point cloud height (i.e., z-value) to obtain P. sorted The Lowest Point Representative (LPR) is introduced, which refers to N. LPR The average of the lowest elevation points is used to ensure that the plane fitting stage is not affected by measurement noise. The elevation in the 3D point cloud data P is within the threshold LPR+Th. seed Points within the range are considered as seed points, and these points form a set of seed points, which are used to build a planar model describing the ground.

[0051] Where, N LPR Indicates the number of lowest elevation points selected for the LPR; Th seed This represents the threshold for selecting seed points, which can be set according to the actual situation. It represents the ground screening conditions without specific limitations.

[0052] In step S103, ground point cloud data is identified to obtain lane lines of the road around the vehicle.

[0053] In one embodiment of this application, identifying ground point cloud data to obtain lane lines of the road surrounding a vehicle includes: classifying the ground point cloud data into lane line point data and non-lane line point data according to scan lines; converting the reflection intensity values ​​of the data within the scan lines into grayscale values; calculating the optimal segmentation threshold based on the respective probabilities of lane line point data and non-lane line point data, and the grayscale values; selecting data within the scan lines whose grayscale values ​​are greater than the optimal segmentation threshold as candidate lane line point cloud data; and using the candidate lane line point cloud data to identify lane lines of the road surrounding the vehicle.

[0054] This application embodiment can filter lane line feature candidate points from ground point cloud data, that is, separate lane line candidate points from ground point cloud data. The reflection intensity of laser light is greatly affected by the material of the irradiated object. Compared to asphalt, the average reflection intensity value of laser scanning on lane lines is significantly higher. Therefore, the reflection intensity of the lidar is selected as an effective feature to distinguish lane lines from the road surface. In actual implementation, even for the same object, the range of echo intensity varies greatly due to different distances and angles between it and the lidar. Directly setting an intensity threshold makes it difficult to filter out lane line candidate points.

[0055] To address the issue of large fluctuations in echo intensity in lidar point cloud data affecting threshold selection, such as... Figure 3 As shown, in this embodiment, an adaptive threshold can be used to extract feature points. The ground point cloud data is classified into lane line point data and non-lane line point data according to the scan lines. The reflection intensity values ​​within the scan lines are then converted to grayscale to obtain grayscale values ​​G. value The optimal segmentation threshold is calculated based on the probabilities of lane line point data and non-lane line point data and their gray values. The gray values ​​of the data within the scan line are then filtered, and data with gray values ​​greater than the optimal threshold are marked as lane line candidate point cloud data, while data with gray values ​​less than the optimal threshold are marked as road surface point cloud data.

[0056] In one embodiment of this application, the optimal segmentation threshold is calculated based on the probabilities of lane line point data and non-lane line point data, as well as grayscale values. This includes: calculating a secondary grayscale mean based on grayscale values ​​greater than the average grayscale value, using the secondary grayscale mean as an initial threshold to determine the threshold selection range; calculating a first mean and a second mean corresponding to the probabilities of lane line point data and non-lane line point data respectively, calculating the inter-class variance based on the first mean, the second mean, and the mean of grayscale values ​​greater than any threshold in the threshold selection range; and using the threshold that maximizes the inter-class variance as the optimal segmentation threshold.

[0057] The embodiments of this application can calculate the global mean G of gray values ​​within the scan line. ave Find all gray values ​​greater than the global mean, count their number (denoted as num), and obtain the secondary gray mean G. ave0 The calculation formula is as follows:

[0058]

[0059] In this embodiment, the reflection intensity of lane lines is significantly higher than that of ground data. After grayscale conversion of the 3D ground point cloud data, the grayscale values ​​of the lane line data are mostly concentrated in the high grayscale values. If the threshold starts from 0, it will waste computing resources and lead to longer detection time. Moreover, the target lane line data accounts for a very small proportion compared to the road surface data. Therefore, in this embodiment, the intra-class secondary grayscale mean G can be used. ave0 As the initial threshold, the threshold selection interval is now narrowed down to [G]. ave0

[255] This avoids the problem of poor lane line candidate point selection due to the inability to take into account the situation in various parts of the point cloud map when using a single global threshold.

[0060] Furthermore, in this embodiment, the probabilities P1 and P2, mean values ​​μ1 and μ2 of lane line point data and non-lane line point data are calculated respectively, along with the gray values ​​greater than G. ave0global mean μ G Finally, the inter-class variance σ is obtained. 2 The optimal threshold th that maximizes the intra-class variance is calculated using the following formula:

[0061] σ 2 =P1(μ1-μ G ) 2 +P2(μ2-μ G ) 2

[0062] In one embodiment of this application, lane lines of the road surrounding a vehicle are identified using lane line candidate point cloud data, including: identifying the lane line direction of the vehicle; performing density clustering on the lane line candidate point cloud data based on the lane line direction to obtain clustering results; and using the clustering results to fit lane lines to obtain the lane lines of the road surrounding the vehicle.

[0063] In the candidate feature point set of lane lines obtained after feature extraction, there will inevitably be a small amount of interference data. This noise data may be similar to the reflection intensity features of lane lines, such as road direction signs, pedestrian crossings and parking signs, or lane lines may be obscured by obstacles such as vehicles. These special road environments all bring difficulties to lane detection.

[0064] Therefore, the embodiments of this application can combine the feature methods of point density and lane marking direction continuity. Based on the lane line direction, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) density clustering is used to aggregate lane line candidate feature points into different classes and remove discrete noise points. Then, the least squares method is used to fit the lane line to obtain the final lane line contour, thereby further improving the lane line detection accuracy and detection efficiency.

[0065] LiDAR-based lane detection methods focus on maximizing the utilization of LiDAR point cloud data. The following section combines... Figure 4 The lane line detection method of this application embodiment will be described in detail, including the following steps:

[0066] Step 1: Use an onboard 3D LiDAR to perform a laser scan of the road environment and obtain 3D point cloud data around the vehicle;

[0067] Step 2: Preprocess the 3D point cloud data to segment the region of interest containing the ground, i.e., set the range of the X-axis and Y-axis, and remove invalid background data;

[0068] Step 3: Extract ground point cloud data from the segmented region of interest;

[0069] Step 4: Select lane line feature clustering candidate points from the ground point cloud data, that is, separate the lane line candidate points from the ground point cloud data;

[0070] Step 5: Combining the density of points with the continuous feature method of lane marking direction, cluster the coarsely extracted lane line candidate points for noise reduction, and then use the least squares method to fit the lane lines to obtain the final lane line contour.

[0071] The lane detection method proposed in this application identifies the region of interest (ROI) in the 3D point cloud data surrounding the vehicle, segments the ROI along the vehicle's driving direction to obtain multiple sub-regions, and fits the point cloud data of each sub-region to the ground plane, ensuring that as much ground point cloud data as possible is obtained. It can also handle ground conditions with certain slope variations. By identifying the ground point cloud data, the lane lines of the road surrounding the vehicle are obtained, further improving the accuracy and efficiency of lane detection. This solves the problems in related technologies where lane detection is based on an ideal flat ground, which cannot guarantee the accuracy and robustness of lane detection in various complex road environments.

[0072] Next, referring to the accompanying drawings, a lane line detection device according to an embodiment of this application is described.

[0073] Figure 5 This is a block diagram of a lane line detection device according to an embodiment of this application.

[0074] like Figure 5 As shown, the lane line detection device 10 includes: an acquisition module 100, a first identification module 200, and a second identification module 300.

[0075] The acquisition module 100 is used to acquire three-dimensional point cloud data around the vehicle; the first recognition module 200 is used to identify the region of interest in the three-dimensional point cloud data, segment the region of interest along the vehicle's driving direction to obtain multiple sub-regions, and fit the point cloud data of each sub-region to the ground plane to obtain ground point cloud data; the second recognition module 300 is used to recognize the ground point cloud data to obtain the lane lines of the road around the vehicle.

[0076] In one embodiment of this application, the first identification module 100 is further used to extract a seed point set of point cloud data for each sub-region; establish a plane model based on the seed point set; use the plane model to fit the ground plane; calculate the orthogonal projection distance from the point cloud data of each sub-region to the ground plane; if the orthogonal projection distance is less than a distance threshold, it is determined that the corresponding point cloud data belongs to the ground plane; otherwise, it is determined that the corresponding point cloud data does not belong to the ground plane; and the point cloud data belonging to the ground plane is filtered to obtain the ground point cloud data.

[0077] In one embodiment of this application, the first identification module 200 is further configured to sort the point cloud data of each sub-region according to the point cloud height to obtain sorted point cloud data; identify the lowest point representative and multiple target height points in the sorted point cloud data, determine the extraction threshold based on the average of the lowest point representative and multiple target height points; take the point cloud data in the sorted point cloud data that is less than or equal to the extraction threshold as seed points, and generate a seed point set based on the extracted seed points.

[0078] In one embodiment of this application, the second identification module 300 is further configured to classify the ground point cloud data into lane line point data and non-lane line point data according to the scan lines; convert the reflection intensity values ​​of the data within the scan lines into grayscale values; calculate the optimal segmentation threshold based on the respective probabilities of lane line point data and non-lane line point data, and the grayscale values; filter the data within the scan lines whose grayscale values ​​are greater than the optimal segmentation threshold as lane line candidate point cloud data; and use the lane line candidate point cloud data to identify the lane lines of the road surrounding the vehicle.

[0079] In one embodiment of this application, the second recognition module 300 is further configured to calculate a secondary grayscale mean based on grayscale values ​​greater than the average grayscale value, and use the secondary grayscale mean as an initial threshold to determine the threshold selection interval; calculate the first mean and the second mean corresponding to the probabilities of lane line point data and non-lane line point data respectively, calculate the inter-class variance based on the first mean, the second mean and the mean of grayscale values ​​greater than any threshold in the threshold selection interval; and use the threshold that makes the inter-class variance reach its maximum value as the optimal segmentation threshold.

[0080] In one embodiment of this application, the second identification module 300 is further used to identify the lane line direction of the vehicle; perform density clustering on the candidate point cloud data of the lane line based on the lane line direction to obtain the clustering result, and use the clustering result to fit the lane line to obtain the lane line of the road around the vehicle.

[0081] In one embodiment of this application, the first identification module 200 is further used to filter out background data of the three-dimensional point cloud data to obtain the region of interest.

[0082] It should be noted that the foregoing explanation of the lane line detection method embodiment also applies to the lane line detection device of this embodiment, and will not be repeated here.

[0083] The lane detection device proposed in this application identifies the region of interest (ROI) in the 3D point cloud data surrounding the vehicle, segments the ROI along the vehicle's driving direction to obtain multiple sub-regions, and fits the point cloud data of each sub-region to the ground plane, ensuring that as much ground point cloud data as possible is acquired. It can also handle ground conditions with certain slope variations. By identifying the ground point cloud data, the lane lines of the road surrounding the vehicle are obtained, further improving the lane detection accuracy and efficiency. This solves the problems in related technologies where lane detection is based on an ideal flat ground, making it impossible to guarantee the accuracy and robustness of lane detection in various complex road environments.

[0084] Figure 6 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0085] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0086] When the processor 602 executes the program, it implements the lane line detection method provided in the above embodiments.

[0087] Furthermore, the vehicle also includes:

[0088] Communication interface 603 is used for communication between memory 601 and processor 602.

[0089] The memory 601 is used to store computer programs that can run on the processor 602.

[0090] The memory 601 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0091] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0092] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0093] The processor 602 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0094] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the lane line detection method described above.

[0095] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0096] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0097] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0098] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0099] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0100] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A lane line detection method, characterized in that, Includes the following steps: Acquire 3D point cloud data around the vehicle; Identify the region of interest in the 3D point cloud data, segment the region of interest along the vehicle's driving direction to obtain multiple sub-regions, and fit the point cloud data of each sub-region to the ground plane to obtain ground point cloud data; Identify the ground point cloud data to obtain the lane lines of the road surrounding the vehicle; The step of identifying the ground point cloud data to obtain the lane lines of the road surrounding the vehicle includes: classifying the ground point cloud data into lane line point data and non-lane line point data according to the scan lines; converting the reflection intensity values ​​of the data within the scan lines into grayscale values; calculating the optimal segmentation threshold based on the probabilities of the lane line point data and the non-lane line point data, and the grayscale values; selecting data whose grayscale values ​​within the scan lines are greater than the optimal segmentation threshold as lane line candidate point cloud data; and using the lane line candidate point cloud data to identify the lane lines of the road surrounding the vehicle. The step of calculating the optimal segmentation threshold based on the probabilities of the lane line point data and the non-lane line point data, and the grayscale value, includes: calculating a secondary grayscale mean of the grayscale value based on the grayscale value greater than the mean grayscale value, using the secondary grayscale mean as an initial threshold to determine the threshold selection range; calculating the first mean and the second mean corresponding to the probabilities of the lane line point data and the non-lane line point data respectively, calculating the inter-class variance based on the first mean, the second mean, and the mean of grayscale values ​​greater than any threshold in the threshold selection range; and using the threshold that makes the inter-class variance reach its maximum value as the optimal segmentation threshold.

2. The method according to claim 1, characterized in that, The process of fitting the point cloud data of each sub-region to the ground plane to obtain ground point cloud data includes: Extract the seed point set of the point cloud data for each sub-region; A planar model is established based on the seed point set, and a ground plane is obtained by fitting the planar model. The orthogonal projection distance from the point cloud data of each sub-region to the ground plane is calculated. If the orthogonal projection distance is less than the distance threshold, the corresponding point cloud data is determined to belong to the ground plane; otherwise, the corresponding point cloud data is determined not to belong to the ground plane. The ground point cloud data is obtained by filtering the point cloud data that belong to the ground plane.

3. The method according to claim 2, characterized in that, The seed point set for extracting the point cloud data of each sub-region includes: The point cloud data of each sub-region is sorted according to the point cloud height to obtain sorted point cloud data; Identify the lowest point representative and multiple target height points in the sorted point cloud data, and determine the extraction threshold based on the average of the lowest point representative and the multiple target height points; Points in the sorted point cloud data that are less than or equal to the extraction threshold are used as seed points, and the seed point set is generated based on the extracted seed points.

4. The method according to claim 1, characterized in that, The step of identifying lane lines on the road surrounding the vehicle using the lane line candidate point cloud data includes: Identify the lane direction of the vehicle; Density clustering is performed on the candidate point cloud data of the lane lines based on the lane line direction to obtain the clustering results. The lane lines of the road around the vehicle are obtained by fitting the lane lines using the clustering results.

5. The method according to claim 1, characterized in that, The identification of the region of interest in the 3D point cloud data includes: The background data of the 3D point cloud data is filtered out to obtain the region of interest.

6. A lane line detection device, characterized in that, include: The acquisition module is used to acquire 3D point cloud data around the vehicle; The first recognition module is used to identify the region of interest in the three-dimensional point cloud data, segment the region of interest along the driving direction of the vehicle to obtain multiple sub-regions, and fit the point cloud data of each sub-region to the ground plane to obtain ground point cloud data. The second identification module is used to identify the ground point cloud data and obtain the lane lines of the road around the vehicle; The second identification module is further configured to classify the ground point cloud data into lane line point data and non-lane line point data according to the scan lines; convert the reflection intensity values ​​of the data within the scan lines into grayscale values; calculate the optimal segmentation threshold based on the probabilities of the lane line point data and the non-lane line point data, and the grayscale values; filter the data within the scan lines whose grayscale values ​​are greater than the optimal segmentation threshold as lane line candidate point cloud data; and use the lane line candidate point cloud data to identify the lane lines of the road surrounding the vehicle. The second recognition module is further configured to calculate a secondary gray mean of the gray value based on the gray value greater than the average gray value, and use the secondary gray mean as an initial threshold to determine the threshold selection range; Calculate the first mean and the second mean corresponding to the probabilities of the lane line point data and the non-lane line point data respectively. Calculate the inter-class variance based on the first mean, the second mean, and the mean of gray values ​​greater than any threshold in the threshold selection interval. The threshold that makes the inter-class variance reach its maximum value is taken as the optimal segmentation threshold.

7. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the lane line detection method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the lane line detection method as described in any one of claims 1-5.

Images