Lane line multi-cluster virtual generation method for autonomous driving

Abstract

This invention discloses a multi-cluster virtual lane line generation method for autonomous driving. It selects M frames using dynamically changing frame interval sampling. Using density clustering and the operating patterns of lane lines, the current frame is identified and classified to determine three types of point features: virtual and real points, straight and curved points, and distance points. A triple clustering method is then used to obtain point sets for the left and right lane lines of the current lane. Finally, a least-squares classification method is used to generate lane line fitting. This method fully considers the dynamic changes of the vehicle during driving, utilizing multi-frame mapping to obtain lane line point features in three dimensions to improve recognition accuracy. Furthermore, a multi-clustering approach is used to analyze multiple lane line categories and improve lane line fitting accuracy.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and intelligent driving, and in particular to a method for virtual generation of lane lines by multi-clustering after image preprocessing through density clustering. Background Art

[0002] Intelligent driving technology has matured, enabling autonomous driving to replace drivers. Leveraging high-precision sensors and rigorous algorithms, accidents can be further reduced. Within the intelligent driving field, lane markings, a key parameter captured by high-precision sensors and a crucial factor in driving safety, are a fundamental and critical technology.

[0003] Traditional lane recognition technology relies primarily on image feature extraction and recognition and model-based recognition. Image feature-based methods primarily utilize lane shape, pixel, and color features captured by high-precision sensors, combined with edge detection operators to identify edge information. While this approach offers rapid recognition speed and good adaptability, it is susceptible to interference from environmental factors such as lighting and exhibits poor robustness. Mathematical model-based methods, on the other hand, utilize established mathematical curve models to fit the extracted edge information. However, this approach requires high curve fitting, is computationally complex, and is susceptible to noise. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and propose a method for virtual generation of lane lines by clustering for autonomous driving. To achieve the above-mentioned purpose, the present invention provides the following technical solution: The method for virtual generation of lane lines by clustering for autonomous driving comprises the following steps:

[0005] S1. Acquire a current road surface image based on real-time information of vehicle travel, dynamically determine the frame interval of the image, and randomly extract M frames at the frame interval;

[0006] S2. Preprocess the randomly selected M frames to obtain M lane line candidate frames m;

[0007] S3. Using density clustering and lane line movement rules, identify and classify all pixel points of the lane line candidate frame of the current M-th frame to obtain three types of point features of the lane line candidate frame of the current M frames: virtual and real point features, straight and curved point features, and distance point features;

[0008] S4. Based on the three types of point features, a triple clustering method is used to obtain the left lane line point set and the right lane line point set of the current lane line;

[0009] S5. Obtain a dashed line point set, a solid line point set, a straight line point set, and a curve point set from the left lane point set and the right lane point set of the current lane respectively; then perform fitting on the solid line point set and the dashed line point set to obtain a solid line fitting point set and a dashed line fitting point set respectively;

[0010] S6. Perform lane line fitting using least squares classification based on the solid line fitting point set, dashed line fitting point set, straight line point set, and curve point set obtained in step S5 to generate lane lines.

[0011] Furthermore, in the aforementioned step S1, the frame interval FI is dynamically determined according to the following formula based on the real-time information of the current vehicle driving:

[0012]

[0013] Where d is the average distance between the dashed and solid lines, v is the current vehicle speed, and δ is the camera sampling speed. Indicates the absolute value of the current vehicle steering average.

[0014] Furthermore, the aforementioned step S2 is specifically as follows: the preprocessing process includes performing inverse perspective transformation on the randomly selected M frames, then converting the lane trapezoidal region of interest into a bird's-eye view, and then performing binarization and background removal processing on each frame to obtain M lane line candidate frames.

[0015] Furthermore, obtaining the virtual and real point features of the current M lane line candidate frames in the aforementioned step S3 includes the following sub-steps:

[0016] S301, calculate the correction value α' of the vehicle steering in the Mth frame and the M-1 frame according to the following formula Mm :

[0017]

[0018] Among them, α Mm is the raw value obtained by the steering sensor, with right turn being positive and left turn being negative, and Δ is the preset threshold for judging the vehicle's steering state;

[0019] S302, for the pixel point at position (x, y) of the candidate frame m corresponding to the Mth frame The mapping pixel point is The pixel position of the Mth frame mapped to the candidate frame m due to vehicle steering is calculated according to the following formula:

[0020]

[0021]

[0022] Among them, X0 is the image resolution in the X direction, and Y0 is the image resolution in the Y direction;

[0023] S303, density clustering is performed on the mapped pixels of the candidate frame m according to the following formula: The density clustering set of candidate frame m

[0024]

[0025] in, is the pixel point at position (x,y) in the candidate frame m, is the grayscale value of the candidate frame m, The pixel point is a suspected lane point;

[0026] S304: According to step S303, traverse 1≤m≤M, take the density set modulus, and get the number of set elements Determine the lane line virtual and real point features based on the number of set elements. but

[0027]

[0028] like but

[0029]

[0030] Among them, th1 is the high density threshold, th2 is the low density threshold, is a high-density counter, which is initialized to 0. is a low-density counter, which is initialized to 0.

[0031] S305: Determine the virtual and real point features of the mapped pixel points according to the following formula:

[0032]

[0033] in, represents a solid line point, represents a dotted point, Indicates non-lane points;

[0034] S306: traverse all pixels in the current Mth frame Get the real and virtual point features of any pixel in the Mth frame

[0035] Furthermore, obtaining the straight and curved point features of the lane line candidate frame image of the current Mth frame in the aforementioned step S3 includes the following sub-steps:

[0036] S311. Calculate the average absolute value of the steering correction value of the vehicle in the current M-th frame:

[0037]

[0038] S312, calculate the straight-curve point feature of the current mapping pixel point according to the following formula

[0039]

[0040] Among them, th3 is the turning threshold, represents a straight line point, represents the curve point, Indicates a non-lane point.

[0041] S313, traverse all pixels in the current M-th frame Get the straight-curve point feature of any pixel point in the Mth frame

[0042] Furthermore, obtaining the distance point features of the current M lane line candidate frame images in the aforementioned step S3 includes the following sub-steps:

[0043] S321, As the boundary, according to the distance from the current mapping pixel to the boundary, the distance point feature of the current mapping pixel is calculated according to the following formula

[0044]

[0045] when When it indicates the left side of the boundary, : indicates the right side of the boundary.

[0046] S322, traverse all pixels in the current Mth frame Get the distance point feature of any pixel point in the Mth frame

[0047] Furthermore, the aforementioned step S4 includes the following sub-steps:

[0048] S401, according to and The value of and Perform clustering and get NS 11 NS 12 NS 21 and NS 22 Four types of point sets, representing real straight line, real curve, dashed straight line and dashed curve point sets respectively;

[0049] S402, based on distance point features Perform K-means clustering to obtain cluster point sets, cluster point sets They are:

[0050]

[0051] Among them, 1≤i≤2, 1≤j≤2 and 1≤k≤K;

[0052] S403: Determine the left and right lane lines based on the distance from the cluster center of the cluster point set to the boundary line using the following formula:

[0053]

[0054]

[0055] Among them, th4 is the threshold value of the number of point set elements, is the cluster point set Cluster center to boundary The distance to the left of the boundary is positive, and the value to the right of the boundary is negative. |·| is the number of elements in the point set, NS 1* Represents the point set of the left lane line where the vehicle is currently located, NS 2* The point set representing the lane line on the right where the vehicle is currently located.

[0056] Furthermore, the aforementioned step S5 is specifically as follows: according to the point set NS of the left and right lane lines where the vehicle is currently located i* , i∈[1,2], perform lane line fitting, if NS i* is a solid line point set, then ID 1 (NS i* )=1, otherwise it is a dotted point set, i.e. ID 1 (NS i* )=2;

[0057] If NS i* is a line point set, then ID 2 (NS i* )=1, otherwise it is a curve point set, i.e. ID 2 (NS i* )=2;

[0058] When ID 1 (NS i* )=1, then select the boundary The point with the smallest absolute distance in the same row is taken as the fitting point, and the solid line fitting point set is obtained by row-by-row scanning according to the following formula

[0059]

[0060] When ID 1 (NS i* )=2, then select the boundary The point with the largest absolute value of the distance in the same row is taken as the fitting point, and the dotted line fitting point set is obtained by row-by-row scanning according to the following formula

[0061]

[0062] Furthermore, in the aforementioned step S6, if NS i* is a line point set, i.e. ID 2 (NS i* )=1, then the fitting lane line function is y=a1x+b1, and the values ​​of a1 and b1 are:

[0063]

[0064]

[0065] Among them, |·| is the number of elements in the point set, (x r ,y r ) is the fitting point set The coordinates of the fitting points in the

[0066] If NS i* is the curve point set, i.e. ID 2 (NS i* )=2, then the fitting lane line function is y=a2x 2 +b2x+c2, where a2, b2, and c2 satisfy the following formula:

[0067]

[0068] Then, the curve parameter a2 is corrected according to the vehicle steering degree according to the following formula;

[0069]

[0070] Then when ID 2 (NS i* )=2, the final fitting lane line function is y=a'2x 2 +b2x+c2.

[0071] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0072] The present invention fully considers the dynamic changes of vehicles during driving, uses multi-frame mapping to obtain the point features of lane lines in three dimensions to improve recognition accuracy, and analyzes multiple categories of lane lines based on a multi-clustering method to improve the fitting accuracy of lane lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is a flow chart of the present invention.

[0074] Figure 2 It is a schematic diagram of the mapping points of the current M-th frame.

[0075] Figure 3 It is a schematic diagram of fitting point selection. DETAILED DESCRIPTION

[0076] In order to better understand the technical content of the present invention, specific embodiments are given and described below with reference to the accompanying drawings.

[0077] Various aspects of the present invention are described herein with reference to the accompanying drawings, which show a number of illustrative embodiments. The embodiments of the present invention are not limited to those described in the accompanying drawings. It should be understood that the present invention can be implemented by any of the various concepts and embodiments described above, as well as the concepts and implementations described in detail below, because the concepts and embodiments disclosed herein are not limited to any particular implementation. In addition, some aspects disclosed herein may be used alone or in any appropriate combination with other aspects disclosed herein.

[0078] The lane line multi-cluster virtual generation method for autonomous driving has the following process: Figure 1 The specific steps are as follows:

[0079] S1. Based on the real-time information of the vehicle, the current road surface image is obtained and the frame interval FI is dynamically determined by the following formula:

[0080]

[0081] Where d is the average distance between the dashed and solid lines, v is the current vehicle speed, and δ is the camera sampling speed. Indicates the absolute value of the current vehicle steering average. Randomly extract M frames of images at the frame interval.

[0082] S2. Perform inverse perspective transformation on the M sample frames to convert the lane trapezoidal region of interest into a bird's-eye view. At the same time, perform binarization and background removal on each frame to obtain M lane line candidate frames m.

[0083] S3. For the M lane line candidate frame images, use density clustering and lane line operation rules to identify and classify all pixel points of the current M-th lane line candidate frame image, and obtain three types of point features of the current M-th lane line candidate frame image: virtual and real point features, straight and curved point features, and distance point features.

[0084] Obtaining the virtual and real point features of the current M lane line candidate frame images includes the following sub-steps:

[0085] S301, using density clustering and lane line operation rules, calculate the correction value α' of the vehicle steering in the Mth frame and the M-1 frame according to the following formula Mm :

[0086]

[0087] Among them, α Mm This is the raw value obtained by the steering sensor. Right turn is positive and left turn is negative.

[0088] S302, for the mth frame corresponding to The mapping pixel point is The pixel point of the Mth frame is mapped to the pixel point position of the mth frame due to the vehicle turning, such as Figure 2 As shown, the specific pixel positions are expressed as follows:

[0089]

[0090]

[0091] Among them, X0 is the image resolution in the X direction, and Y0 is the image resolution in the Y direction.

[0092] S303, density clustering is performed on the mapped pixels of the candidate frame m according to the following formula: The density clustering set in candidate frame m

[0093]

[0094] in, is the pixel point at position (x,y) in the candidate frame m, is the grayscale value of the candidate frame m, This pixel is a suspected lane point.

[0095] S304: According to step S303, traverse 1≤m≤M, take the density set modulus, and get the number of set elements Determine the lane line virtual and real point features based on the number of set elements. but

[0096]

[0097] like but

[0098]

[0099] Among them, th1 is the high density threshold, th2 is the low density threshold, is a high-density counter, which is initialized to 0. is a low-density counter, which is initialized to 0.

[0100] S305: Determine the virtual and real point features of the mapped pixel points according to the following formula:

[0101]

[0102] in, represents a solid line point, represents a dotted point, Indicates a non-lane point.

[0103] S306: traverse all pixels in the current Mth frame Get the real and virtual point features of any pixel in the Mth frame

[0104] Obtaining the distance point features of the current M-frame lane line candidate frame image includes the following sub-steps:

[0105] S321, As the boundary, according to the distance from the current mapping pixel to the boundary, the distance point feature of the current mapping pixel is calculated according to the following formula

[0106]

[0107] when When it indicates the left side of the boundary, : indicates the right side of the boundary.

[0108] S322, traverse all pixels in the current Mth frame Get the distance point feature of any pixel point in the Mth frame

[0109] S4. Based on the three types of point features, a triple clustering method is used to obtain the left lane line point set and the right lane line point set of the current lane line;

[0110] S401, according to and The value of and Perform clustering and get NS 11 、NS 12 NS 21 and NS 22 Four types of point sets, representing real straight line, real curve, dashed straight line and dashed curve point sets respectively;

[0111] S402, based on distance point features Perform K-means clustering. When K is 4, it means there are at most four lanes, and the sixteen cluster point sets are They are:

[0112]

[0113] Among them, 1≤i≤2, 1≤j≤2 and 1≤k≤K;

[0114] S403: Determine the left and right lane lines based on the distance from the cluster center of the cluster point set to the boundary line using the following formula:

[0115]

[0116]

[0117] Among them, th4 is the threshold value of the number of point set elements, is the cluster point set Cluster center to boundary The distance to the left of the boundary is positive, and the value to the right of the boundary is negative. |·| is the number of elements in the point set, NS 1* Represents the point set of the left lane line where the vehicle is currently located, NS 2* The point set representing the lane line on the right where the vehicle is currently located.

[0118] S5. Obtain a dashed line point set, a solid line point set, a straight line point set, and a curve point set from the left lane point set and the right lane point set of the current lane respectively; then perform fitting on the solid line point set and the dashed line point set to obtain a solid line fitting point set and a dashed line fitting point set respectively;

[0119] Specifically: according to the point set NS of the left and right lane lines where the vehicle is currently located i* , i∈[1,2], perform lane line fitting, if NS i* is a solid line point set, then ID 1 (NS i* )=1, otherwise it is a dotted point set, i.e. ID 1 (NS i* )=2;

[0120] If NS i* is a line point set, then ID 2 (NS i* )=1, otherwise it is a curve point set, i.e. ID 2 (NS i* )=2;

[0121] For different point sets, different fitting points are selected. The fitting points are as follows: Figure 3 As shown:

[0122] When ID 1 (NS i* )=1, then select the boundary The point with the smallest absolute distance in the same row is taken as the fitting point, and the solid line fitting point set is obtained by row-by-row scanning according to the following formula

[0123]

[0124] When ID 1 (NS i* )=2, then select the boundary The point with the largest absolute value of the distance in the same row is taken as the fitting point, and the dotted line fitting point set is obtained by row-by-row scanning according to the following formula

[0125]

[0126] S6, solid line fitting point set obtained according to step S5 Dashed line fitting point set As well as straight line point sets and curve point sets, lane lines are fitted using the least squares classification method to generate lane lines.

[0127] If NS i* is a line point set, i.e. ID 2 (NS i* )=1, then the fitting lane line function is y=a1x+b1, and the values ​​of a1 and b1 are:

[0128]

[0129]

[0130] Among them, |·| is the number of elements in the point set, (x r ,y r ) is the fitting point set The coordinates of the fitting points in the

[0131] If NS i* is the curve point set, i.e. ID 2 (NS i* )=2, then the fitting lane line function is y=a2x 2 +b2x+c2, where a2, b2, and c2 satisfy the following formula:

[0132]

[0133] Then, the curve parameter a2 is corrected according to the vehicle steering degree according to the following formula;

[0134]

[0135] Then when ID 2 (NS i* )=2, the final fitting lane line function is y=a'2x 2 +b2x+c2.

[0136] While the present invention has been described above with reference to preferred embodiments, this is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A multi-cluster virtual generation method for lane lines for autonomous driving, characterized by: The steps include: S1. Acquire a current road surface image based on real-time information of vehicle travel, dynamically determine the frame interval of the image, and randomly extract M frames at the frame interval; S2. Preprocess the randomly selected M frames to obtain M lane line candidate frames m; S3. Using density clustering and lane line movement rules, identify and classify all pixel points of the current M-th lane line candidate frame to obtain three types of point features of the current M-th lane line candidate frame: virtual and real point features, straight and curved point features, and distance point features; where: Obtaining the virtual and real point features of the current M-frame lane line candidate frame includes the following sub-steps: S301, calculate the correction value α' of the vehicle steering in the Mth frame and the M-1 frame according to the following formula Mm : Among them, α Mm is the raw value obtained by the steering sensor, with right turn being positive and left turn being negative, and Δ is the preset threshold for judging the vehicle's steering state; S302, for the pixel point at position (x, y) of the candidate frame m corresponding to the Mth frame The mapping pixel point is The pixel position of the Mth frame mapped to the candidate frame m due to vehicle steering is calculated according to the following formula: Among them, X0 is the image resolution in the X direction, and Y0 is the image resolution in the Y direction; S303, density clustering is performed on the mapped pixels of the candidate frame m according to the following formula: The density clustering set in candidate frame m in, is the pixel point at position (x,y) in the candidate frame m, is the grayscale value of the candidate frame m, The pixel point is a suspected lane point; S304: According to step S303, traverse 1≤m≤M, take the density set modulus, and get the number of set elements Determine the lane line virtual and real point features based on the number of set elements. but like but Among them, th1 is the high density threshold, th2 is the low density threshold, is a high-density counter, which is initialized to 0. is a low-density counter, which is initialized to 0. S305: Determine the virtual and real point features of the mapped pixel points according to the following formula: in, represents a solid line point, represents a dotted point, Indicates non-lane points; S306: traverse all pixels in the current Mth frame Get the real and virtual point features of any pixel in the Mth frame Obtaining the straight and curved point features of the lane line candidate frame image of the current M frame includes the following sub-steps: S311. Calculate the average absolute value of the steering correction value of the vehicle in the current M-th frame: S312, calculate the straight-curve point feature of the current mapping pixel point according to the following formula Among them, th3 is the turning threshold, represents a straight line point, represents the curve point, Indicates non-lane points; S313, traverse all pixels in the current M-th frame Get the straight-curve point feature of any pixel point in the Mth frame Obtaining the distance point features of the current M lane line candidate frame images includes the following sub-steps: S321, As the boundary, according to the distance from the current mapping pixel to the boundary, the distance point feature of the current mapping pixel is calculated according to the following formula when When it indicates the left side of the boundary, When it is on the right side of the boundary; S322, traverse all pixels in the current Mth frame Get the distance point feature of any pixel point in the Mth frame S4. Based on the three types of point features, a triple clustering method is used to obtain the left lane line point set and the right lane line point set of the current lane line; S5. Obtain a dashed line point set, a solid line point set, a straight line point set, and a curve point set from the left lane point set and the right lane point set of the current lane respectively; then perform fitting on the solid line point set and the dashed line point set to obtain a solid line fitting point set and a dashed line fitting point set respectively; S6. Perform lane line fitting using least squares classification based on the solid line fitting point set, dashed line fitting point set, straight line point set, and curve point set obtained in step S5 to generate lane lines.

2. The lane line multi-cluster virtual generation method for autonomous driving according to claim 1, characterized in that: In step S1, the frame interval FI is dynamically determined according to the following formula based on the real-time information of the current vehicle driving: Where d is the average distance between the dashed and solid lines, v is the current vehicle speed, and δ is the camera sampling speed. Indicates the absolute value of the current vehicle steering average.

3. The lane line multi-cluster virtual generation method for autonomous driving according to claim 2, characterized in that: Step S2 is specifically as follows: the preprocessing process includes performing inverse perspective transformation on M randomly selected frames, then converting the lane trapezoidal region of interest into a bird's-eye view, and then performing binarization and background removal processing on each frame to obtain M lane line candidate frames.

4. The lane line multi-cluster virtual generation method for autonomous driving according to claim 1, characterized in that: Step S4 includes the following sub-steps: S401, according to and The value of and Perform clustering and get NS 11 NS 12 NS 21 and NS 22 Four types of point sets, representing real straight line, real curve, dashed straight line and dashed curve point sets respectively; S402, based on distance point features Perform K-means clustering to obtain cluster point sets, cluster point sets They are: Among them, 1≤i≤2, 1≤j≤2 and 1≤k≤K; S403: Determine the left and right lane lines based on the distance from the cluster center of the cluster point set to the boundary line using the following formula: Among them, th4 is the threshold value of the number of point set elements, is the cluster point set Cluster center to boundary The distance to the left of the boundary is positive, and the value to the right of the boundary is negative. |·| is the number of elements in the point set, NS 1* Represents the point set of the left lane line where the vehicle is currently located, NS 2* The point set representing the lane line on the right where the vehicle is currently located.

5. The lane line multi-cluster virtual generation method for autonomous driving according to claim 1, characterized in that: Step S5 is specifically as follows: according to the point set NS of the left and right lane lines where the vehicle is currently located i* , i∈[1,2], perform lane line fitting, if NS i* is a solid line point set, then ID 1 (NS i* )=1, otherwise it is a dotted point set, i.e. ID 1 (NS i* )=2; If NS i* is a line point set, then ID 2 (NS i* )=1, otherwise it is a curve point set, i.e. ID 2 (NS i* )=2; When ID 1 (NS i* )=1, then select the boundary The point with the smallest absolute distance in the same row is taken as the fitting point, and the solid line fitting point set is obtained by row-by-row scanning according to the following formula When ID 1 (NS i* )=2, then select the boundary The point with the largest absolute value of the distance in the same row is taken as the fitting point, and the dotted line fitting point set is obtained by row-by-row scanning according to the following formula 6. The lane line multi-cluster virtual generation method for autonomous driving according to claim 1, characterized in that: In step S6, if NS i* is a line point set, i.e. ID 2 (NS i* )=1, then the fitting lane line function is y=a1x+b1, and the values ​​of a1 and b1 are: Among them, |·| is the number of elements in the point set, (x r ,y r ) is the fitting point set The coordinates of the fitting points in the If NS i* is the curve point set, i.e. ID 2 (NS i* )=2, then the fitting lane line function is y=a2x 2 +b2x+c2, where a2, b2, and c2 satisfy the following formula: Then, the curve parameter a2 is corrected according to the vehicle steering degree according to the following formula; Then when ID 2 (NS i* )=2, the final fitting lane line function is y=a'2x 2 +b2x+c2.

Images