Driving lane line identification method and related equipment
Through multi-frame fusion and rasterization processing of laser point cloud data and navigation data, lane lines are extracted, and missed detection problems easily caused by single-frame data are solved, and the accuracy and reliability of lane line identification are improved. Autonomous navigation system suitable for smart vehicles.
Patent Information
- Application Number
- CN202411112313.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-06
AI Technical Summary
The existing lane line identification methods are ineffective under insufficient lighting or inclement weather conditions, and the identification algorithm based on single-frame point cloud data is prone to missed detection and missed detection, making it difficult to distinguish lane lines from other road markings.
By continuously obtaining laser point cloud data and combining navigation data, data preprocessing and multi-frame point cloud fusion, point cloud rasterization, raster clustering and linear fitting, extracting the final lane line, and using multi-frame data to improve continuity and accuracy to reduce misdetection caused by environmental factors.
It improves the continuity and accuracy of lane line detection, enhances the reliability and stability of the system, and can maintain high performance during night or inclement weather conditions, reducing the false detection rate.
Smart Images

Figure CN119091406A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automatic navigation technology, and in particular to a driving lane line recognition method and related equipment. Background Art
[0002] Lane recognition is one of the main technical points of vehicle autonomous navigation system and intelligent vehicle auxiliary safety system, and plays an important role in environmental perception, auxiliary positioning, path planning, motion decision-making and high-precision map production. The basic task of lane recognition is to obtain high-precision and real-time lane position information based on environmental perception data collected by sensors through a series of data analysis and processing.
[0003] The mainstream lane line recognition method is camera-based image processing technology, which has always been favored by scholars and has become one of the important research contents in the field of computer vision. The technology has been relatively mature, but due to the mechanism of the camera itself, it is easily affected by the ambient light conditions. When the light is insufficient, especially at night, the reliability is obviously insufficient. With the development of laser radar related technologies, coupled with the advantages of laser radar itself such as high precision and not being restricted by weather and light, lane line recognition technology based on three-dimensional laser point cloud has also received more and more attention. In the current recognition algorithm based on laser radar, single-frame point cloud is mostly used for processing. The single-frame data of general multi-line radar has low continuity, which is easy to lead to missed detection and low recall rate. It is also easily affected by outliers, sensitive to noise, and difficult to distinguish lane lines from other road markings such as crosswalks or guide lines, resulting in a high false detection rate. Summary of the invention
[0004] The embodiments of the present application provide a driving lane line recognition method and related equipment, which can more accurately identify driving lane lines from traffic marking lines and reduce the false detection rate of driving lane lines.
[0005] A first aspect of an embodiment of the present application provides a driving lane line recognition method, the method comprising:
[0006] Continuously acquiring laser point cloud data and integrated navigation data for a period of time, and performing data preprocessing operations on the laser point cloud data and the integrated navigation data to obtain multi-frame point cloud fusion data;
[0007] Performing point cloud rasterization operations, preliminary ground marking grid extraction operations and other marking filtering operations on the above multi-frame point cloud fusion data to obtain a filtered lane line grid;
[0008] Based on the above filtered lane line grid, grid clustering operation, cluster straight line segment fitting operation, lane straight line segment post-processing operation and lane line extraction operation are performed to obtain the final lane line.
[0009] In a feasible implementation manner, the data preprocessing operation is performed on the laser point cloud data and the combined navigation data to obtain multi-frame point cloud fusion data, including:
[0010] The laser point cloud data is converted to the vehicle body coordinates according to the calibration parameters to obtain the vehicle body coordinate laser point cloud data;
[0011] The obstacle point cloud is filtered out using a ray-based ground segmentation method on the above-mentioned vehicle body coordinate laser point cloud data to obtain filtered laser point cloud data;
[0012] Based on the above combined navigation data, the above filtered laser point cloud data is fused based on a coordinate transformation relationship to obtain the above multi-frame point cloud fusion data.
[0013] In a feasible implementation manner, the above-mentioned point cloud rasterization operation, the preliminary extraction operation of the ground marking grid and other marking filtering operations are performed on the above-mentioned multi-frame point cloud fusion data to obtain the filtered lane line grid, including:
[0014] Performing a point cloud rasterization operation on the multi-frame point cloud fusion data and determining raster attribute information, wherein the raster attribute information is determined based on the arithmetic average of the reflection intensity and the coordinate Z value of the multi-frame point cloud fusion data;
[0015] A direct filtering method is used to perform a preliminary extraction operation on the ground marking grid for the above-mentioned grid attributes to obtain the candidate lane line grid;
[0016] Other marking filtering operations are performed on the candidate lane line grid based on the grid erosion algorithm to obtain a filtered lane line grid.
[0017] In a feasible implementation manner, the above-mentioned performing other marking filtering operations on the above-mentioned candidate lane line grid based on the grid erosion algorithm to obtain the filtered lane line grid includes:
[0018] Traversing and judging at least two parts of candidate lane grids in a first traversal direction and a second traversal direction through a scanning window to delete other marking grids in the scanning window, wherein the scanning window is located at the starting side of the traversal direction of the currently traversed valid grid, and the first traversal direction and the second traversal direction are both perpendicular to the driving lane line and point to opposite directions;
[0019] The remaining candidate lane line grids are used as the above filtered lane line grids.
[0020] In a feasible implementation manner, a reserved interval is set between the scanning window and the currently traversed valid grid, and the reserved interval is determined according to the line width of the driving lane line to retain the valid grid within the reserved interval.
[0021] In a feasible implementation manner, the method further includes:
[0022] The lane line grid after the above filtering is filtered and optimized again based on the spatial density method.
[0023] In a feasible implementation, the above-mentioned grid clustering operation, cluster straight line segment fitting operation, lane straight line segment post-processing operation and lane line extraction operation are performed based on the above-mentioned filtered lane line grid to obtain the final lane line, including:
[0024] Perform grid clustering operation on the above filtered lane grid based on the main direction distance clustering method to obtain preliminary clustering results;
[0025] The least square method is used to perform cluster straight line segment fitting operation on the above preliminary clustering results to obtain the rough lane line extraction results;
[0026] The lane line rough extraction result is subjected to lane straight line segment post-processing based on the coefficient difference and the preset threshold value to obtain the lane line cluster merging result;
[0027] The lane line cluster merging result is subjected to a lane line extraction operation based on the least squares principle, and the obtained fitting cubic curve is determined as the final lane line.
[0028] A second aspect of an embodiment of the present application provides a driving lane line recognition device, the device comprising:
[0029] A first acquisition unit is used to continuously acquire laser point cloud data and combined navigation data for a period of time, and perform data preprocessing operations on the laser point cloud data and the combined navigation data to obtain multi-frame point cloud fusion data;
[0030] The second acquisition unit is used to perform point cloud rasterization operation, preliminary ground marking grid extraction operation and other marking filtering operations on the above-mentioned multi-frame point cloud fusion data to obtain a filtered lane line grid;
[0031] The third acquisition unit is used to perform grid clustering operation, cluster straight line segment fitting operation, lane straight line segment post-processing operation and lane line extraction operation based on the above filtered lane line grid to obtain the final lane line.
[0032] A third aspect of an embodiment of the present application provides an electronic device, which includes at least one processor and at least one memory connected to the processor, wherein the processor is used to call program instructions in the memory to execute the driving lane line recognition method of any one of the first aspects.
[0033] A fourth aspect of an embodiment of the present application provides a storage medium, which includes a stored program, wherein when the above-mentioned program is run, the device where the above-mentioned storage medium is located is controlled to execute the driving lane line recognition device method described in any one of the above-mentioned first aspects.
[0034] In summary, the embodiment of the present application provides a driving lane line recognition method, including: continuously acquiring laser point cloud data and combined navigation data for a period of time, and performing data preprocessing operations on the above laser point cloud data and the above combined navigation data to obtain multi-frame point cloud fusion data; performing point cloud rasterization operations, preliminary ground marking grid extraction operations and other marking filtering operations on the above multi-frame point cloud fusion data to obtain filtered lane line grids; performing grid clustering operations, cluster straight line segment fitting operations, lane straight line segment post-processing operations and lane line extraction operations based on the above filtered lane line grids to obtain the final lane line. The method proposed in the present application solves the problems of missed detection and continuity that are easily caused by single-frame point cloud data through multi-frame data fusion, and greatly improves the continuity and accuracy of lane line detection. Through precise data preprocessing and rasterization processing, non-lane line elements are effectively filtered out, and false detections caused by environmental factors are reduced. Compared with the camera-based lane line recognition technology, this solution is not limited by lighting and weather conditions, and can maintain high performance even at night or in severe weather conditions, thereby enhancing the reliability and stability of the system. In summary, this solution provides a more accurate, stable and practical lane line recognition method, which is suitable for the autonomous navigation system of modern intelligent vehicles and helps to improve the safety and efficiency of autonomous driving technology.
[0035] Correspondingly, the driving lane line device, electronic device and computer-readable storage medium provided in the embodiments of the present application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0037] Figure 1 A schematic flow chart of a driving lane line recognition method provided in an embodiment of the present application;
[0038] Figure 2 A schematic diagram of the principle of a driving lane line recognition method provided in an embodiment of the present application;
[0039] Figure 3 A schematic comparison diagram of radar point cloud data provided in an embodiment of the present application;
[0040] Figure 4 A schematic comparison diagram of a raster image before filtering and a raster image after filtering provided in an embodiment of the present application;
[0041] Figure 5 A schematic structural block diagram of a driving lane line recognition device provided in an embodiment of the present application;
[0042] Figure 6 A schematic hardware structure diagram of a driving lane line recognition device provided in an embodiment of the present application;
[0043] Figure 7 A schematic structural block diagram of an electronic device provided in an embodiment of the present application;
[0044] Figure 8 A schematic structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION
[0045] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.
[0046] See also Figure 1 , is a schematic flow chart of a driving lane line recognition method provided in an embodiment of the present application, Figure 2 A schematic diagram of a driving lane line recognition method provided in an embodiment of the present application may specifically include: S110-S130.
[0047] S110, continuously acquiring laser point cloud data and combined navigation data for a period of time, and performing data preprocessing operations on the laser point cloud data and the combined navigation data to obtain multi-frame point cloud fusion data;
[0048] For example, since the laser radar on the vehicle body generally continuously scans the environment and sends point cloud data at a certain frequency (such as 10 Hz), the amount of data in the above-mentioned multi-frame point cloud grid map is different when the vehicle is in different states. Figure 3 A schematic comparison diagram of radar point cloud data provided in an embodiment of the present application, such as Figure 3 As shown in the figure, a total of 20 frames of data within 2 seconds are shown. Figure 3 (a) The vehicle is travelling at normal speed. Figure 3 In (b), the vehicle stops at the intersection and waits for the indicator light. As can be seen from the figure, the point cloud coverage when the vehicle stops is much lower than when the vehicle is driving normally in the same period of time, that is, the effective point cloud is insufficient. Therefore, when the vehicle is driving normally, the N frames of point cloud closest in time can be quickly obtained. When the vehicle is driving slowly, the positions of the N frames of point cloud taken above are highly overlapped, the data redundancy is large, and the effective information is insufficient. Therefore, the effective point cloud should be obtained by combining the time and space relationship. Considering the effectiveness of the point cloud and the amount of data calculation, it is appropriate to select historical data with a longer time and delete the point cloud with a closer time but lower speed.
[0049] The method proposed in this application continuously collects point cloud data and combined navigation data generated by LiDAR. Navigation data may include GPS information, inertial measurement unit (IMU), etc., which help determine the precise position and direction of the vehicle. The collected data is preprocessed to integrate multiple frames of data to improve continuity problems in single frame data, reduce data noise, and improve data quality. This step is crucial for subsequent lane line recognition because it provides a clearer and continuous point cloud foundation for accurate recognition.
[0050] S120, performing point cloud rasterization operation, preliminary ground marking grid extraction operation and other marking filtering operations on the multi-frame point cloud fusion data to obtain a filtered lane grid;
[0051] Exemplarily, after data preprocessing, this step first performs point cloud rasterization processing, converting the point cloud into a raster map, simplifying the data structure, and improving the subsequent processing speed. Next, a preliminary extraction of the ground marking grid is performed. This step is mainly to identify and separate the grid area that may belong to the lane line. By filtering out non-lane line markings (such as crosswalks, guide lines and other markings), this step significantly reduces the possibility of false detection and improves the accuracy of recognition.
[0052] S130, performing a grid clustering operation, a cluster straight line segment fitting operation, a lane straight line segment post-processing operation, and a lane line extraction operation based on the above filtered lane line grid to obtain a final lane line.
[0053] Exemplarily, the obtained filtered lane line grid is further processed by a clustering algorithm to form clusters. Straight line segments are fitted for each cluster, and then these straight line segments are post-processed to merge the line segments belonging to the same lane. Finally, through the fitting and post-processing operations, clear and accurate lane lines are extracted.
[0054] In summary, the method proposed in this application solves the problems of missed detection and continuity that are easily caused by single-frame point cloud data through multi-frame data fusion, greatly improving the continuity and accuracy of lane line detection. Through precise data preprocessing and rasterization processing, non-lane line elements are effectively filtered out, reducing false detections caused by environmental factors. Compared with camera-based lane line recognition technology, this solution is not limited by lighting and weather conditions, and can maintain high performance even at night or in bad weather conditions, enhancing the reliability and stability of the system. In summary, this solution provides a more accurate, stable and practical lane line recognition method, which is suitable for autonomous navigation systems of modern intelligent vehicles and helps to improve the safety and efficiency of autonomous driving technology.
[0055] In some examples, the data preprocessing operation is performed on the laser point cloud data and the combined navigation data to obtain multi-frame point cloud fusion data, including:
[0056] The laser point cloud data is converted to the vehicle body coordinates according to the calibration parameters to obtain the vehicle body coordinate laser point cloud data;
[0057] The obstacle point cloud is filtered out using a ray-based ground segmentation method on the above-mentioned vehicle body coordinate laser point cloud data to obtain filtered laser point cloud data;
[0058] Based on the above combined navigation data, the above filtered laser point cloud data is fused based on a coordinate transformation relationship to obtain the above multi-frame point cloud fusion data.
[0059] For example, the system needs to perform calibration parameter conversion on the collected laser point cloud data, that is, convert the point cloud data from its original coordinate system (usually the coordinate system inherent to the laser radar device) to the vehicle body coordinate system. This conversion is based on the vehicle's own positioning system (such as GPS and IMU combined navigation data) to ensure that the point cloud data is aligned with the actual position and posture of the vehicle. Doing so can improve the accuracy of subsequent processing, especially in complex traffic environments.
[0060] After the point cloud data is converted to vehicle body coordinates, the next step is ground segmentation. Here, a ray-based ground segmentation method is used, which uses the relative angle between the ray emitted from the lidar and the ground to distinguish between ground and non-ground point clouds (i.e. obstacles). In this way, the obstacle point cloud can be effectively filtered out from the vehicle body coordinate laser point cloud data, leaving the main ground point cloud, which is crucial for subsequent lane line detection.
[0061] Finally, the laser point cloud data after filtering out obstacles is combined with the integrated navigation data to fuse multi-frame point cloud data. This step involves the coordinate transformation relationship based on the timestamp and spatial position, fusing the point cloud data acquired at different times to the current moment in the same vehicle body coordinate system. Through this fusion, a more continuous and complete point cloud data set in time and space can be constructed, laying the foundation for accurate lane line recognition.
[0062] Specifically, the installation position and angle of the lidar sensor will deviate to a certain extent from the ideal state. Therefore, before ground segmentation, the original point cloud in the radar coordinate system must be converted to the vehicle body coordinate system in combination with the lidar calibration parameters.
[0063] The coordinate system includes the laser radar coordinate system L_xyz, the vehicle body coordinate system V_xyz, and the world coordinate system W_xyz. The data processing of the multi-frame point cloud is performed at the current time (denoted as T now ) is performed in the vehicle body coordinate system, and the point clouds under V_xyz at different times cannot be directly converted and need to be realized with the help of the world coordinate system W_xyz. now The conversion relationship of time V_xyz is:
[0064]
[0065] in: In T i At time , the transformation matrix from the radar coordinate system L_xyz to the vehicle body coordinate system V_xyz is obtained by converting the calibration parameters;
[0066] In T i At this moment, the transformation matrix from the body coordinate system V_xyz to the world coordinate system W_xyz is given by T i Provide integrated navigation data at all times;
[0067] In T now At this moment, the transformation matrix from the world coordinate system W_xyz to the body coordinate system V_xyz is given by T now Provide integrated navigation data at all times;
[0068] [XYZ]L : Coordinate values of point cloud (i.e., lidar data points) in the radar coordinate system L_xyz;
[0069] In T now At this moment, the coordinate value of any point cloud after being converted to the vehicle body coordinate system V_xyz.
[0070] The above transformations are all Euclidean transformations, including rotation and translation, and the transformation matrix R = f(roll, pitch, yaw, ΔX, ΔY, ΔZ).
[0071] For each frame of point cloud, the obstacle point cloud is filtered out using a ray-based ground segmentation method to remove moving targets such as vehicles and pedestrians on the road, as well as stationary targets such as trees, street lights, and signboards, to reduce interference information for lane line recognition. However, considering the complex elements of the road environment and the limitations of the ground segmentation algorithm, the filtered laser point cloud data contains a large amount of valid ground point data, but still contains a small amount of non-ground point data such as curbs, sidewalks, and green belts, which need to be further filtered and screened.
[0072] In some examples, the above-mentioned point cloud rasterization operation, the preliminary extraction operation of the ground marking grid and other marking filtering operations are performed on the above-mentioned multi-frame point cloud fusion data to obtain the filtered lane line grid, including:
[0073] Performing a point cloud rasterization operation on the multi-frame point cloud fusion data and determining raster attribute information, wherein the raster attribute information is determined based on the arithmetic average of the reflection intensity and the coordinate Z value of the multi-frame point cloud fusion data;
[0074] A direct filtering method is used to perform a preliminary extraction operation on the ground marking grid for the above-mentioned grid attributes to obtain the candidate lane line grid;
[0075] Other marking filtering operations are performed on the candidate lane line grid based on the grid erosion algorithm to obtain a filtered lane line grid.
[0076] Exemplarily, the multi-frame point cloud fusion data is rasterized to convert the continuous point cloud data into a discrete grid image format, where each grid represents the point cloud information in a certain spatial area. The attribute information of the grid is determined by calculating the arithmetic mean of the reflection intensity and coordinate Z value of all point clouds in the grid. This conversion not only simplifies the data structure, making it easier to process, but also helps to more accurately identify and analyze lane lines in subsequent steps.
[0077] After the point cloud is rasterized, the next step is to perform a preliminary extraction of the ground marking grid. This step uses a pass-through filtering method, which selects grids that may represent ground markings based on the attribute information of the grid, such as reflection intensity and Z value. Ground markings are generally reflective paints, and the reflection intensity of their point clouds is generally significantly stronger than other parts of the ground; at the same time, the height of the ground (Z value) is generally at the lowest position in the environment, and other objects are generally located above the ground, and their Z values are also higher than the ground. In this example, point cloud data points with reflection intensity higher than a certain threshold and Z value lower than a certain threshold can be specifically selected for pass-through filtering. This screening process generates a set of candidate lane line grids, which provide candidate areas for further identification of actual lane lines.
[0078] Finally, the candidate lane grids are further screened, and the grid erosion algorithm is used to remove grids that may represent non-lane markings (such as crosswalks or other road markings). The grid erosion algorithm effectively removes isolated or atypical lane markings by analyzing the spatial relationship and attribute continuity around the grid, thereby purifying the lane point cloud data. The result of this step is to obtain filtered lane grids, which will be used for the final lane extraction and recognition.
[0079] Specifically, select the two-dimensional region of interest (xmin, xmax, ymin, ymax) and the grid size (pixel) under the vehicle body coordinate system V_xyz. x ,pixel y ), for the laser points falling on the same grid, the arithmetic mean of their reflection intensity and coordinate Z value is calculated as the attribute value of the grid.
[0080] Then in the grid coordinate system P_uv, the value of the region of interest is u∈[0,u max ],v∈[0,v max ],in
[0081]
[0082] The reflection intensity of laser radar for different materials is different. The reflection intensity at traffic markings will be significantly higher than that of general roads. Therefore, a suitable threshold can be selected and a straight-through filtering method can be used to obtain the alternative lane line grid.
[0083] Road traffic markings include various lines, arrows, text, patterns, etc. These markings have consistent performance in terms of reflection intensity. In addition, the reflection intensity characteristics of the green belts on both sides of the road are similar to those of traffic markings. It should also be noted that the geometric characteristics of the curb cause the incident angle of the laser to be close to a right angle when it hits it, making its reflection intensity higher than the road surface and close to the traffic markings. Therefore, it is necessary to identify the lane line grid from various markings, green belts, curbs and other interference information.
[0084] Identifying lane line grids from candidate grids is done in two steps. First, a grid erosion algorithm is used to remove most of the points in non-lane line marking areas such as stop lines, direction arrows, crosswalks, and green belts. Then, isolated points in non-lane line areas are removed based on the spatial density of grid points.
[0085] In some examples, the above-mentioned performing other marking filtering operations on the above-mentioned candidate lane line grid based on the grid erosion algorithm to obtain the filtered lane line grid includes:
[0086] Traversing and judging at least two parts of candidate lane line grids in a first traversal direction and a second traversal direction through a scanning window to delete other marking grids in the scanning window, that is, non-lane first grids, wherein the scanning window is located at the starting side of the traversal direction of the currently traversed valid grid, and the first traversal direction and the second traversal direction are both perpendicular to the driving lane line and point to opposite directions;
[0087] The remaining candidate lane line grids are used as the above filtered lane line grids.
[0088] For example, Figure 4 This is a multi-frame point cloud grid image before filtering. The point cloud in the image represents the traffic marking lines. It can be seen that there are no valid grids in the areas between the lane driving lines and stop lines, direction arrows, and crosswalks on both sides. Figure 4 The two white arrows in a represent the first traversal direction and the second traversal direction respectively. The scanning window is located at the starting side of the traversal direction of the valid grid currently traversed. For example, when Figure 4 When the valid grid on the left side of line A in a is traversed, the scanning window is located on the left side of the valid grid currently traversed. Figure 4 When the valid grids on the right side of the straight line A in a are traversed, the scanning window is located on the right side of the valid grid currently traversed.
[0089] Specifically, the grid erosion algorithm includes:
[0090] (1) Each row of grids in the multi-frame point cloud grid image can be segmented using formula (1):
[0091] v half = int(v max / 2)+1 (1)
[0092] Among them, v represents the y-axis direction of the above multi-frame point cloud grid image, v max Indicates the length range of the y-axis direction of the multi-frame point cloud grid image, v half Indicates the middle value of the y-axis direction of the above multi-frame point cloud grid image.
[0093] The above multi-frame point cloud grid image can be divided into two equal left and right parts by formula (1).
[0094] (2) For v≤v half , starting from v=0 and traversing to v=v half , when Cand i (u i , v i )=1, there is a valid grid point p that is traversed i (u i , v i ), define the following ranges:
[0095]
[0096] Then the determination method is as follows: any grid point p w (u w , v w ), if it is within the range of formula (2), it is determined to be other marking grids, that is, non-driving lane grids, then p w (u w , v w ) is deleted from the candidate lane grid. Wherein, u represents the x-axis direction of the multi-frame point cloud grid image, and v represents the y-axis direction of the multi-frame point cloud grid image. and Indicates the range of the above scanning window.
[0097] (3) Same as step (2), for v>v half , from v = v max Start traversing until v=v half -1, when Cand i (u i , v i )=1, grid point p i (u i , v i ) exists, in the above scanning window, the u range is the same as step (2), and the v range is as follows:
[0098]
[0099] The judgment method is that if p i (u i , v i ) is the grid point of the driving lane line, then the grid point p within the window range that satisfies formula (3) w (u w , v w ), will be judged as other marking grids, that is, non-driving lane grids, then p w (u w ,v w) is deleted from the candidate lane grid.
[0100] (4) Repeat steps (2) and (3) to complete the traversal of all grid rows in the above multi-frame point cloud grid image.
[0101] In some examples, a reserved interval is set between the scanning window and the currently traversed valid grid, and the reserved interval is determined according to the line width of the driving lane line to retain the valid grid within the reserved interval.
[0102] Exemplarily, a reserved interval is set between the scanning window and the currently traversed valid grid. The reserved interval is related to the line width of the driving lane line. In this way, during the traversal process, the driving lane line finally obtained has a certain width, which can be better distinguished from other traffic marking lines, thereby improving the accuracy of recognition.
[0103] A reserved interval may be set between the scanning window and the currently traversed valid grid, and the valid grid within the reserved interval is retained. For example, the above formula (2) and formula (3) may be respectively transformed into the following forms:
[0104]
[0105] in, It represents the reserved interval mentioned above, and laneWidthThr represents the width of the driving lane line.
[0106] In some examples, this also includes:
[0107] The lane line grid after the above filtering is filtered and optimized again based on the spatial density method.
[0108] For example, after running the grid erosion algorithm, there may still be a small number of isolated grids in the non-lane line area in the filtered candidate lane line grid. Here, a method based on spatial density is used to remove them. Spatial density (density) calculation can use conventional spatial density calculation methods, such as directly counting the number of lane line grid points in the neighborhood or using the inverse distance square method, kernel density and other methods for calculation. It should be noted that the neighborhood selection here should take into account the characteristics of the lane line. The neighborhood values in this example are density_wx=3, density_wy=0.3, and the unit is meter. Figure 4 The figure shows example data of candidate grids and lane line grids after filtering out. In the figure, (a) and (b) are: the candidate grid on the left and the lane line grid after filtering out on the right.
[0109] In some examples, the above-mentioned grid clustering operation, cluster straight line segment fitting operation, lane straight line segment post-processing operation and lane line extraction operation are performed based on the above-mentioned filtered lane line grid to obtain the final lane line, including:
[0110] Perform grid clustering operation on the above filtered lane grid based on the main direction distance clustering method to obtain preliminary clustering results;
[0111] The least square method is used to perform cluster straight line segment fitting operation on the above preliminary clustering results to obtain the rough lane line extraction results;
[0112] The lane line rough extraction result is subjected to lane straight line segment post-processing based on the coefficient difference and the preset threshold value to obtain the lane line cluster merging result;
[0113] The lane line cluster merging result is subjected to a lane line extraction operation based on the least squares principle, and the obtained fitting cubic curve is determined as the final lane line.
[0114] For example, the filtered lane grids are clustered. This step uses a clustering method based on the main direction distance. This method focuses on identifying and clustering grids that are close in distance in the main direction to form a preliminary clustering result. This clustering method helps distinguish the positions of different lane lines and provides preliminary grouping for subsequent line segment fitting.
[0115] The least squares method is used to fit the straight line segments of each cluster grid to obtain the rough extraction results of the lane lines. The least squares method provides a mathematically accurate way to represent the geometry of each cluster by minimizing the sum of squared errors to determine the best fit line. Converting scattered grid points into organized straight line forms provides a basis for further processing.
[0116] The lane line rough extraction results are post-processed based on the coefficient difference and the preset threshold. The geometric coefficients (such as slope and intercept) of each straight line segment are compared, and whether to merge adjacent straight line segments is determined based on whether the difference of these coefficients is lower than the preset threshold. This can reduce lane line breaks caused by grid discontinuity or noise, and improve the continuity and integrity of the lane line.
[0117] Finally, the lane line extraction operation is performed based on the least squares method for the lane line cluster merging results, and the final lane line is determined by fitting a cubic curve. Cubic curve fitting provides sufficient flexibility to adapt to various bends and changes on the road, ensuring that the lane line model is both accurate and capable of describing complex road conditions. This high-order polynomial fitting is ultimately determined as the geometric representation of the lane line, providing an important visual basis for navigation and driving decisions.
[0118] Specifically, according to the geometric characteristics of the lane lines, the grid clustering is performed using the principal direction distance clustering method (Principal Direction Cluster, PDC). The distance metric of PDC is the principal direction distance vector D. For any n-dimensional point, D = (d1, d2, ..., d n ), accordingly, the threshold vector is THR=(thr1,thr2,…,thr n ). In the present invention, for a two-dimensional grid point, its main direction distance vector is:
[0119]
[0120] Among them, θ is the angle between the main direction and the X-axis of the coordinate system, the counterclockwise direction is positive, and the value range is [-π,π).
[0121] In reality, traffic markings are straight lines or curves with a certain width and length. Therefore, in the embodiment of the present application, the main direction of the driving lane line grid clustering can be set as the vehicle forward direction and the body lateral direction perpendicular thereto. Then, in the above formula (6), θ is the angle between the x-axis of the body coordinate system and the vehicle forward direction. Since the vehicle generally moves along the direction of the driving lane line, θ=0. Then, in formula (6), d1 and d2 are x and y respectively, that is, the main direction distance vector D=(x, y), and the threshold vector THR=(thr x ,thr y ), based on the actual situation, thr x The value of can be greater than the longitudinal distance of the lane line, such as 6m. y The value of can be determined based on the length of each grid, for example, it can be 0.15m.
[0122] The clustering implementation process is as follows:
[0123] (1) PD(THR) neighborhood definition: In the filtered lane grid, for any grid point p0(u0, v0), if the grid point p i (u i , v i )satisfy
[0124]
[0125] in, Then p i is a point in the neighborhood of grid point p0, and the neighborhood of PD(THR) is all p points that meet the conditions. i A collection of points;
[0126] (2) Select any grid point p1 as the first-level seed point, use the grid index to perform PD(THR) neighborhood search, and find the points in the neighborhood (denoted as p1)11 , p 12 , p 13 , ...) are classified into the same cluster and put into class Q1, the neighboring points are marked as processed points, and these neighboring points are recorded as secondary seed points;
[0127] (3) Using the secondary seed points in class Q1 as seed points, repeat step (2) to perform neighborhood search, and the obtained neighborhood points are recorded as tertiary seed points;
[0128] (4) The search is repeated in this way until no new points are added to class Q1, and the clustering of Q1 is completed.
[0129] (5) Get the total number num of points in cluster Q1. When num>N_min, retain cluster Q1.
[0130] (6) Select another unprocessed grid point p2 and repeat steps (2-5) to complete the clustering of Q2 until all grid points are clustered.
[0131] For each cluster Q of the above clustering results, the least squares method is used to perform rough lane line extraction. There are n points p in cluster Q. i (x i ,y i ), (i=1,2,3,...,n), and we want to fit it into a straight line. The straight line expression is:
[0132] h C (x) = C0 + C1x (8)
[0133] Among them, C = (C0, C1) is the parameter to be determined. The residual sum of squares is used as the objective function:
[0134]
[0135] When the objective function is minimized, the value of the parameter C is what is required.
[0136] For the rough extraction of lane lines, multiple straight lines are obtained. When the coefficient difference (dC0, dC1) of two straight lines is less than a certain threshold, they are considered to belong to the same lane line. Then the clusters corresponding to the multiple fitted straight lines belonging to the same lane line are merged, and new fitted straight lines are refitted and compared with other fitted straight lines until no straight lines can be merged.
[0137] The width of each lane on the road is generally 3.5 to 3.75 meters, and the width of each lane at the intersection is generally 2.3 to 2.5 meters. Based on this, some misidentified straight line segments can be filtered out, such as driving direction indicator lines, etc. The fused lane fitting straight lines are sorted in descending order according to the length of the line segments. For two line segments that are closer, the shorter line segment is considered to be a non-lane line area and deleted.
[0138] Finally, the post-processed clusters are fitted with a cubic curve h based on the least squares rule. C (x) = C0 + C1x + C1x 2 , which is the final lane line.
[0139] See also Figure 5 In the embodiments of the present application, an embodiment of a driving lane line recognition device is described, which may include:
[0140] The first acquisition unit 401 is used to continuously acquire laser point cloud data and integrated navigation data for a period of time, and perform data preprocessing operations on the laser point cloud data and the integrated navigation data to obtain multi-frame point cloud fusion data;
[0141] The second acquisition unit 402 is used to perform point cloud rasterization operation, preliminary ground marking grid extraction operation and other marking filtering operations on the above-mentioned multi-frame point cloud fusion data to obtain a filtered lane line grid;
[0142] The third acquisition unit 40 is used to perform grid clustering operation, cluster straight line segment fitting operation, lane straight line segment post-processing operation and lane line extraction operation based on the above filtered lane line grid to obtain the final lane line.
[0143] above Figure 5 The lane line recognition device in the embodiment of the present application is described from the perspective of modular functional entities. The lane line recognition device in the embodiment of the present application is described in detail from the perspective of hardware processing. Please refer to Figure 6 , a schematic hardware structure diagram of a lane line recognition device provided in an embodiment of the present application, the lane line recognition device 500 includes:
[0144] An input device 501, an output device 502, a processor 503 and a memory 504, wherein the number of the processor 503 can be one or more. Figure 5 In some embodiments of the present application, the input device 501, the output device 502, the processor 503 and the memory 504 may be connected via a bus or other means, wherein: Figure 6 The example of connecting through bus is taken in the following.
[0145] Wherein, by calling the operation instructions stored in the memory 504, the processor 503 is used to execute the steps of the above-mentioned lane line recognition method proposed in any one of the above-mentioned first aspects.
[0146] In the specific implementation process, please refer to Figure 7 , Figure 7This is a schematic structural block diagram of an electronic device provided in an embodiment of the present application. When the processor 620 executes the computer program 611 in the memory 610, it can achieve Figure 1 Any implementation method in the corresponding embodiment. Since the electronic device introduced in this embodiment is a device used to implement a system resource management device in the embodiment of the present application, based on the method introduced in the embodiment of the present application, the technical personnel in this field can understand the specific implementation method of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as the equipment used by the technical personnel in this field to implement the method in the embodiment of the present application belongs to the scope of protection of this application.
[0147] See also Figure 8 , Figure 8 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present application.
[0148] like Figure 8 As shown, this embodiment provides a computer-readable storage medium 700 on which a computer program 711 is stored. When the computer program 711 is executed by a processor, the steps of the lane line recognition method proposed in any one of the first aspects above are implemented.
[0149] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0150] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0152] The present application also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device is caused to execute the following Figure 1 The process in the lane line recognition method in the corresponding embodiment.
[0153] The above-mentioned computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, the above-mentioned process or function according to the embodiment of the present application is generated in whole or in part. The above-mentioned computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The above-mentioned computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The above-mentioned available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (solid state disk, SSD)), etc.
[0154] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0155] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0156] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0157] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0158] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0159] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A driving lane line recognition method, characterized in that: include: Continuously acquiring laser point cloud data and integrated navigation data for a period of time, and performing data preprocessing operations on the laser point cloud data and the integrated navigation data to obtain multi-frame point cloud fusion data; Performing point cloud rasterization operation, preliminary ground marking grid extraction operation and other marking filtering operations on the multi-frame point cloud fusion data to obtain a filtered lane line grid; Based on the filtered lane line grid, a grid clustering operation, a cluster straight line segment fitting operation, a lane straight line segment post-processing operation and a lane line extraction operation are performed to obtain a final lane line.
2. The driving lane line recognition method according to claim 1, characterized in that: The performing of data preprocessing operation on the laser point cloud data and the combined navigation data to obtain multi-frame point cloud fusion data includes: Converting the laser point cloud data to vehicle body coordinates according to calibration parameters to obtain vehicle body coordinate laser point cloud data; Using a ray-based ground segmentation method to filter out obstacle point clouds from the vehicle body coordinate laser point cloud data to obtain filtered laser point cloud data; Based on the combined navigation data, the filtered laser point cloud data is fused based on a coordinate transformation relationship to obtain the multi-frame point cloud fusion data.
3. The driving lane line recognition method according to claim 1, characterized in that: The performing point cloud rasterization operation, preliminary ground marking grid extraction operation and other marking filtering operations on the multi-frame point cloud fusion data to obtain a filtered lane grid includes: Performing a point cloud rasterization operation on the multi-frame point cloud fusion data, and determining raster attribute information, wherein the raster attribute information is determined based on the arithmetic average of the reflection intensity and the coordinate Z value of the multi-frame point cloud fusion data; A through-filter method is used to perform a preliminary extraction operation of the ground marking grid on the grid attributes to obtain a candidate lane grid; Other marking line filtering operations are performed on the candidate lane line grid based on a grid erosion algorithm to obtain a filtered lane line grid.
4. The driving lane line recognition method according to claim 3, characterized in that: The performing other marking filtering operations on the candidate lane line grid based on the grid erosion algorithm to obtain a filtered lane line grid includes: Traversing and judging at least two parts of candidate lane grids in a first traversal direction and a second traversal direction through a scanning window to delete other marking grids in the scanning window, wherein the scanning window is located at the starting side of the traversal direction of the currently traversed valid grid, and the first traversal direction and the second traversal direction are both perpendicular to the driving lane line and point to opposite directions; The remaining candidate lane line grids are used as the filtered lane line grids.
5. The driving lane line recognition method according to claim 4, characterized in that: A reserved interval is set between the scanning window and the currently traversed valid grid, and the reserved interval is determined according to the line width of the driving lane line to retain the valid grid within the reserved interval.
6. The driving lane line recognition method according to claim 3, characterized in that: Also includes: The filtered lane grid is filtered and optimized again based on a spatial density method.
7. The driving lane line recognition method according to claim 1, characterized in that: The performing grid clustering operation, cluster straight line segment fitting operation, lane straight line segment post-processing operation and lane line extraction operation based on the filtered lane line grid to obtain the final lane line includes: Performing a grid clustering operation on the filtered lane grid based on a clustering method of main direction distance to obtain a preliminary clustering result; Performing cluster straight line segment fitting operation on the preliminary clustering result by using the least square method to obtain a rough lane line extraction result; Performing lane line segment post-processing operation on the lane line rough extraction result based on the coefficient difference and the preset threshold value to obtain the lane line cluster merging result; A lane line extraction operation is performed on the lane line cluster merging result based on the least squares principle, and the obtained fitting cubic curve is determined as the final lane line.
8. A driving lane line recognition device, characterized in that: include: A first acquisition unit is used to continuously acquire laser point cloud data and integrated navigation data for a period of time, and perform data preprocessing operations on the laser point cloud data and the integrated navigation data to obtain multi-frame point cloud fusion data; A second acquisition unit is used to perform point cloud rasterization operation, preliminary ground marking grid extraction operation and other marking filtering operations on the multi-frame point cloud fusion data to obtain a filtered lane line grid; The third acquisition unit is used to perform a grid clustering operation, a cluster straight line segment fitting operation, a lane straight line segment post-processing operation and a lane line extraction operation based on the filtered lane line grid to obtain a final lane line.
9. An electronic device, characterized in that: The electronic device includes at least one processor and at least one memory connected to the processor, wherein the processor is used to call program instructions in the memory to execute the driving lane line recognition method as described in any one of claims 1 to claim 7.
10. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the driving lane line recognition method according to any one of claims 1 to claim 7.
Citation Information
Patent Citations
Lane line detection method and device
CN111832536A
Lane line detection method, data processing unit and vehicle-mounted laser radar system
CN113534170A
Lane line detection method, terminal and storage medium
CN115346183A
Information completion method, lane line recognition method, intelligent driving method and related product
WO2021082745A1
