Lane line identification method

Abstract

The invention provides a lane line identification method, and relates to the technical field of data processing. The method comprises the following steps: acquiring a to-be-processed image frame of a target vehicle; wherein the to-be-processed image frame comprises a lane line basic recognition result; performing feature extraction on the to-be-processed image frame to obtain a pixel point set of a lane line basic recognition result; wherein each pixel point in the pixel point set comprises a coordinate value and a direction angle; clustering the pixel points based on the coordinate values and the direction angles to generate a lane line pixel point cluster; and recognizing the lane line according to the pixel point cluster of the lane line. According to the lane line identification method and device, the noise points with the suddenly changed directions in the lane line pixel point set are filtered out through direction constraint, the pixel points in different directions are promoted to be separated, the pixel points in the same direction basically can be gathered together more easily, the lane line pixel point cluster is generated, and the lane line identification accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a lane line recognition method. Background Technology

[0002] Lane detection plays a crucial role in the field of autonomous driving and is an important component of the autonomous driving environment perception module. Currently, there are two main methods for lane detection: traditional image processing methods and deep learning methods.

[0003] However, traditional image processing methods and deep learning methods suffer from issues such as missed detections when there are a large number of lane lines, and the identified lane lines may be missing, broken, or stuck together. For example, using only the YOLOPv2 algorithm for lane line identification results in lane line sticking together, leading to inaccurate lane line recognition. Summary of the Invention

[0004] The purpose of this application is to provide a lane line recognition method, which filters out noise points with abrupt changes in direction in the lane line pixel set through directional constraints, and promotes the separation of pixels in different directions, making it easier for pixels with basically the same direction to cluster together to generate lane line pixel clusters, thereby improving the accuracy of lane line recognition.

[0005] In a first aspect, embodiments of this application provide a lane line recognition method, the method comprising: acquiring a target vehicle image frame to be processed; wherein the image frame to be processed includes lane line basic recognition results; performing feature extraction on the image frame to be processed to obtain a set of pixels of the lane line basic recognition results; wherein each pixel in the pixel set includes coordinate values ​​and orientation angles; clustering the pixels based on coordinate values ​​and orientation angles to generate a lane line pixel cluster; and recognizing lane lines based on the lane line pixel clusters.

[0006] In this embodiment, after obtaining the basic lane line recognition result of the target vehicle, the lane line basic recognition result is further processed. Specifically, feature extraction is performed on the image frame to be processed to obtain the pixel set of the lane line basic recognition result. The pixels are clustered according to the coordinate value and orientation angle of each pixel in the pixel set to generate lane line pixel clusters for lane line recognition. During the clustering process, not only the spatial distance of the pixels but also the spatial orientation of the pixels is considered when generating lane line pixel clusters. Noise points with abrupt changes in orientation are filtered out by orientation constraints, which promotes the separation of pixels with different orientations. This makes it easier for pixels with basically the same orientation to cluster together, thereby reducing the influence of spatially adjacent but differently oriented pixels on the recognition result and improving the accuracy of lane line recognition.

[0007] In some embodiments, clustering pixels based on coordinate values ​​and orientation angles to generate lane line pixel clusters includes: for each pixel in the pixel set, calculating a first comprehensive distance between the current pixel and other pixels in the pixel set based on the coordinate values ​​and orientation angles of the current pixel; if the number of other pixels whose first comprehensive distance is not greater than a preset distance threshold is greater than or equal to a core point threshold, then the current pixel is taken as a core point, and other pixels whose comprehensive distance is not greater than the preset distance threshold are taken as neighborhood points of the core point; if it is determined that there are unvisited target pixels among the neighborhood points, then cluster expansion is performed based on the target pixels to generate lane line pixel clusters.

[0008] In this embodiment, a first comprehensive distance between two pixels is calculated by combining the coordinates and orientation angle of the pixels, and the neighborhood relationship between the two pixels is determined based on this first comprehensive distance. In this process, by utilizing the directional features of the pixels for clustering, pixels with different orientations can be separated, ensuring that the pixels in the final clustered lane line pixel clusters are generally in the same direction, thus improving the accuracy of lane line recognition.

[0009] In some embodiments, if it is determined that there are unvisited target pixels among the neighboring points, then cluster expansion is performed based on the target pixels to generate a lane line pixel cluster, including: incorporating the target pixel into the cluster to which the core point belongs, and calculating a second comprehensive distance between the target pixel and other pixels in the pixel set based on the coordinate value and orientation angle of the target pixel; and calculating the orientation difference between the target pixel and the core point based on the orientation angle of the target pixel and the orientation angle of the core point; if it is determined that the number of other pixels whose second comprehensive distance is not greater than a preset distance threshold is greater than or equal to a core point threshold, and the orientation difference is less than or equal to a preset orientation angle threshold, then the neighboring points of the target pixel are used as the neighboring points of the core point to generate new neighboring points of the core point; if it is determined that there are unvisited new target pixels among the new neighboring points, then cluster expansion is performed based on the new target pixels until there are no unvisited pixels among the latest neighboring points of the core point, thereby generating a lane line pixel cluster.

[0010] In this embodiment, the lane line pixel cluster is expanded by using the density of pixels that satisfy a preset relationship with the second comprehensive distance. During this process, by recursively incorporating qualified neighboring points, the lane line pixel cluster is gradually expanded, reducing the possibility of a single lane line being identified as multiple lane lines, improving the completeness of lane line recognition, and thus enhancing the accuracy of lane line recognition.

[0011] In some embodiments, the method further includes: if it is determined that there are no unvisited target pixels in the neighborhood, then generating a lane line pixel cluster based on the core point and the neighborhood points.

[0012] In this embodiment, when cluster expansion is not possible, lane line pixel clusters are generated based on core points and neighboring points to improve the integrity and independence of lane line recognition, thereby improving the accuracy of lane line recognition.

[0013] In some embodiments, for each pixel in the pixel set, calculating its first comprehensive distance with other pixels in the pixel set based on the current pixel's coordinates and orientation angle includes: for each pixel in the pixel set, calculating its first comprehensive distance with other pixels in the pixel set according to the formula... Calculate its first comprehensive distance from other pixels in the pixel set; where, Indicates the first overall distance. This represents the Euclidean distance between the current pixel and other pixels. This indicates the angular difference between the current pixel and other pixels. and Indicates the weight.

[0014] In this embodiment of the application, when calculating the first comprehensive distance based on coordinate values ​​and direction angles, different weights are assigned to the Euclidean distance and direction angle difference between two pixels, thereby balancing the different influences of the two factors on the first comprehensive distance, improving the accuracy of the calculation of the first comprehensive distance, and thus improving the accuracy of the generated lane line pixel cluster.

[0015] In some embodiments, after identifying lane lines based on lane line pixel clusters, the method further includes: generating a lane line image frame corresponding to the image frame to be processed based on the lane line pixel clusters; performing optical flow tracking on the lane line image frame based on each key point of the previous historical image frame of the image frame to be processed, to obtain the coordinates of each key point in the lane line image frame and the motion vector of each key point; determining the tracking confidence based on the coordinates of the key points and the motion vectors of the key points; and performing compensation calibration on the lane lines based on the tracking confidence to obtain the corrected lane lines.

[0016] In this embodiment, considering the possibility of lane lines being obscured, in order to further improve the accuracy of lane line recognition, after recognizing the lane lines based on the lane line pixel clusters, a lane line image frame is generated based on the lane line pixel clusters. Optical flow tracking is then performed on the lane line image frame based on key points to perform occlusion compensation calibration on the recognized lane lines, thereby further improving the accuracy of lane line recognition.

[0017] In some embodiments, determining the tracking confidence level based on the coordinates and motion vectors of key points includes: determining two adjacent key points belonging to the same lane line based on their coordinates; and determining the tracking confidence level by the deviation of the motion vectors of the two adjacent key points.

[0018] The embodiments of this application determine the tracking confidence level by the deviation of the motion vectors of adjacent key points, thereby improving the accuracy of tracking confidence level determination and thus improving the accuracy of optical flow tracking.

[0019] In some embodiments, the lane lines are compensated and calibrated based on the tracking confidence to obtain the corrected lane lines, including: if the tracking confidence is determined to be greater than a preset confidence threshold, then curve fitting is performed based on the coordinates of each key point in the lane line image frame to compensate and calibrate the lane lines to obtain the corrected lane lines.

[0020] In this embodiment of the application, when the tracking confidence level is greater than the preset confidence threshold, it indicates that the optical flow tracking result is reliable. At this time, curve fitting is performed based on the optical flow tracking result to compensate and calibrate the lane line, thereby improving the accuracy of lane line recognition.

[0021] In some embodiments, the lane lines are compensated and calibrated based on the tracking confidence to obtain the corrected lane lines. The method further includes: if the tracking confidence is determined to be less than or equal to a preset confidence threshold, the coordinates of the key points in the lane line image frames are predicted based on the historical coordinates of each key point in the most recent N historical lane line image frames to obtain predicted coordinates; and curve fitting is performed based on the predicted coordinates to compensate and calibrate the lane lines to obtain the corrected lane lines.

[0022] In this embodiment of the application, considering that optical flow tracking may produce instantaneous errors due to image noise, making the tracking results unreliable, when the tracking confidence is less than or equal to the confidence threshold, historical cache is used for prediction, and curve fitting is performed based on the prediction results to maintain the spatial continuity of lane lines and avoid the phenomenon of lane lines suddenly disappearing or jumping, thereby improving the accuracy of lane line recognition.

[0023] In some embodiments, after acquiring the image frame to be processed of the target vehicle, the method further includes: determining the target recognition region of the image to be processed based on the target vehicle; performing feature extraction on the image frame to be processed to obtain a set of pixels for the basic recognition result of the lane line, including: performing feature extraction on the target recognition region to obtain a set of pixels for the basic recognition result of the lane line.

[0024] In this embodiment, considering that vehicle parameters differ for each model (e.g., different vehicle heights), the coverage area of ​​the camera varies, leading to differences in the acquired images. Furthermore, lane lines at a distance tend to merge, hindering lane line recognition. Therefore, the target recognition region of the image to be processed is determined based on the target vehicle, and features are extracted from the target recognition region to obtain a set of pixels. This reduces errors caused by blurred lane lines at a distance and improves the accuracy of lane line recognition.

[0025] In a second aspect, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, are capable of performing the method steps of any embodiment of the first aspect.

[0026] Thirdly, embodiments of this application provide a computer-readable storage medium, including: computer program instructions stored on the computer-readable storage medium, wherein the computer program instructions are executed by a processor to perform the method steps of any embodiment of the first aspect.

[0027] Fourthly, embodiments of this application provide a computer program product, including: computer program instructions, which are executed by a processor to perform the method steps of any embodiment of the first aspect.

[0028] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A schematic flowchart illustrating a lane line recognition method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the electronic device structure provided in an embodiment of this application. Detailed Implementation

[0031] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0032] It should be noted that all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0033] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0034] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0035] Lane detection plays a crucial role in the field of autonomous driving and is an important component of the autonomous driving environment perception module. Currently, there are two main methods for lane detection: traditional image processing methods and deep learning methods.

[0036] Traditional image processing methods include those based on color thresholding, edge detection, and Hough transform. Deep learning methods include lane line recognition methods such as LaneNet (based on instance segmentation), UFLDv2 (based on row classification), CLRnet (based on object detection anchor boxes), and YOLOPv2 (based on semantic segmentation).

[0037] However, traditional image processing methods and deep learning methods suffer from issues such as missed detections when there are a large number of lane lines, and the identified lane lines may be missing, broken, or stuck together. For example, using only the YOLOPv2 algorithm for lane line identification results in lane line sticking together, leading to inaccurate lane line recognition.

[0038] To address the aforementioned issues, this application provides a lane line recognition method that filters out noise points with abrupt changes in direction from the lane line pixel set through directional constraints, and promotes the separation of pixels in different directions, making it easier for pixels with essentially the same direction to cluster together to generate lane line pixel clusters, thereby improving the accuracy of lane line recognition.

[0039] Figure 1This is a flowchart illustrating a lane line recognition method provided in an embodiment of this application. It is understood that the lane line recognition method provided in this embodiment can be applied to terminal devices (also known as electronic devices) and servers; the terminal device can specifically be a smartphone, tablet computer, computer, personal digital assistant (PDA), etc.; the server can specifically be an application server or a web server. To facilitate understanding of the technical solution provided in this embodiment, the application scenarios of the lane line recognition method provided in this embodiment are described below using a server as the execution subject.

[0040] like Figure 1 As shown, the method includes: Step S101: Obtain the image frame to be processed for the target vehicle; wherein, the image frame to be processed includes the lane line basic recognition result.

[0041] Before acquiring the image frames to be processed from the target vehicle, the video captured by the target vehicle's dashcam is obtained. The acquired video is then preprocessed using traditional image processing methods or deep learning methods to obtain the image frames to be processed.

[0042] For example, the acquired video is input into the YOLOPv2 algorithm. After processing by the YOLOPv2 algorithm, the output is a processing image frame including lane lines. In the processing image frame, lane lines are marked in red and road surface is marked in green.

[0043] Although the YOLOPv2 algorithm yielded lane line recognition results, since all lanes were marked with the same red color, the result only separated the lanes and the road surface, and could not distinguish the lane lines in detail.

[0044] Therefore, further processing of the recognition result is required. Based on this, the image frame to be processed includes the lane line basic recognition result.

[0045] It should be understood that traditional image processing methods or deep learning methods are not limited to the algorithms listed above.

[0046] Step S102: Perform feature extraction on the image frame to be processed to obtain a set of pixels for the basic recognition result of lane lines; wherein, each pixel in the set of pixels includes coordinate values ​​and orientation angles.

[0047] Based on the above, it can be seen that lane lines cannot be accurately distinguished from the image frame to be processed. In order to further improve the accuracy of lane line recognition, a secondary recognition is performed on the image frame to be processed.

[0048] Furthermore, since lane lines themselves are composed of continuous pixels in an image, in practice, edge detection or gradient transformation methods are used to extract features from the image frame to be processed, obtaining a set of pixels representing the basic lane line recognition result. Each pixel in this set includes coordinate values ​​and a direction angle.

[0049] The coordinates of a pixel are its horizontal and vertical coordinates in a two-dimensional image, i.e., (x, y) coordinates.

[0050] The orientation angle of a pixel refers to the edge orientation angle. Specifically, it refers to the tangent direction of the image edge at that pixel, which is perpendicular to the direction of the fastest change in grayscale value at that point (i.e., the gradient direction).

[0051] Step S103: Cluster the pixels based on coordinate values ​​and orientation angles to generate lane line pixel clusters.

[0052] Step S104: Identify lane lines based on lane line pixel clusters.

[0053] In the above implementation process, it is considered that if only coordinate values ​​are used for clustering, pixels from different lane lines may be merged due to spatial proximity, leading to inaccurate recognition. The orientation angle of a pixel measures the orientation of the pixel in the image and can reflect the physical direction of the pixel.

[0054] Therefore, in this embodiment of the application, after obtaining the set of pixels for lane line basic recognition, the pixels are clustered based on coordinate values ​​and orientation angles.

[0055] In one implementation, the overall distance between pixels is calculated by combining the coordinate values ​​and orientation angles of the pixels. The pixels are then clustered based on this overall distance to generate lane line pixel clusters. For a detailed implementation process, please refer to the following embodiment.

[0056] In another implementation, the initial distance between pixels can be calculated based on coordinate values, and preliminary clustering can be performed based on the initial distance. Then, the clusters after preliminary clustering can be filtered using the orientation angle of the pixels to remove pixels with abrupt changes in orientation, so as to generate lane line pixel clusters.

[0057] After obtaining the lane line pixels, the lane lines are identified based on the lane line pixel clusters.

[0058] It should be noted that one lane line pixel cluster corresponds to one lane line; therefore, the lane line pixel cluster is an instance-level pixel cluster.

[0059] In this embodiment, after obtaining the basic lane line recognition result of the target vehicle, the lane line basic recognition result is further processed. Specifically, feature extraction is performed on the image frame to be processed to obtain the pixel set of the lane line basic recognition result. The pixels are clustered according to the coordinate value and orientation angle of each pixel in the pixel set to generate lane line pixel clusters for lane line recognition. During the clustering process, not only the spatial distance of the pixels but also the spatial orientation of the pixels is considered when generating lane line pixel clusters. Noise points with abrupt changes in orientation are filtered out by orientation constraints, which promotes the separation of pixels with different orientations. This makes it easier for pixels with basically the same orientation to cluster together, thereby reducing the influence of spatially adjacent but differently oriented pixels on the recognition result and improving the accuracy of lane line recognition.

[0060] In some embodiments, clustering pixels based on coordinate values ​​and orientation angles to generate lane line pixel clusters includes: for each pixel in the pixel set, calculating a first comprehensive distance between the current pixel and other pixels in the pixel set based on the coordinate values ​​and orientation angles of the current pixel; if the number of other pixels whose first comprehensive distance is not greater than a preset distance threshold is greater than or equal to a core point threshold, then the current pixel is taken as a core point, and other pixels whose comprehensive distance is not greater than the preset distance threshold are taken as neighborhood points of the core point; if it is determined that there are unvisited target pixels among the neighborhood points, then cluster expansion is performed based on the target pixels to generate lane line pixel clusters.

[0061] In the above implementation process, each pixel in the pixel set is traversed. If the current pixel is an unvisited pixel p, its neighboring pixels p_neighbors are searched first by the first comprehensive distance (a weighted distance combining spatial distance and directional difference). If the number of pixels in the neighborhood is greater than or equal to the core point threshold MinPts, then the current pixel p is a "core point" and the cluster expansion logic is triggered; otherwise, the current pixel p is marked as a noise point.

[0062] Specifically, the Euclidean distance between the current pixel p and any other pixel in the pixel set is calculated based on the coordinates of the current pixel p. The angle difference between the current pixel p and any other pixel in the pixel set is calculated based on the orientation angle of the current pixel p, and the absolute value of the angle difference is taken.

[0063] Then, according to the formula Calculate the first comprehensive distance between the current pixel p and all other pixels in the pixel set; where, Indicates the first overall distance. This represents the Euclidean distance between the current pixel p and other pixels. This represents the angular difference between the current pixel p and other pixels. and Indicates weight, .

[0064] The formula for calculating the Euclidean distance between the current pixel p and other pixels is: ,in, This represents the x and y coordinates of the current pixel p. This represents the x and y coordinates of any other pixel.

[0065] The formula for calculating the angle difference between the current pixel p and other pixels is: ,in, This represents the absolute value of the angular difference between the current pixel p and other pixels. This represents the orientation angle of the current pixel p. This represents the orientation angle of any other pixel. To handle the periodicity of angles, the orientation angle difference is limited to [0, π]. The orientation angle difference between the current pixel p and other pixels is: .

[0066] After obtaining the first comprehensive distance between the current pixel p and other pixels in the pixel set, the first comprehensive distance is compared with a preset distance threshold to determine whether other pixels are neighbors of the current pixel p.

[0067] Furthermore, the formation of a cluster requires a starting point as the core for cluster expansion. Therefore, it is also necessary to compare the number of neighboring points of the current pixel p with the preset core point threshold MinPts to further determine whether the current pixel p is a core point.

[0068] The preset distance threshold is a pre-set value, which can be adaptively set according to actual conditions; this application does not impose specific limitations on it. The core point threshold is also a pre-set value, which can be adaptively set according to actual conditions; this application does not impose specific limitations on it.

[0069] If the number of other pixels whose first comprehensive distance is not greater than the preset distance threshold is greater than or equal to the core point threshold MinPts, then the current pixel p is taken as the core point, and the other pixels whose comprehensive distance is not greater than the preset distance threshold are taken as the neighboring points of the core point.

[0070] If it is determined that there are unvisited target pixels in the neighborhood, then cluster expansion is performed based on the target pixels to recursively include eligible neighborhood pixels, gradually increasing the size of the cluster to generate a lane line pixel cluster.

[0071] In this embodiment, a first comprehensive distance between two pixels is calculated by combining the coordinates and orientation angle of the pixels, and the neighborhood relationship between the two pixels is determined based on this first comprehensive distance. In this process, by utilizing the directional features of the pixels for clustering, pixels with different orientations can be separated, ensuring that the pixels in the final clustered lane line pixel clusters are generally in the same direction, thus improving the accuracy of lane line recognition.

[0072] In some embodiments, if it is determined that there are unvisited target pixels among the neighboring points, then cluster expansion is performed based on the target pixels to generate a lane line pixel cluster, including: incorporating the target pixel into the cluster to which the core point belongs, and calculating a second comprehensive distance between the target pixel and other pixels in the pixel set based on the coordinate value and orientation angle of the target pixel; and calculating the orientation difference between the target pixel and the core point based on the orientation angle of the target pixel and the orientation angle of the core point; if it is determined that the number of other pixels whose second comprehensive distance is not greater than a preset distance threshold is greater than or equal to a core point threshold, and the orientation difference is less than or equal to a preset orientation angle threshold, then the neighboring points of the target pixel are used as the neighboring points of the core point to generate new neighboring points of the core point; if it is determined that there are unvisited new target pixels among the new neighboring points, then cluster expansion is performed based on the new target pixels until there are no unvisited pixels among the latest neighboring points of the core point, thereby generating a lane line pixel cluster.

[0073] In the above implementation process, cluster expansion starts from the core point p and gradually expands the cluster by recursively incorporating neighboring points that meet the conditions.

[0074] The cluster expansion logic is as follows: Initialize the cluster as a set including the core point p, and label the cluster ID of p.

[0075] Traverse the neighborhood points of the core point p: For unvisited target pixel q, first mark target pixel q as visited and include it in the cluster of core point p. Then search the neighborhood points q_neighbors of target pixel q, and calculate the second comprehensive distance between target pixel q and other pixels in the pixel set. The calculation process is the same as the process of calculating the first comprehensive distance of p described above, and will not be repeated here.

[0076] If the target pixel q is itself a core point (the number of q_neighbors ≥ MinPts) and satisfies the direction constraint (see "Direction Constraint Check" below), then the neighboring points of the target pixel q are also included in the neighboring points p_neighbors of the core point p, so that p_neighbors is updated to obtain a new p_neighbors_1, realizing the recursive growth of the cluster. It should be noted that regardless of whether the target pixel q is a core point, as long as it has not been assigned to another lane line pixel cluster, it is added to the current core point p's cluster.

[0077] In the new p_neighbors_1, find the next unvisited new target pixel and perform cluster expansion logic on the new target pixel until there are no unvisited target pixels in the latest neighborhood of the core point p, and generate a lane line pixel cluster containing all pixels that meet the conditions.

[0078] Directional constraint checks ensure that pixels within a lane line pixel cluster have the same orientation. The logic for the directional constraint check is as follows: Calculate the directional difference between the core point p and the target pixel q based on the directional angle difference between them. Only when the orientation difference between the core point p and the target pixel q is... Only when the value is less than or equal to the preset orientation angle threshold theta_thresh, is it allowed to include the neighborhood points of the target pixel q into the neighborhood points of the core point p.

[0079] By checking the orientation constraints, the orientation of all pixels within the lane line pixel cluster is forced to remain within the allowable deviation range, avoiding the inclusion of points with excessively different orientations, such as points in opposing lanes, into the same lane line pixel cluster, thereby improving the accuracy of the lane line pixel cluster.

[0080] In this embodiment, the lane line pixel cluster is expanded by using the density of pixels that satisfy a preset relationship with the second comprehensive distance. During this process, by recursively incorporating qualified neighboring points, the lane line pixel cluster is gradually expanded, reducing the possibility of a single lane line being identified as multiple lane lines, improving the completeness of lane line recognition, and thus enhancing the accuracy of lane line recognition.

[0081] In some embodiments, the method further includes: if it is determined that there are no unvisited target pixels in the neighborhood, then generating a lane line pixel cluster based on the core point and the neighborhood points.

[0082] In this embodiment, there are no unvisited target pixels in the neighborhood, indicating that all neighborhood points have been included in the corresponding lane line pixel cluster. Therefore, when cluster expansion is not possible, lane line pixel clusters are generated based on the core point and neighborhood points, improving the completeness and independence of lane line recognition, thereby improving the accuracy of lane line recognition.

[0083] To further understand the process of generating lane line pixel clusters, sample code is provided below: # Input parameter definition points: A set of points (each point contains coordinates x, y and direction angle theta). eps: Neighborhood radius (preset distance threshold) MinPts: Core Point Threshold alpha: Euclidean distance weight (0.0-1.0) beta: Directional difference weight (beta = 1 - alpha) theta_thresh: Threshold for allowable directional difference (in radians) # Auxiliary function: Comprehensive distance calculation def combined_distance(p1, p2): # Euclidean distance term, where p1 and p2 represent two pixels, and x and y represent the horizontal and vertical coordinates. euclidean_dist = sqrt((p1.x - p2.x) 2 + (p1.y - p2.y) 2 ) # Directional difference term (handling angular periodicity) angle_diff = abs(p1.theta - p2.theta) / / angle_diff represents the absolute value of the angle difference between the directions of two pixels p1 and p2. / / Guarantee [0, pi] # Composite distance, representing a weighted linear combination of Euclidean distance and directional distance. return alpha × euclidean_dist + beta × direction_diff # Main Algorithm Flow def DirectionConstrainedDBSCAN(): cluster_id = 0 visited = empty_set() For p in points: if p in visited: continue visited.add(p) neighbors = find_neighbors(p, eps) # Neighbor search based on combined_distance if len(neighbors) <MinPts: mark_as_noise(p) else: # Create new clusters and expand cluster = expand_cluster(p, neighbors, cluster_id) clusters.append(cluster) cluster_id += 1 # Cluster Extension Logic def expand_cluster(p, neighbors, cluster_id): cluster = [p] p.cluster = cluster_id for q in neighbors: if q is not visited: visited.add(q) q_neighbors = find_neighbors(q, eps) # Direction constraint check if direction_diff(q.theta, p.theta)<= theta_thresh: if len(q_neighbors)>= MinPts: neighbors += q_neighbors # Merge only neighbors with the same direction. if q.cluster is None: q.cluster = cluster_id cluster.append(q) return cluster Based on the above code, it can be seen that through the above process, the pixel set of the basic recognition result of lane lines is divided into multiple lane line pixel clusters and noise points.

[0084] Each lane line pixel cluster consists of pixels that are spatially close (overall distance ≤ preset distance threshold eps) and have the same orientation (orientation difference ≤ preset orientation angle threshold theta_thresh). Spatial proximity ensures the density of lane line pixel clusters, while orientation constraints ensure their directionality. Noise points are spatially isolated or have orientations that differ significantly from their surroundings and are not assigned to any cluster.

[0085] The final result is multiple lane line pixel clusters, each representing a lane line.

[0086] In some embodiments, after identifying lane lines based on lane line pixel clusters, the method further includes: generating a lane line image frame corresponding to the image frame to be processed based on the lane line pixel clusters; performing optical flow tracking on the lane line image frame based on each key point of the previous historical image frame of the image frame to be processed, to obtain the coordinates of each key point in the lane line image frame and the motion vector of each key point; determining the tracking confidence based on the coordinates of the key points and the motion vectors of the key points; and performing compensation calibration on the lane lines based on the tracking confidence to obtain the corrected lane lines.

[0087] In the above implementation process, it is considered that lane lines may be obscured, leading to errors in lane line recognition, such as identifying two lanes when the lane line is obscured by a vehicle. Therefore, to further improve the accuracy of lane line recognition, after recognizing the lane line based on the lane line pixel clusters, occlusion compensation is needed for the recognition results.

[0088] Specifically, after obtaining the lane line pixel clusters, a lane line image frame to be processed is generated based on the lane line pixel clusters. Key points are selected, and optical flow tracking is performed on the lane line image frame based on the key points to obtain the position coordinates of the key points in the current lane line image frame.

[0089] The optical flow algorithm for optical flow tracking of lane line image frames based on key points can be either the LK optical flow algorithm or the TV-L1 optical flow algorithm; this application does not make any specific limitation on this.

[0090] Key points refer to points that effectively capture the complete geometry and motion state of a lane. For example, key points include, but are not limited to, curve vertices, lane line endpoints, abrupt edge changes, and uniformly sampled points; among these, Crest of a curve: The point where the curvature of the lane line is greatest (such as the most convex / concave point at the turn).

[0091] Lane line endpoints: The start and end points of the lane lines in the current frame (points near the vehicle or the edge of the field of view).

[0092] Edge abrupt change points: locations where the lane line width or color changes (such as the endpoints of dashed lane lines).

[0093] Uniform sampling points: Points at fixed intervals (e.g., 50cm) on a straight line segment, used to maintain tracking continuity.

[0094] In the specific implementation process, the first frame of the image to be processed is processed using the image processing library OpenCV or other image processing methods to obtain key points, and optical flow tracking is performed on subsequent frames based on the key points of the first frame of the image to be processed.

[0095] It should be noted that, to improve the accuracy of key points, the key points should be dynamically updated during algorithm operation. For example, based on tracking confidence, the number of key points, and scene changes, the detection results of the current frame should be used periodically or triggered to resample and supplement key points to ensure the integrity and accuracy of the tracked target.

[0096] Since optical flow tracing tracks the positions of known key points from the previous frame in the current frame, the set of key points of the lane line image frame corresponding to the previous historical image frame is obtained before optical flow tracing is performed on the current lane line image frame.

[0097] When performing optical flow tracking, optical flow tracking is performed on each key point in the key point set of the lane line image frame corresponding to the previous historical image frame to obtain the coordinates of each key point in the lane line image frame and the motion vector of each key point.

[0098] Here, the motion vector refers to the displacement (dx, dy) of the keypoint from the previous frame to the current lane line image frame. The coordinates of the keypoint in the lane line image frame represent the position of the keypoint in the current lane line image frame (x, y) = (x_prev + dx, y_prev + dy), where (x_prev, y_prev) represents the position of the keypoint in the lane line image frame corresponding to the previous historical image frame.

[0099] After obtaining the coordinates and motion vectors of the keypoints in the current lane image frame, the tracking confidence is determined based on these coordinates and motion vectors. The tracking confidence is used to quantify the reliability of the tracking results.

[0100] In one implementation, the tracking confidence is determined based on the coordinates and motion vectors of key points, including: determining two adjacent key points belonging to the same lane line based on their coordinates; and determining the tracking confidence by the deviation of the motion vectors of the two adjacent key points.

[0101] Specifically, based on coordinates, two adjacent key points belonging to the same lane line are determined, and the Euclidean distance or Manhattan distance between the two adjacent key points is calculated based on the coordinate values ​​to measure the deviation between the two adjacent key points.

[0102] If the deviation between two adjacent key points is less than or equal to a preset threshold, the tracking confidence is high; otherwise, the tracking confidence is low. The calculation method for tracking confidence can be adaptively set according to actual conditions, as long as the relationship of "the deviation between two adjacent key points is less than or equal to a preset threshold, the tracking confidence is high; otherwise, the tracking confidence is low" is satisfied.

[0103] Finally, based on the tracking confidence level, the lane lines are compensated and calibrated to obtain the corrected lane lines.

[0104] This application's embodiments generate lane line image frames based on lane line pixel clusters, and perform optical flow tracking on the lane line image frames based on key points to perform occlusion compensation calibration on the identified lane lines, thereby further improving the accuracy of lane line recognition. Furthermore, the tracking confidence level is determined by the deviation of the motion vectors of adjacent key points, improving the accuracy of tracking confidence level determination, thereby further improving the accuracy of optical flow tracking.

[0105] In some embodiments, the lane lines are compensated and calibrated based on the tracking confidence to obtain the corrected lane lines, including: if the tracking confidence is determined to be greater than a preset confidence threshold, then curve fitting is performed based on the coordinates of each key point in the lane line image frame to compensate and calibrate the lane lines to obtain the corrected lane lines.

[0106] If the tracking confidence is determined to be less than or equal to the preset confidence threshold, the coordinates of the key points in the lane line image frames are predicted based on the historical coordinates of each key point in the most recent N historical lane line image frames to obtain the predicted coordinates; curve fitting is performed based on the predicted coordinates to compensate and calibrate the lane lines to obtain the corrected lane lines.

[0107] In the above implementation process, when the tracking confidence is greater than the preset confidence threshold, it indicates that the optical flow tracking result is reliable. At this time, curve fitting is performed based on the optical flow tracking result to compensate and calibrate the lane line, thereby improving the accuracy of lane line recognition.

[0108] Meanwhile, each lane tracker maintains a sliding window history cache, which stores features of key points in the most recent N historical image frames, such as key point coordinates and motion trends. The sliding window history cache is a fixed-length queue, for example, a queue that can cache 5 or 10 image frames.

[0109] For example, if the window length of the sliding window history cache is 5, the key point data from frame t-4 to frame t is cached.

[0110] When tracking is reliable, the coordinates of key points obtained from tracking are cached in the sliding window history cache for future updates. This ensures that the sliding window history cache incorporates the latest lane line status, guaranteeing the timeliness of historical information and avoiding delays and misidentifications caused by outdated data.

[0111] If the queue length of the sliding window history cache exceeds the window limit (e.g., 5 frames), the historical data of the earliest frame is deleted to maintain a fixed window length.

[0112] When tracking is unreliable, the coordinates of each keypoint in the lane line image frame are predicted based on the historical coordinates of each keypoint in the most recent N frames of historical lane line image frames. Based on the predicted coordinates, curve fitting is performed to compensate and calibrate the lane line to obtain the corrected lane line.

[0113] Specifically, the key point coordinates stored in the sliding window history cache are obtained, and the motion trajectory of the key points of each lane line is fitted (e.g., linear fitting or quadratic polynomial fitting) to obtain a prediction model of the key point displacement changing over time, such as: x(t) = at + b, y(t) = ct² + d×t + e), where x(t) represents the x-coordinate of the predicted key point and y(t) represents the y-coordinate of the predicted key point.

[0114] Based on the aforementioned prediction model, the coordinates of key points in the lane line image frame are predicted to obtain the predicted coordinates. Curve fitting is then performed based on the predicted coordinates to compensate and calibrate the lane lines, resulting in the corrected lane lines.

[0115] The embodiments of this application take into account that optical flow tracking can provide sub-pixel level motion estimation of adjacent frames (strong real-time performance), but it is prone to failure in scenarios such as occlusion and changes in lighting; historical prediction is based on temporal continuity (strong stability), but there is a risk of lag.

[0116] Therefore, a "multi-source information optimization mechanism" is provided: real-time optical flow tracking data is used as the primary reference, supplemented by historical time-series predictions, and the dynamic switching between the two is achieved by judging the tracking confidence level. This ensures that even in complex scenarios, such as occlusion, noise, and rapid movement, the real-time performance of optical flow is preserved, while historical predictions are used to compensate for deficiencies when they fail, providing a reliable data foundation for subsequent lane line fitting and display.

[0117] Furthermore, single-frame optical flow tracking may introduce instantaneous errors due to image noise (such as keypoint jumps), while the temporal predictions from the historical cache have a smoothing effect (similar to a moving average). By using real-time results when reliable and historical predictions when unreliable, single-frame noise can be filtered out, avoiding the accumulation of tracking drift.

[0118] In addition, optical flow tracking fails when lane lines are temporarily obscured (e.g., by other vehicles). In this case, historical trend prediction can maintain the spatial continuity of lane lines and avoid the phenomenon of "lane lines suddenly disappearing or changing" in the output results.

[0119] Therefore, when the tracking confidence is less than or equal to the confidence threshold, historical cache is used for prediction, and curve fitting is performed based on the prediction results to maintain the spatial continuity of lane lines and avoid the phenomenon of lane lines suddenly disappearing or changing, thereby improving the accuracy of lane line recognition.

[0120] After compensating and calibrating the lane lines to obtain the corrected lane lines, the total number of lane lines in the image frame to be processed is counted, and the final total number of lane lines is output.

[0121] In one implementation, if the tracking is reliable, the number of lane lines in the current lane line image frame is counted as the final total number of lane lines.

[0122] In one implementation, if tracking is unreliable, the total number of lane lines in the most recent N historical image frames is obtained, and the mode of the total number of lane lines in the most recent N historical image frames is taken as the final total number of lane lines.

[0123] After compensating and calibrating the lane lines to obtain the corrected lane lines, the slope of each lane line is calculated and sorted, and the current lane is determined based on the sorting result. For example, the slopes of the four lane lines are (-2, -1, 1, 2). Due to the viewing angle, the current lane is between lane lines -1 and 1, that is, between the two lane lines whose slope changes from negative to positive.

[0124] In some embodiments, after acquiring the image frame to be processed of the target vehicle, the method further includes: determining the target recognition region of the image to be processed based on the target vehicle; performing feature extraction on the image frame to be processed to obtain a set of pixels for the basic recognition result of the lane line, including: performing feature extraction on the target recognition region to obtain a set of pixels for the basic recognition result of the lane line.

[0125] In the above implementation process, considering the different vehicle parameters of each model, such as different vehicle heights, the coverage area of ​​the camera varies, resulting in differences in the captured images. For example, the area occupied by distant lane lines in the image varies. Furthermore, lane lines in distant locations tend to merge together, which is also not conducive to lane line recognition.

[0126] Therefore, the embodiments of this application customize different target recognition regions according to different vehicle models. Furthermore, since lane lines are below the vehicle, the target recognition region is typically located in the lower part of the image.

[0127] Therefore, when determining the target recognition region of the image to be processed based on the target vehicle, the lower region of the image frame to be processed is usually determined based on the target vehicle, thereby improving the accuracy of lane separation.

[0128] Specifically, the first step is to acquire vehicle parameters, including vehicle model, parking space, wheelbase, and camera installation height. Then, based on these parameters, the target recognition region of the image to be processed is determined.

[0129] In one implementation, the acquired vehicle parameters can be input into a pre-trained deep learning model to automatically determine the target recognition region of the target vehicle.

[0130] In one implementation, a table of vehicle-target recognition regions can be maintained in advance. After obtaining vehicle parameters, the target recognition region of the target vehicle is determined based on a lookup in the table. For example, for sedans, the target recognition region is the area below 60% to 100% of the image frame to be processed; for trucks, the target recognition region is the area below 50% to 100% of the image frame to be processed; and for SUVs, the target recognition region is the area below 55% to 100% of the image frame to be processed.

[0131] Therefore, in this embodiment of the application, the target recognition area of ​​the image to be processed is determined based on the target vehicle, and the feature of the target recognition area is extracted to obtain a set of pixels, thereby reducing the error caused by the blurring of lane lines in the distance and improving the accuracy of lane line recognition.

[0132] In summary, this application generates lane line pixel clusters by combining position and orientation clustering algorithms. This ensures that the generated lane line pixel clusters consider not only spatial distance but also spatial orientation. This promotes the separation of pixels with different orientations, making it easier for pixels with essentially the same orientation to cluster together. This reduces the impact of spatially adjacent but differently oriented pixels on the recognition results, thereby improving the accuracy of lane line recognition.

[0133] Furthermore, after obtaining the lane line pixel clusters, considering that the lane lines may be occluded, a compensation calibration algorithm is used to compensate and calibrate the recognition results based on the lane line pixel clusters, thereby further improving the accuracy of lane line recognition.

[0134] And / or, before recognition, considering that the vehicle parameters of each vehicle type are different (e.g., different vehicle heights), the coverage area of ​​the camera is different, resulting in differences in the acquired images. In addition, lane lines in the distance are prone to sticking together, which is not conducive to lane line recognition. Therefore, the target recognition area is determined based on the target vehicle, thereby reducing the error caused by the blurring of lane lines in the distance and reducing the algorithm complexity. This not only improves the accuracy of lane line recognition but also improves the system resource utilization.

[0135] Figure 2 This is a schematic diagram of the electronic device structure provided in the embodiments of this application, such as... Figure 2 As shown, the electronic device includes a processor 201, a memory 202, and a bus 203; wherein the processor 201 and the memory 202 communicate with each other via the bus 203. The processor 201 is used to call program instructions in the memory 202 to execute the methods provided in the above-described method embodiments.

[0136] Processor 201 can be an integrated circuit chip with signal processing capabilities. The aforementioned processor 201 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0137] The memory 202 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0138] This application provides a computer program product, including: computer program instructions, which are executed by a processor to perform the methods provided in the above-described method embodiments.

[0139] This application provides a computer-readable storage medium, including: computer program instructions stored on the computer-readable storage medium, which, when executed by a processor, perform the methods provided in the above-described method embodiments.

[0140] Computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0141] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0142] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0144] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A lane line recognition method, characterized in that, The method includes: Acquire the image frame to be processed for the target vehicle; wherein, the image frame to be processed includes the basic lane line recognition result; Feature extraction is performed on the image frame to be processed to obtain a set of pixels for the basic recognition result of the lane line; wherein, each pixel in the set of pixels includes coordinate values ​​and orientation angles; Based on the coordinate values ​​and the orientation angle, the pixels are clustered to generate lane line pixel clusters; Lane lines are identified based on the lane line pixel clusters.

2. The method according to claim 1, characterized in that, The step of clustering the pixels based on the coordinate values ​​and the direction angle to generate lane line pixel clusters includes: For each pixel in the pixel set, calculate its first comprehensive distance with other pixels in the pixel set based on the current pixel's coordinates and orientation angle; If it is determined that the number of other pixels whose first comprehensive distance is not greater than the preset distance threshold is greater than or equal to the core point threshold, then the current pixel is taken as the core point, and the other pixels whose comprehensive distance is not greater than the preset distance threshold are taken as the neighboring points of the core point. If it is determined that there are unvisited target pixels in the neighborhood, then cluster expansion is performed based on the target pixels to generate lane line pixel clusters.

3. The method according to claim 2, characterized in that, If it is determined that there are unvisited target pixels among the neighboring points, then cluster expansion is performed based on the target pixels to generate lane line pixel clusters, including: The target pixel is included in the cluster to which the core point belongs, and a second comprehensive distance between the target pixel and other pixels in the pixel set is calculated based on the coordinate value and orientation angle of the target pixel; and the orientation difference between the target pixel and the core point is calculated based on the orientation angle of the target pixel and the orientation angle of the core point. If it is determined that the number of other pixels whose second comprehensive distance is not greater than the preset distance threshold is greater than or equal to the core point threshold, and the directional difference is less than or equal to the preset directional angle threshold, then the neighboring points of the target pixel are taken as the neighboring points of the core point to generate new neighboring points of the core point. If it is determined that there is an unvisited new target pixel in the new neighboring points, then cluster expansion is performed based on the new target pixel until there are no unvisited pixels in the latest neighboring points of the core point, and the lane line pixel cluster is generated.

4. The method according to claim 2, characterized in that, The method further includes: If it is determined that there are no unvisited target pixels in the neighborhood, then the lane line pixel cluster is generated based on the core point and the neighborhood points.

5. The method according to claim 2, characterized in that, The step of calculating the first comprehensive distance between each pixel in the pixel set and other pixels in the pixel set based on the current pixel's coordinates and orientation angle includes: For each pixel in the set of pixels, according to the formula Calculate its first comprehensive distance with other pixels in the pixel set; where, Indicates the first overall distance. This represents the Euclidean distance between the current pixel and other pixels. This indicates the angular difference between the current pixel and other pixels. and Indicates the weight.

6. The method according to claim 1, characterized in that, After identifying lane lines based on the lane line pixel clusters, the method further includes: Generate the lane line image frame corresponding to the image frame to be processed based on the lane line pixel cluster; Based on each key point of the previous historical image frame of the image frame to be processed, optical flow tracking is performed on the lane line image frame to obtain the coordinates of each key point in the lane line image frame and the motion vector of each key point. The tracking confidence level is determined based on the coordinates of the key points and the motion vectors of the key points. Based on the tracking confidence level, the lane lines are compensated and calibrated to obtain the corrected lane lines.

7. The method according to claim 6, characterized in that, The step of determining the tracking confidence level based on the coordinates of the key points and the motion vectors of the key points includes: Based on the coordinates, determine two adjacent key points belonging to the same lane line; The tracking confidence level is determined by the deviation of the motion vectors of two adjacent key points.

8. The method according to claim 6, characterized in that, The step of compensating and calibrating the lane lines based on the tracking confidence to obtain the corrected lane lines includes: If the tracking confidence level is determined to be greater than a preset confidence threshold, then curve fitting is performed based on the coordinates of each key point in the lane line image frame to compensate and calibrate the lane line, thereby obtaining the corrected lane line.

9. The method according to claim 8, characterized in that, The step of compensating and calibrating the lane lines based on the tracking confidence level to obtain the corrected lane lines further includes: If the tracking confidence is determined to be less than or equal to the preset confidence threshold, then based on the historical coordinates of each key point in the most recent N historical lane line image frames, the coordinates of the key point in the lane line image frame are predicted to obtain the predicted coordinates. Curve fitting is performed based on the predicted coordinates to compensate and calibrate the lane lines, thereby obtaining the corrected lane lines.

10. The method according to any one of claims 1-9, characterized in that, After acquiring the image frame to be processed for the target vehicle, the method further includes: The target recognition region of the image to be processed is determined based on the target vehicle; The step of extracting features from the image frame to be processed to obtain the set of pixels for the basic lane line recognition result includes: Feature extraction is performed on the target recognition region to obtain the pixel set of the basic recognition result of the lane line.

Images