A lane line detection and post-processing method, device, equipment, medium and program product

Abstract

The application discloses a lane line detection and post-processing method, device, equipment, medium and program product, and relates to the technical field of vehicles.The method comprises the following steps: performing lane line feature extraction and feature optimization processing on a road image to obtain current lane line information; judging a road intersection scene state in which a current vehicle is located according to the current lane line information, wherein the road intersection scene state comprises a state before a road intersection and a state at the road intersection; performing differentiated deletion and reordering processing on lane lines in different regions according to the road intersection scene state to determine a reserved lane line set; and performing lane line completion processing on the reserved lane line set to obtain a lane line post-processing result.The above technical feature guarantees the integrity and continuity of output lane lines, and improves the stability and accuracy of lane line detection and post-processing under complex road conditions.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a lane line detection and post-processing method, apparatus, equipment, medium, and program product. Background Technology

[0002] In autonomous driving and intelligent driver assistance systems, accurate lane line recognition is one of the fundamental technologies for ensuring safe vehicle operation and achieving efficient navigation.

[0003] However, in complex real-world road conditions, lane line detection faces numerous severe challenges. Factors such as changes in lighting, road wear, dirt coverage, occlusion, and road construction can all negatively impact lane line detection, leading to the loss or discontinuity of some lane line detections. This affects the integrity and reliability of lane line detection results, making it difficult for post-processing solutions that rely on single detection results to stably output high-quality lane line information. In complex scenarios, lane line misalignment or missing lane lines are prone to occur. Summary of the Invention

[0004] This invention provides a lane line detection and post-processing method, apparatus, equipment, medium, and program product, which ensures the integrity and continuity of the output lane lines and improves the stability and accuracy of lane line detection and post-processing under complex road conditions.

[0005] In a first aspect, embodiments of the present invention provide a lane line detection and post-processing method, including: Lane line feature extraction and feature optimization are performed on road images to obtain current lane line information; The current intersection scene state of the vehicle is determined based on the current lane line information. The intersection scene state includes the state before the intersection and the state when the vehicle is at the intersection. Based on the intersection scene status, lane lines in different areas are subject to differentiated deletion and reordering to determine the set of lane lines to be retained. The reserved lane line set is subjected to lane line completion processing to obtain the lane line post-processing result.

[0006] Secondly, embodiments of the present invention provide a lane line detection and post-processing device, comprising: The information extraction module is used to extract lane line features and optimize features from road images to obtain current lane line information. The scene judgment module is used to determine the intersection scene state where the current vehicle is located based on the current lane line information. The intersection scene state includes the state before the intersection and the state when the vehicle is at the intersection. The differentiation processing module is used to perform differentiated deletion and reordering of lane lines in different areas according to the intersection scene status, and determine the set of lane lines to be retained. The lane line completion module is used to perform lane line completion processing on the reserved lane line set to obtain the lane line post-processing result.

[0007] Thirdly, embodiments of the present invention provide an electronic device, including: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor can perform the lane detection and post-processing method provided in the first aspect embodiment described above.

[0008] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute a lane detection and post-processing method provided in the first aspect of the embodiments described above.

[0009] Fifthly, embodiments of the present invention provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements a lane line detection and post-processing method provided in the first aspect of the embodiments described above.

[0010] The technical solution of this invention includes extracting and optimizing lane line features from a road image to obtain current lane line information; determining the intersection scene state where the vehicle is currently located based on the current lane line information, wherein the intersection scene state includes the pre-intersection state and the current intersection state; performing differentiated deletion and reordering processing on lane lines in different areas based on the intersection scene state to determine a set of retained lane lines; and performing lane line completion processing on the set of retained lane lines to obtain the lane line post-processing result. These technical features, through lane line feature extraction and optimization processing on the road image, can obtain stable and reliable current lane line information, and differentiate the processing of lane lines in different areas according to different intersection scene states, avoiding lane line confusion or loss due to changes in scene state, further ensuring the integrity and continuity of the output lane lines. This effectively improves the stability and accuracy of lane line detection and post-processing under complex road conditions.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a lane line detection and post-processing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a lane line detection and post-processing device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Lane markings not only guide vehicle trajectories but also serve as crucial evidence for autonomous driving systems to perceive their surroundings. The quality of lane marking detection directly impacts key functions such as vehicle localization, path planning, speed control, and collision warning. However, in complex real-world road conditions, lane marking detection faces numerous challenges. Factors such as changes in lighting, road wear, dirt accumulation, occlusion, and road construction can all negatively affect lane marking detection, leading to the loss or discontinuity of some lane markings, thus affecting the completeness and reliability of the detection results. Existing lane marking detection methods can accurately extract lane markings under ideal conditions, such as clear weather, clear road conditions, and good lighting. However, these methods often fail to adequately optimize for complex and changing real-world road conditions. Under complex conditions, their accuracy and reliability drop significantly, with frequent issues such as lane marking misjudgment and discontinuity. This makes it difficult to meet the requirements of autonomous driving and intelligent driver assistance systems for accurate lane marking recognition, and ultimately, it cannot guarantee safe driving in complex road conditions. For example, in scenarios involving road wear and tear, lane markings may become discontinuous or misjudged. While deep learning-based methods offer significant improvements in detection accuracy, they are highly dependent on large amounts of labeled data and require substantial computational power for model training.

[0017] Meanwhile, existing technologies for post-processing lane detection results are generally inadequate, lacking post-processing techniques that can be flexibly combined with different lane detection models. This makes it difficult to perform targeted optimization for different scenarios (such as intersections, curves, and straight sections), resulting in poor display and usage performance of lane lines in complex environments. Furthermore, post-processing processes such as lane line clustering and tracking are susceptible to noise interference, leading to issues such as difficulty in guaranteeing the accuracy of lane line clustering results and insufficient robustness in lane line tracking, affecting the stability and consistency of lane lines.

[0018] Therefore, embodiments of the present invention provide a lane detection and post-processing solution that can provide high efficiency, accuracy and robustness under complex road conditions.

[0019] In one embodiment, Figure 1 This is a flowchart of a lane line detection and post-processing method provided by an embodiment of the present invention. This embodiment can be applied to the situation of detecting and post-processing lane lines. The method can be executed by a lane line detection and post-processing device, which can be implemented in hardware and / or software.

[0020] like Figure 1 As shown, the method includes: S101. Perform lane line feature extraction and feature optimization processing on the road image to obtain the current lane line information.

[0021] In this embodiment, the road image can be understood as an image of the road scene in front of the vehicle, captured by an onboard camera. The current lane line information can be understood as a data set representing the position and shape of lane lines in the current frame, obtained after feature extraction and optimization. The lane line information includes the length attribute, longitudinal position attribute, and geometric relationship attribute of each lane line. The length attribute represents the effective length of the lane line, the longitudinal position attribute represents the location of the lane line in the road image, and the geometric relationship attribute represents the curvature and parallelism of the lane line.

[0022] Specifically, road images are input into a deep learning model, generating multi-scale feature maps through convolutional and pooling layers. These maps are then converted into confidence values ​​for each lane line point via fully connected layers. After filtering lane line points using a confidence threshold, the selected lane line points with high confidence are divided into multiple lane line point sets based on spatial connectivity. Feature normalization is performed on the lane line points within each set. All normalized lane line points are then clustered using K-Means clustering. Averaging is performed along the Y-axis based on the model output bias to smooth the lane lines. Lane lines with similar endpoints are connected based on the cosine similarity of the residual endpoints, resulting in a clustered lane line point set. Then, the Mean-Shift moving average method is used to smooth and denoise the clustered lane line point set. This set is then matched with lane line information from historical frames. Weighted fusion and correction based on historical trends are applied to obtain stable and consistent current lane line information.

[0023] S102. Determine the current intersection scene state of the vehicle based on the current lane line information. The intersection scene state includes the state before the intersection and the state when the vehicle is at the intersection.

[0024] In this embodiment, the current vehicle can be understood as a vehicle equipped with a camera and performing lane line detection and post-processing. The intersection scene state can be understood as the current vehicle's position relative to the intersection, including at least the pre-intersection state and the intersection state. The pre-intersection state can be understood as the stage where the current vehicle has not yet entered the intersection and is approaching it; the intersection state can be understood as the stage where the current vehicle is passing through the intersection.

[0025] Specifically, after acquiring the current lane line information, the changes in lane line length and curvature are determined. Based on at least one of the changes in length, curvature, and blank areas in the road image, the system identifies whether the vehicle is currently in a state before an intersection or at an intersection. For example, when a sudden decrease in lane line length and abrupt change in curvature are detected, if a large blank area exists in the near-end region of the road image, it indicates that the vehicle has entered the intersection and the near-end markings have disappeared, thus determining it to be in an intersection state. When a large blank area is detected in the far-end region and significant changes in lane line length and curvature are detected, it indicates that the vehicle is approaching the intersection and the far-end markings are not yet reliable, thus determining it to be in a state before an intersection.

[0026] S103. Based on the intersection scene status, perform differentiated deletion and reordering of lane lines in different areas to determine the set of lane lines to be retained.

[0027] In this embodiment, the retained lane line set can be understood as the set of lane line data that has been retained after differential deletion and reordering to participate in subsequent completion processing.

[0028] Specifically, when determining that the current vehicle is at an intersection, lane lines in the near-end area are deleted, while lane lines in the far-end area are retained. The retained lane lines in the far-end area are then reordered in descending order of confidence level, and if confidence levels are the same, they are sorted in ascending order of X-coordinate. When determining that the current vehicle is before the intersection, lane lines in the far-end area are deleted, while lane lines in the near-end area are retained. The retained lane lines in the near-end area are then reordered according to the same rules. This process determines the lane lines in the reliable area as the set of retained lane lines.

[0029] S104. Perform lane line completion processing on the retained lane line set to obtain the lane line post-processing result.

[0030] In this embodiment, the lane line post-processing result can be understood as the lane line detection result that can be used for autonomous driving decision-making after scene adaptive optimization and completion processing. It includes at least the reliable lane lines retained after scene judgment and differential selection, the new lane lines generated by copying in the case of a single line, the new lane lines generated by endpoint extension and parameter translation in the case of multiple lines, and the complete set of lane lines after sorting and numbering all lane lines.

[0031] Specifically, the retained lane line set is first filtered, removing lane lines whose length is less than the filtering threshold, and lane lines whose distance from the current vehicle is less than the filtering threshold and whose length is less than a set multiple of the filtering threshold, resulting in a filtered lane line set. Then, the number of lane lines in the filtered lane line set is determined. If only one lane line is contained, a second lane line is generated symmetrically or parallelly based on the lateral position of the current vehicle and according to the preset lane spacing. If at least two lane lines are contained, the first and second lane lines with the highest confidence are determined. If the two main lane lines are parallel, their endpoints are extended, and a new lane line with the same shape as the extended main lane line but with a shifted position is generated at the expected lane position where no valid lane line is detected. If they are not parallel, the extension and generation operations are not performed, resulting in the lane line post-processing result.

[0032] This invention provides a lane line detection and post-processing method, which includes extracting and optimizing lane line features from a road image to obtain current lane line information; determining the intersection scene state where the vehicle is located based on the current lane line information, where the intersection scene state includes the pre-intersection state and the current intersection state; performing differentiated deletion and reordering processing on lane lines in different areas according to the intersection scene state to determine a set of retained lane lines; and performing lane line completion processing on the retained lane line set to obtain the lane line post-processing result. These technical features, through lane line feature extraction and optimization processing on the road image, can obtain stable and reliable current lane line information, and differentiate the processing of lane lines in different areas according to different intersection scene states, avoiding lane line confusion or loss due to changes in scene state, further ensuring the integrity and continuity of the output lane lines. This effectively improves the stability and accuracy of lane line detection and post-processing under complex road conditions.

[0033] As a first optional embodiment of this example, lane line feature extraction and feature optimization processing are performed on the road image to obtain current lane line information, including: S1011. Extract lane line points from the road image and filter them according to the confidence level of each lane line point. Divide the filtered lane line points into multiple lane line point sets according to their spatial connectivity. Each lane line point set corresponds to a lane line instance.

[0034] In this embodiment, lane line points can be understood as pixels or feature points in a road image that belong to lane lines, identified by a deep learning model. Confidence can be understood as the value output by the deep learning model, representing the probability or degree of trust in each lane line point belonging to a real lane line. A set of lane line points can be understood as a collection of spatially connected lane line points. A lane line instance can be understood as a complete, independent lane line entity.

[0035] Specifically, road images are input into a deep learning model. The model's feature extraction layer, through the collaboration of convolutional and pooling layers, extracts features from the road images, generating multi-scale feature maps. These feature maps contain lane line feature information at different spatial resolutions and levels of abstraction. Then, a fully connected layer converts the feature maps into confidence values ​​for each lane line point. A confidence threshold is set, and only lane line points with confidence values ​​exceeding this threshold are retained. The spatial connectivity between the retained lane line points is determined, and an adjacency matrix is ​​constructed to describe the connections between points. A graph algorithm is used to progressively merge adjacent and closely connected point sets, ultimately grouping interconnected lane line points into the same lane line instance. The lane line points corresponding to each lane line instance constitute a lane line point set. During training, the deep learning model uses road images with different road conditions, weather conditions, and lighting conditions as training samples to ensure the model's adaptability to complex road conditions in multiple scenarios. A lightweight network architecture is also employed to meet the real-time requirements of the onboard computing platform.

[0036] S1012. Perform feature optimization processing on multiple lane line point sets to obtain the current lane line information.

[0037] In this embodiment, clustering optimization is first performed on multiple lane line point sets. That is, feature normalization is performed on the lane line points in each lane line point set, and K-Means clustering is performed on all normalized lane line points. The average is calculated in the Y-axis direction based on the model output bias to smooth the lane lines, and lane lines with similar endpoints are connected based on the cosine similarity of the residual endpoints to obtain the clustered lane line point set. Then, the clustered lane line point set is subjected to time-series tracking processing. The Mean-Shift point moving average method is used for smoothing and denoising, and then it is matched with the lane line information of historical frames. The direction is corrected by weighted fusion and historical trend, resulting in stable and continuous current lane line information.

[0038] Furthermore, feature optimization processing is performed on multiple lane line point sets to obtain the current lane line information, including: a1. After normalizing the features of the lane line points in each lane line point set, cluster the lane line points in all lane line point sets, and smooth and connect the broken lines in the clustering results to obtain the clustered lane line point set.

[0039] In this embodiment, the feature information of lane line points includes multi-dimensional information such as the geometric features, texture features, and relationship with the surrounding environment of the lane lines. These feature information undergoes normalization preprocessing to ensure that their data format and feature representation are suitable for the K-Means algorithm. The K-Means algorithm is then used to cluster these normalized lane line points, dividing them into different clusters according to a preset number of cluster centers. Then, for each cluster, the lane line points are averaged along the Y-axis based on the Y-axis offset of the lane line points output by the model. Only one Y value is retained for the same X-coordinate to smooth the fluctuations of the lane lines on the Y-axis. Finally, the cosine similarity of the residual endpoints of different lane lines is calculated. Lane lines to which endpoints (lane line points) with similarity higher than a threshold belong are connected, allowing for smooth transitions at the endpoints and avoiding lane line breaks and discontinuities. In the K-Means clustering process, the silhouette coefficient is introduced as an evaluation index for clustering effect. The clustering quality is monitored in real time. When the clustering effect is not good (e.g., the silhouette coefficient is lower than the preset threshold), the number of cluster centers or the number of iterations is automatically adjusted and the clustering is re-clustered until a satisfactory lane line subdivision effect is achieved.

[0040] b1. Smooth the clustered lane line point set, and then perform weighted fusion and trend correction on the smoothed lane line point set based on historical lane line information to obtain the current lane line information.

[0041] In this embodiment, historical lane line information can be understood as the lane line detection results of recent (e.g., within the last few seconds) consecutive frames stored during vehicle driving, that is, the historical lane line point set record that is adjacent to the current position (e.g., within 10-30 meters).

[0042] Specifically, the Mean-Shift moving average method is used to construct a sliding window on the clustered lane line point set. Within each window, the average position is calculated based on the distribution of lane line points, and the average position is used as the updated lane line point position to reduce noise interference in the lane line detection results. At the same time, the smoothed lane line point set is matched with the historical lane line point set in the historical lane line information. For lane lines that are successfully matched (e.g., the lateral distance between the current frame lane line and the historical frame lane line is less than the corresponding threshold, or the shape similarity between the two is higher than the corresponding threshold), the new and old point information are fused using an average or weighted average method to obtain a preliminary update result. Then, the lane line change trend is analyzed in combination with the historical lane line information to predict and correct the lane line direction in the preliminary update result, preventing sudden jumps and abnormal changes in lane lines, and obtaining stable and continuous current lane line information.

[0043] As a second optional embodiment of this example, determining the intersection scene state where the vehicle is currently located based on the current lane line information includes: S1021. Based on at least one of the changes in lane length and lane curvature corresponding to the current lane line information, and the blank area of ​​the road image, determine whether the current vehicle is in the state before the intersection or at the intersection.

[0044] In this embodiment, lane line length variation can be understood as the change in the effective length of the same lane line between the current frame and historical frames. Lane line curvature variation can be understood as the change in the degree of lane line bending. Blank areas can be understood as areas in the road image where lane lines are not detected.

[0045] Specifically, after acquiring the current lane line information, the length and geometric relationship attributes in the current lane line information are compared with those in the historical lane line information to obtain the changes in lane line length and curvature. Based on at least one of the length changes, curvature changes, and blank areas in the road image, the system identifies whether the current vehicle is in a state before an intersection or at an intersection. For example, when a sudden decrease in lane line length and abrupt change in curvature are detected (e.g., lane line length decreases by more than 50%, curvature change rate exceeds 30%), if a large blank area exists in the near-end region of the road image (e.g., the blank area accounts for more than 60% of the total area of ​​the near-end region), it indicates that the current vehicle has entered the intersection and the near-end markings have disappeared, thus determining it to be in an intersection state. When a large blank area exists in the far-end region (e.g., the blank area accounts for more than 60% of the total area of ​​the far-end region) and the lane line length and curvature change significantly, it indicates that the current vehicle is approaching the intersection and the far-end markings are not yet reliable, thus determining it to be in a state before an intersection.

[0046] As a third optional embodiment of this example, differentiated deletion and reordering of lane lines in different areas are performed based on the intersection scene status, including: S1031. When in the state of intersection, delete the lane lines in the first area, retain the lane lines in the second area, and reorder the lane lines in the second area.

[0047] The reordering includes sorting by confidence level in descending order, and sorting by X-coordinate in ascending order when the confidence levels are the same.

[0048] In this embodiment, the first region can be understood as the near-end region of the road image closest to the current vehicle, such as the lower half of the road image with a larger Y-coordinate, which corresponds to the nearby road segment within a range of approximately 0-30 meters in front of the vehicle. The second region can be understood as the far-end region of the road image far from the current vehicle, such as the upper half of the image with a smaller Y-coordinate, which corresponds to the farther road segment within a range of approximately 30-80 meters in front of the vehicle.

[0049] Specifically, when the vehicle is at the intersection, all lane lines in the first area are deleted, and only the lane lines in the second area are retained. The lane lines retained in the second area are then sorted from high to low confidence. When multiple lane lines have the same confidence, they are sorted from small to large X coordinate, with the left lane line given priority. Finally, the reordered set of lane lines in the second area is obtained.

[0050] S1032. When in the state before an intersection, delete the lane lines in the second area, retain the lane lines in the first area, and reorder the lane lines in the first area.

[0051] Specifically, when determining that the current vehicle is in the state before the intersection, all lane lines in the second area are deleted, and only the lane lines in the first area are retained. The lane lines retained in the first area are sorted from high to low confidence. When multiple lane lines have the same confidence, they are sorted from small to large X coordinate, so that the left lane line is prioritized. Finally, the reordered set of lane lines in the first area is obtained.

[0052] As a fourth optional embodiment of this example, lane line completion processing is performed on the retained lane line set, including: S1041. When the set of retained lane lines contains only one lane line, based on the current lateral position of the vehicle, a second lane line that is symmetrical or parallel to the first lane line is generated according to the preset lane spacing.

[0053] In this embodiment, the preset lane spacing can be understood as the standard lateral distance between two adjacent lane lines (e.g., 3.5 meters).

[0054] Specifically, in intersection scenarios, after differential deletion processing, the remaining lane line set may only contain one lane line. At this point, a single lane line is insufficient to form a complete lane structure. To maintain the integrity and symmetry of the lane structure, the orientation of the lane line relative to the lateral position of the current vehicle is first determined. If the lane line is to the left of the current vehicle, a new lane line is generated by offsetting to the right by a preset lane spacing, based on the current vehicle's lateral position. If the lane line is to the right of the current vehicle, a new lane line is generated by offsetting to the left by a preset lane spacing. The newly generated lane line is symmetrical or parallel to the original lane line with respect to the current vehicle, and no interpolation extension is performed to avoid errors introduced by over-extrapolation due to insufficient information. Finally, a lane line set containing two lane lines is obtained.

[0055] S1042. When the set of retained lane lines contains at least two lane lines, complete the process based on the confidence level and spatial relationship of each lane line in the set of retained lane lines.

[0056] In this embodiment, confidence level can be understood as the degree of reliability of the lane line detection results. Spatial positional relationship can be understood as the relative positional relationship between each lane line, including parallel relationship and lateral spacing relationship.

[0057] Specifically, when the set of retained lane lines contains at least two lane lines, the set of retained lane lines is first filtered to remove lane lines that are too short or whose positions are unreliable. Then, the two lane lines with the highest confidence are determined from the filtered set of lane lines as the main lane lines. Based on the spatial relationship between these two main lane lines, it is determined whether the parallel condition is met. If the condition is met, the positions of other missing lane lines are filled in based on the main lane lines. If the condition is not met, no filling process is performed to avoid generating incorrect lane line structures.

[0058] Furthermore, completion processing is performed based on the confidence level and spatial relationship of each lane line in the retained lane line set, including: a2. After filtering the set of retained lane lines, determine the first lane line and the second lane line with the highest confidence in the filtered set of retained lane lines.

[0059] In this embodiment, the first lane line can be understood as the lane line with the highest confidence level in the filtered set of retained lane lines. The second lane line can be understood as the lane line with the second highest confidence level in the filtered set of retained lane lines.

[0060] Specifically, the set of retained lane lines is first filtered to remove lane lines that are too short or whose positions are unreliable, resulting in a filtered set of retained lane lines. Then, from the filtered set of retained lane lines, the two lane lines with the highest confidence are determined in descending order of confidence and are used as the first lane line and the second lane line, respectively, for subsequent parallel judgment and completion processing.

[0061] b2. If the first lane line is parallel to the second lane line, perform endpoint extension processing on the first lane line and the second lane line, and generate a new lane line with the same shape as the extended first lane line or the second lane line but with a shifted position at the expected lane position where no valid lane line is detected.

[0062] In this embodiment, a valid lane line can be understood as a lane line that has been confirmed as reliable after the aforementioned steps and can be used as a reference for subsequent lane line completion; it is a lane line instance in the retained lane line set. A expected lane position can be understood as the position of a lane line that should exist but has not been detected, calculated based on the preset lane spacing and the existing valid lane line positions.

[0063] Specifically, it is determined whether the first lane line and the second lane line are parallel. If they are parallel, the lane lines are extended outwards at both ends to cover a more complete road segment. Then, the expected lane position where no valid lane line was detected is determined. Using the polynomial coefficients of the extended first or second lane line as a template, the parameters representing the lateral position are offset according to the preset lane spacing. A new lane line with the same shape as the template is reconstructed at the expected lane position, which is only laterally shifted by one lane spacing, to fill in the missing lane line. If the first lane line and the second lane line are not parallel, the endpoint extension and new lane line generation operations are not performed to avoid generating incorrect lane lines when the road structure is abnormal.

[0064] Furthermore, the set of retained lane lines is filtered, including: a21. Remove lane lines whose length is less than the filtering threshold from the set of retained lane lines.

[0065] In this embodiment, the filtering threshold can be understood as a standard length value (e.g., t meters) used to determine whether a lane line is too short.

[0066] Specifically, each lane in the set of retained lane lines is traversed, its effective length is calculated, and lane lines with a length less than the filtering threshold are removed from the set of retained lane lines. Only lane lines with a length greater than or equal to the filtering threshold are retained to eliminate the interference of excessively short lane lines on subsequent completion processing.

[0067] a22. Remove lane lines whose distance from the current vehicle is less than the filtering threshold and whose lane line length is less than a set multiple of the filtering threshold from the set of retained lane lines.

[0068] In this embodiment, the set multiplier can be understood as a standard value (e.g., 2 times) used to determine whether the lane line length is reliable enough.

[0069] Specifically, in the set of retained lane lines after the a21 filter, nearby lane lines whose distance from the current vehicle is less than the filtering threshold are further selected. If the length of the nearby lane line is less than a set multiple of the filtering threshold (e.g., 2t), it is removed from the set of retained lane lines, and only nearby lane lines whose length meets the set multiple requirement are retained to improve the reliability of the nearby lane lines.

[0070] In one embodiment, Figure 2 This is a schematic diagram of the structure of a lane line detection and post-processing device provided in an embodiment of the present invention. Figure 2 As shown, the device includes: Information extraction module 21 is used to extract lane line features and optimize features from road images to obtain current lane line information; Scene determination module 22 is used to determine the intersection scene state where the current vehicle is located based on the current lane line information. The intersection scene state includes the state before the intersection and the state when the vehicle is at the intersection. The differentiation processing module 23 is used to perform differentiated deletion and reordering of lane lines in different areas according to the intersection scene state, and determine the set of lane lines to be retained. The lane line completion module 24 is used to perform lane line completion processing on the retained lane line set to obtain the lane line post-processing result.

[0071] The lane line detection and post-processing device used in this technical solution ensures the integrity and continuity of the output lane lines, and improves the stability and accuracy of lane line detection and post-processing under complex road conditions.

[0072] Optionally, the differentiation processing module 23 is specifically used for: When at an intersection, delete the lane lines in the first area, keep the lane lines in the second area, and reorder the lane lines in the second area. When in the state before the intersection, delete the lane lines in the second area, keep the lane lines in the first area, and reorder the lane lines in the first area. The reordering includes sorting by confidence level in descending order, and sorting by X-coordinate in ascending order when the confidence levels are the same.

[0073] Optionally, the lane line completion module 24 includes: The first completion unit is used to copy and generate a second lane line that is symmetrical or parallel to the first lane line, based on the lateral position of the current vehicle and according to a preset lane spacing, when the set of retained lane lines contains only one lane line. The second completion unit is used to complete the lane lines based on the confidence level and spatial relationship of each lane line in the reserved lane line set when the reserved lane line set contains at least two lane lines.

[0074] Optionally, the second completion unit includes: The lane line filtering subunit is used to filter the set of retained lane lines and then determine the first lane line and the second lane line with the highest confidence in the filtered set of retained lane lines. The lane line completion subunit is used to perform endpoint extension processing on the first lane line and the second lane line if the first lane line is parallel to the second lane line, and generate a new lane line with the same shape and position shift as the extended first lane line or the second lane line at the expected lane position where no valid lane line is detected.

[0075] Optionally, the lane line filtering subunit is specifically used for: Lanes with a length less than the filtering threshold are removed from the set of retained lanes; Lanes whose distance from the current vehicle is less than the filtering threshold and whose lane length is less than a set multiple of the filtering threshold are removed from the set of retained lanes.

[0076] Optionally, the scene determination module 22 is specifically used for: Based on at least one of the changes in lane length and lane curvature corresponding to the current lane line information, and the blank area of ​​the road image, it is determined whether the current vehicle is in a state before the intersection or at the intersection.

[0077] Optionally, the information extraction module 21 includes: The lane line point extraction unit is used to extract lane line points from the road image and filter them according to the confidence of each lane line point. The filtered lane line points are divided into multiple lane line point sets according to their spatial connection relationship, and each lane line point set corresponds to a lane line instance. The lane line point optimization unit is used to perform feature optimization processing on the multiple lane line point sets to obtain the current lane line information.

[0078] Optionally, the lane alignment optimization unit is specifically used for: After performing feature normalization on the lane line points in each of the lane line point sets, the lane line points in all the lane line point sets are clustered, and the clustering results are smoothed and disconnected to obtain the clustered lane line point set. The clustered lane line point set is smoothed, and based on historical lane line information, the smoothed lane line point set is weighted, fused, and trend-corrected to obtain the current lane line information.

[0079] The lane line detection and post-processing device provided in the embodiments of the present invention can execute the lane line detection and post-processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0080] In one embodiment, Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. For example... Figure 3The diagram illustrates a schematic representation of an electronic device 10 that can be used to implement embodiments of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0081] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0082] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0083] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as lane line detection and post-processing methods.

[0084] In some embodiments, the lane detection and post-processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the lane detection and post-processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the lane detection and post-processing method by any other suitable means (e.g., by means of firmware).

[0085] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), payload programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0086] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0087] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0088] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube or liquid crystal display) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0089] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0090] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system to address the shortcomings of traditional physical host and virtual machine services, such as high management difficulty and weak business scalability.

[0091] This invention also provides a computer program product, including a computer program that, when executed by a processor, can implement the lane line detection and post-processing method provided in any embodiment of this invention.

[0092] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0093] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0094] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A lane line detection and post-processing method, characterized in that, include: Lane line feature extraction and feature optimization are performed on road images to obtain current lane line information; The current intersection scene state of the vehicle is determined based on the current lane line information. The intersection scene state includes the state before the intersection and the state when the vehicle is at the intersection. Based on the intersection scene status, lane lines in different areas are subject to differentiated deletion and reordering to determine the set of lane lines to be retained. The reserved lane line set is subjected to lane line completion processing to obtain the lane line post-processing result.

2. The method according to claim 1, characterized in that, The step of performing differentiated deletion and reordering of lane lines in different areas based on the intersection scene status includes: When at an intersection, delete the lane lines in the first area, keep the lane lines in the second area, and reorder the lane lines in the second area. When in the state before the intersection, delete the lane lines in the second area, keep the lane lines in the first area, and reorder the lane lines in the first area. The reordering includes sorting by confidence level in descending order, and sorting by X-coordinate in ascending order when the confidence levels are the same.

3. The method according to claim 1, characterized in that, The process of completing the lane lines on the reserved lane line set includes: When the set of reserved lane lines contains only one lane line, a second lane line that is symmetrical or parallel to the first lane line is generated based on the lateral position of the current vehicle and according to the preset lane spacing. When the set of reserved lane lines contains at least two lane lines, completion processing is performed based on the confidence level and spatial relationship of each lane line in the set of reserved lane lines.

4. The method according to claim 3, characterized in that, The completion process based on the confidence level and spatial relationship of each lane line in the retained lane line set includes: After filtering the set of retained lane lines, the first lane line and the second lane line with the highest confidence level in the filtered set of retained lane lines are determined. If the first lane line is parallel to the second lane line, the endpoints of the first lane line and the second lane line are extended, and a new lane line with the same shape and a shifted position as the extended first lane line or the second lane line is generated at the expected lane position where no valid lane line is detected.

5. The method according to claim 4, characterized in that, The filtering of the set of reserved lane lines includes: Lanes with a length less than the filtering threshold are removed from the set of retained lanes; Lanes whose distance from the current vehicle is less than the filtering threshold and whose lane length is less than a set multiple of the filtering threshold are removed from the set of retained lanes.

6. The method according to claim 1, characterized in that, The step of determining the intersection scene state of the current vehicle based on the current lane line information includes: Based on at least one of the changes in lane length and lane curvature corresponding to the current lane line information, and the blank area of ​​the road image, it is determined whether the current vehicle is in a state before the intersection or at the intersection.

7. The method according to claim 1, characterized in that, The process of extracting and optimizing lane line features from the road image to obtain current lane line information includes: Lane line points are extracted from the road image and filtered according to the confidence level of each lane line point. The filtered lane line points are divided into multiple lane line point sets according to their spatial connectivity, and each lane line point set corresponds to a lane line instance. Feature optimization processing is performed on the multiple lane line point sets to obtain the current lane line information.

8. The method according to claim 7, characterized in that, The step of performing feature optimization processing on the multiple lane line point sets to obtain the current lane line information includes: After performing feature normalization on the lane line points in each of the lane line point sets, the lane line points in all the lane line point sets are clustered, and the clustering results are smoothed and disconnected to obtain the clustered lane line point set. The clustered lane line point set is smoothed, and based on historical lane line information, the smoothed lane line point set is weighted, fused, and trend-corrected to obtain the current lane line information.

9. A lane line detection and post-processing device, characterized in that, include: The information extraction module is used to extract lane line features and optimize features from road images to obtain current lane line information. The scene judgment module is used to determine the intersection scene state where the current vehicle is located based on the current lane line information. The intersection scene state includes the state before the intersection and the state when the vehicle is at the intersection. The differentiation processing module is used to perform differentiated deletion and reordering of lane lines in different areas according to the intersection scene status, and determine the set of lane lines to be retained. The lane line completion module is used to perform lane line completion processing on the reserved lane line set to obtain the lane line post-processing result.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a lane line detection and post-processing method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the lane line detection and post-processing method according to any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements a lane line detection and post-processing method according to any one of claims 1-8.

Images