Lane line detection method based on YOLOv8 and LSD algorithm

Abstract

The invention discloses a lane line detection method based on YOLOv8 and LSD algorithms. The lane line detection method is suitable for accurately identifying lane lines in complex environments such as rainy days, nights or severe marked line abrasion. Canny edge detection is introduced into the YOLOv8 network, and the straight line segments in the detection area are subjected to enhanced recognition in combination with the LSD, so that the ability of capturing fuzzy or partially-shielded lane lines can be remarkably improved. Firstly, a general area of a lane line is positioned by using YOLOv8, then global edge features are extracted by using a Canny algorithm, and the length, direction and integrity of a line segment are judged under the assistance of an LSD algorithm. And if the detection frame clings to the edge of the image, further cutting the sub-image, detecting the line segment continuity by using LSD, and evaluating whether the lane line completely enters the visual field through the perimeter-area ratio and the compactness. The method can effectively reduce leak detection and false detection, is suitable for various illumination and weather conditions, and provides high-reliability lane line detection results for automatic driving and advanced driving assistance systems.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving and autonomous driving assistance, and particularly to a lane line detection method based on the combination of YOLOv8, LSD and Canny algorithms. In the process of using the YOLOv8 network for lane line detection, the Canny algorithm is introduced to preprocess the overall edge features of the image, and the LSD algorithm is used to enhance the recognition of lane line segments, which is applicable to complex road environments such as rainy days, nights, blurred or severely worn lane lines. By combining deep learning with a line detection algorithm and improving the precise positioning of key line segments with the assistance of Canny edge detection, the false detection and missed detection caused by the lane line not fully entering the camera's field of view or being blocked are effectively reduced, realizing the real-time monitoring and warning of vehicle lane departure and improving the safety of vehicle driving. Background Art

[0002] With the rapid development of artificial intelligence, autonomous driving and intelligent transportation technologies, the accurate perception of lane lines and the demand for safety assistance by vehicles are becoming increasingly important. The evolution of advanced driver assistance systems (ADAS, Advanced Driver Assistance Systems) and autonomous driving functions is inseparable from the real-time detection of lane lines and offset warning. Traditional lane line detection methods mostly rely on algorithms based on edges or morphological features, such as extracting lane lines through classic Canny edge detection, Hough transform and other means. However, in increasingly complex road environments, including scenes such as blurred, damaged lane lines, changing lighting, night driving, and rainy and foggy weather, traditional methods often suffer from problems such as insufficient detection accuracy, poor stability, and high false alarm and missed detection rates, and it is difficult to meet the requirements for safety and real-time performance in actual driving scenarios.

[0003] In recent years, deep learning has made significant progress in the field of computer vision, and object detection models have continued to evolve. The YOLO (You Only Look Once) series of models have shown a good balance of speed and accuracy in many visual tasks. Among them, YOLOv8 further improves the model's ability to capture detailed features and detection efficiency, providing new possibilities for the real-time detection of lane lines. However, when using deep learning models for lane detection, if the lane line is partially blocked or has not fully entered the camera's field of view, or the lane markings are unclear due to wear or paint fading, there may still be problems of missed detection and false detection.

[0004] To address the above difficulties, the LSD line detection algorithm is highly targeted in extracting straight or approximately straight lane markings, complementing deep learning models. LSD can quickly locate line segments in an image, excluding irrelevant textures and noises, and has high accuracy and interpretability especially in scenarios of straight or nearly straight lane markings. Based on the integration of traditional edge detection methods such as Canny, combining the deep learning capabilities of YOLOv8 for various scene features in complex road environments and the precise detection capabilities of LSD for lane line segments can effectively make up for the deficiencies of single methods.

[0005] The present invention aims to combine YOLOv8 with the LSD algorithm to establish a lane line detection method applicable to various road types (highways, urban streets, rural roads, etc.) and various lighting and weather conditions. By introducing the integrity detection and position judgment of lane line segments, it can effectively identify lane lines that have not fully entered the camera's field of view or are partially blocked by obstacles, reduce the false detection of lane lines due to incomplete lane lines, and effectively improve the safety and reliability of vehicle driving, providing important technical support for the development of intelligent transportation and autonomous driving. Summary of the Invention

[0006] The object of the present invention is to solve the problems of missed detection and false detection caused by lane lines not fully entering the frame or being blocked in the camera video stream in traditional lane line detection methods.

[0007] To achieve the above object of the invention, the present invention proposes a lane line detection method based on improved YOLOv8 combined with Canny and LSD line detection algorithms. The specific steps are as follows:

[0008] Step 1: First, collect lane line image data in various road scenarios, including various typical environments such as highways, urban roads, and rural roads, and cover various lighting and weather conditions such as day, night, rain, and fog. On this basis, perform data enhancement and preprocessing operations such as rotation, scaling, flipping, and noise addition to construct a diverse and representative lane line image dataset to help improve the generalization ability of the model and enable the detection model to better identify lane lines in different scenarios.

[0009] Step 2: While YOLOv8 performs lane line object detection, introduce the Canny algorithm to perform edge detection on the input image to obtain a global edge feature map; and further use the LSD algorithm to extract obvious straight line segments to obtain the position and direction information where the lane lines may be located. By fusing the edge features output by Canny and the straight line segment information obtained by LSD with the lane detection results output by the YOLOv8 model, the perception of lane line details can be strengthened, enabling the model to still identify complete lane lines in cases of lane blur, damage, or complex lighting.

[0010] The LSD (Line Segment Detector) is a fast and efficient algorithm for detecting straight line segments in images. It can extract significant straight line features from edge information, providing the precise position and direction information of lane lines. LSD is sensitive to geometric properties and can accurately extract straight lines under low noise interference, making it suitable for enhancing the perception of structured features in images.

[0011] The Canny algorithm is suitable for refining the contour of the detection target, making the model pay more attention to the key areas in the image, enabling the model to still perceive the contour of the lane under complex lighting, blurred or damaged conditions, thereby improving the detection accuracy and robustness.

[0012] Step 3: Based on the lane line detection boxes initially obtained by YOLOv8, calculate the distance between the detection boxes and the image edges. If the edge of the detection box is adjacent to the image edge, it indicates that the lane line may not have fully entered the field of view or there is a risk of partial occlusion. At this time, based on the fused output information of Canny and LSD, further check the continuity and integrity of the line segments to avoid missed detections and misjudgments caused by incomplete lane lines. When the detection box is indeed close to the image edge, more refined judgments need to be made in the subsequent steps.

[0013] Step 4: If it is initially determined that the detection box is at the edge position of the image, crop this area into a sub-image, perform further LSD straight line segment detection on the lane lines within the sub-image, and combine the Canny edge information to determine whether the lane lines are broken at the edge of the sub-image. By analyzing the closure, length, and direction consistency of the line segments, it can be confirmed whether the lane lines have truly entered the camera image completely; if the line segments are interrupted or significantly deformed at the edge of the sub-image, it indicates that the lane lines may still be outside the image or severely occluded, and at this time, continue to detect in the subsequent frames to further confirm and identify the lane lines.

[0014] Step 5: Combine the improved YOLOv8 detection network, the Canny edge detection module, and the LSD straight line detection module to write the lane line detection process. If it cannot be confirmed temporarily due to insufficient lane line integrity, multi-frame temporal tracking can be entered to improve the accuracy of the determination.

[0015] The beneficial effects of the present invention are as follows: The present invention combines YOLOv8 with the Canny and LSD algorithms to improve the detection accuracy and real-time performance of lane lines. By strengthening the recognition of blurred or damaged lane lines through edge features and line segment detection, false detections and missed detections are reduced; and by combining the position of the detection box and the integrity of the line segments for judgment, errors caused by lane lines not fully entering the field of view or being occluded are avoided. In summary, this method can significantly adapt to various complex road conditions as a whole, effectively improving vehicle driving safety and system robustness. Description of the Drawings

[0016] Figure 1It is a flow chart of lane line detection by combining the YOLOv8 model with edge detection algorithms and the LSD detection algorithm. Figure 2 It is an architecture diagram of the Canny edge detection algorithm added to YOLOv8. Figure 3 It is a horizontal line angle diagram of the straight line detection algorithm. Specific implementation manners

[0017] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0018] Step 1: To construct a diverse and representative lane line dataset, the present invention collects lane line images covering various typical scenarios such as highways, urban roads, and rural roads, covering different lighting and weather conditions such as day, night, rain, and fog. The collected images are marked with the positions of lane lines through annotation tools, and data augmentation operations are performed, including rotation, scaling, flipping, brightness adjustment, noise addition, etc., to improve the model's adaptability to complex environments. At the same time, preprocessing such as de-distortion and ROI cropping is performed on the images according to actual needs to filter out interference regions, and finally a rich and high-quality dataset is constructed to support model training and verification.

[0019] Step 2: Introduce an edge detection module based on the YOLOv8 network to enhance the detection ability of target edges and detailed features. Edge feature extraction is an important part of image data processing, and the integrity and accuracy of edge feature data extraction of image data are related to the effect of image applications. First, perform edge detection preprocessing on the image, use the Canny edge detection algorithm to extract the edge information of the lane lines in the image, form an edge feature map, and combine it with the input image of the YOLOv8 model to form richer input features. The steps and key formulas are as follows.

[0020] 1. Gaussian filtering: First, perform Gaussian blurring on the image I(x, y) to suppress the interference of noise on subsequent edge detection.

[0021] Gaussian filtering can be expressed in the following form:

[0022] I smooth (x, y) = G(x, y) * I(x, y) (1)

[0023] Among them, * represents the convolution operation, and G(x, y) is the Gaussian kernel function.

[0024] 2. Gradient calculation: Use the Sobel operator to take the derivative of the smoothed image in the x and y directions respectively to obtain the gradient magnitude G and the gradient direction θ.

[0025] Gradient magnitude (edge intensity):

[0026] Gradient direction:

[0027] Among them, G x and G y are the first-order derivatives of the image in the horizontal and vertical directions respectively. By calculating G and θ, the potential edge regions can be located.

[0028] The Sobel operator is used to calculate the first-order derivatives of the image in the horizontal and vertical directions, that is, the luminance change rate. Through this process, the gradient information of the image can be extracted, reflecting the intensity and direction of the edges. Specifically, the Sobel operator calculates the gradient values in the horizontal and vertical directions respectively, and then calculates the magnitude (i.e., the edge intensity) and direction of the gradient according to the above formula, so as to determine the significance and change trend of the edges in the image. After extracting the edge structure of the image, by calculating the magnitude and direction of the gradient, the Canny algorithm can locate the potential edges.

[0029] 3. Non-maximum suppression: According to the gradient direction θ, check the gradient magnitudes of the current pixel and its neighboring pixels. If the gradient value of the current pixel is not the local maximum, suppress it to 0 to remove the redundant edge points and retain the true strong edges.

[0030] 4. Double-threshold detection and edge connection:

[0031] Set two thresholds T high and T low . For the pixels with gradient values greater than T high , mark them as strong edges. For the pixels between T low and T high , mark them as weak edges. For the pixels below T low , directly discard them. Then, by connecting the strong edges with the adjacent weak edges, the discontinuous edges can be complemented to obtain the coherent lane line contour edges.

[0032] Through the above steps, the Canny algorithm can effectively extract the edge information, which is suitable for refining the object contour. The edge information can be combined with the feature map of YOLOv8 to help the YOLOv8 network more clearly distinguish the lane line contour, make the model pay more attention to the key regions in the image, and improve the detection accuracy of detecting the lane line edges.

[0033] Step 3: After the YOLOv8 outputs the lane line detection box, the present invention first records the four edge positions of the detection box Δy top = y min , Δy bottom = H - y max , Δx left = x min , Δx right = W - x max, where W is the image width and H is the image height, and (xmin, xmax), (ymin, ymax) represent the left - right and up - down edge coordinates of the detection box respectively. Then, two thresholds are set to determine whether the detection box is close to the image edge: If Δy top / H < δ1 or Δx left / W < δ1, then the lane line is at the upper or left edge; if Δy bottom / H < δ2 or Δx right / W < δ2, then the lane line is at the lower or right edge. When the detection box is determined to be close to the image edge, it indicates that the lane line may not have fully entered the camera's field of view, and it is necessary to further confirm the integrity of the lane line to avoid missed detection or misjudgment caused by incomplete lane lines.

[0034] Step 4: When it is preliminarily determined that the lane line is at the edge of the image through the detection box position judgment in the previous step, it is necessary to further check whether the lane line has fully entered the camera's field of view. For this purpose, the present invention first crops the detection box area into a new sub - image to exclude other irrelevant background interferences. Then, the LSD (Line Segment Detector) line detection is performed on this sub - image, and the Canny edge information is combined to determine whether the lane line is broken or discontinuous at the edge of the sub - image.

[0035] The Canny edge detection algorithm is used to perform edge contour extraction on the sub - image within the detection box, obtaining a high - quality binary edge map of the lane line detection box area, providing more accurate potential lane line contours for LSD. This map only contains the edge features of the sub - image at the position of the detection box where the lane line is located, excluding other background interferences.

[0036] Subsequently, LSD performs geometric analysis on these edge pixels, screening out a set of line segments that meet requirements such as length L, angle θ, and continuity. For the line segments suspected of being lane lines, the present invention records and determines their features such as length, direction, and whether there are breakpoints. The formulas are as follows:

[0037] Line segment length:

[0038] Line segment direction:

[0039] Line segment fitting: r = xcosθ + ysinθ (6)

[0040] Among them, θ represents the angle between the line segment and the horizontal direction. If |θ - θ road | significantly deviates from the known road lane line direction θ road , then it is necessary to be vigilant against missed detection or misdetection, and r is the distance from the straight line to the origin.

[0041] By setting a length threshold, if a line segment is much shorter than the typical lane line length range, it may indicate that the lane line has not fully appeared or there is an obvious break; at the same time, comparing the direction of the line segment with the overall direction of the known lane, if the difference is too large, one needs to be vigilant about missed detection or false detection. Secondly, if a line segment is close to the edge of the sub - figure but cannot be connected to other line segments, it means that it is not fully displayed in the picture; on the contrary, if a line segment is continuous within the sub - figure and has a reasonable gap from the edge, it can be initially regarded that the lane line has fully entered the field of view.

[0042] If a certain line segment is truncated at the edge of the sub - figure and cannot be connected to the surrounding line segments, it indicates that the lane line is still outside the picture or is seriously blocked by other objects; if the line segment is relatively complete and its length and direction match the expected values of common lane lines, it can be initially determined that the lane line has fully entered the camera's field of view, providing a reliable basis for subsequent vehicle offset calculation and alarm.

[0043] To more accurately distinguish between "complete lane lines" and "lane lines that have not fully entered the picture", the detected line segments will be further analyzed, comprehensively referring to concepts such as "closure" and "compactness". For the closure of the contour, the ratio R of the perimeter P to the area A of the contour is used to judge the degree of closure. A closed contour usually has a smaller ratio. If R < R thresh , it is considered that the contour is closed; if it is greater than this value, it may be non - closed. Among them, R thresh is the threshold for determining contour closure, which can be fine - tuned through data in different scenarios.

[0044]

[0045] Then, according to the compactness C compactness of the contour, it further helps to determine the closure of the contour. By calculating whether the contour shape is tight and without breaks to check the contour closure, the judgment formula is as follows:

[0046]

[0047] Among them, when the value of C compactness is close to 1, it indicates that the contour is relatively tight and close to a closed shape. If it is significantly lower than 1, it is judged that the contour is non - closed, that is, the lane line has not fully entered the picture.

[0048] Through the above - mentioned method, in the case where the detection frame may be close to the edge of the picture, the present invention can effectively distinguish whether the lane line has truly "fully entered" the field of view or has been seriously blocked. When it is confirmed that the line segment meets the integrity standard, the system then calculates the vehicle offset or triggers an alarm, greatly reducing missed detection and false detection caused by incomplete lane lines, thereby improving the accuracy and overall robustness of lane line detection.

[0049] The above are only embodiments of the present invention, and thus do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of this specification and the accompanying drawings, or directly or indirectly applied in the relevant technical fields, shall be similarly included in the patent protection scope of the present invention.

Claims

1. A lane line detection method based on YOLOv8 and LSD algorithm, characterized in that: The specific steps include: S1: Build a diverse and representative lane line dataset, annotate, enhance and preprocess the collected images to ensure that the model can accurately identify lane lines. S2: Based on the existing YOLOv8 network, the Canny edge detection module is introduced to obtain the global edge feature map of the lane line. The LSD (Line Segment Detector) algorithm is further used to extract the straight line segment information with significant edges and combined with the input image of the YOLOv8 model. S3: First, determine the position of the lane line detected by YOLOv8. Based on the position determination of the lane line detection frame, combine the edge detection algorithm to further determine whether the lane line is completely within the camera's field of view and confirm its integrity. S4: Perform sub-image cropping on the detection frame at the edge of the image and perform LSD straight line detection. Combined with the Canny edge information, the closure, length and direction consistency of the line segment are determined to determine whether the lane line has truly entered the picture completely. S5: Integrate the improved YOLOv8 network, Canny edge detection and LSD line detection modules to complete the lane detection process and trigger an alarm when the vehicle is detected to deviate from the safety threshold.

2. The lane line detection method based on YOLOv8 and LSD algorithm according to claim 1, characterized in that: The construction and preprocessing of the lane line data set in step S1 includes: S11: Collect lane line images of different types of road scenes, including highways, urban roads and rural roads, and cover various weather and lighting conditions such as day, night, rain and fog. S12: Annotate the collected images to obtain lane line position or contour information, and perform data enhancement by means of rotation, scaling, flipping, brightness adjustment, noise addition, etc. S13: Perform pre-processing such as de-distortion and ROI cropping on the image according to actual needs, filter out interference such as invalid areas or vehicle front ends, and improve data quality. S14: Unify and organize the enhanced and preprocessed images to build a high-quality lane line dataset to provide diverse scenario support for model training and verification.

3. The lane line detection method based on YOLOv8 and LSD algorithm according to claim 2, characterized in that: The specific steps of introducing the edge detection module into the YOLOv8 network in step S2 are: S21: Add Canny edge detection preprocessing at the input of the YOLOv8 network, perform Gaussian filtering to denoise the original image I(x, y), then use the Sobel operator to obtain the gradient amplitude and direction, and then obtain the global lane line edge through non-maximum suppression and double threshold connection; S22: Concatenate or fuse the "global edge feature map" output by Canny with the original image in the channel dimension to form richer input features. S23: The fused image is input into the YOLOv8 network for preliminary lane line detection, and LSD is used to extract straight line segments from the detected edge area to obtain the possible position and direction information of the lane line.

4. The lane line detection method based on YOLOv8 and LSD algorithm according to claim 3, characterized in that: The specific steps and key formulas for using the Canny algorithm to extract lane edge information in step S2 are: S211: Gaussian filtering: First, perform Gaussian blur on the image I (x, y) to reduce noise interference. The formula for Gaussian filtering is: I smooth (x,y)=G(x,y)*I(x,y) Among them, * represents the convolution operation, and G(x, y) is the Gaussian kernel function. S212: Gradient calculation: Use the Sobel operator to derive the smoothed image in the x and y directions respectively to obtain the gradient magnitude G and the gradient direction θ. Gradient magnitude (edge ​​strength): Gradient direction: Among them, G x and G y are the first-order derivatives of the image in the horizontal and vertical directions respectively. By calculating G and θ, the potential edge area can be located. The Sobel operator is used to calculate the first-order derivative of the image in the horizontal and vertical directions, that is, the rate of change of brightness. Through this process, the gradient information of the image can be extracted to reflect the strength and direction of the edge. Specifically, the Sobel operator calculates the gradient values ​​in the horizontal and vertical directions respectively, and then calculates the size of the gradient (that is, the edge strength) and the gradient direction according to the above formula, thereby determining the significance and change trend of the edge in the image. The edge structure of the image is extracted, and by calculating the size and direction of the gradient, the Canny algorithm can locate the potential edge. S213: Non-maximum suppression: According to the gradient direction θ, check the gradient magnitude of the current pixel and its neighboring pixels. If the gradient value of the current pixel is not the local maximum, suppress it to 0 to remove redundant edge points and retain the real strong edge. S214: Setting two thresholds T high With T low , for gradient values ​​greater than T high The pixels between T low With T high The pixels between are marked as weak edges, below T low Pixels with strong edges are directly discarded. Then, by connecting the strong edge with the adjacent weak edge, the discontinuous edge is completed to obtain a coherent lane line contour edge.

5. The lane line detection method based on YOLOv8 and LSD algorithm according to claim 4, characterized in that: The specific steps of step S3 are: S31: Get the upper, lower, left and right edge coordinates (xmin, xmax), (ymin, ymax) of the lane line detection frame output by YOLOv8 detection to obtain the four edge positions Δy of the detection frame top =y min , Δy bottom =Hy max , Δx left =x min , Δx right =Wx max . Where W is the image width and H is the image height. S32: Set two thresholds to determine whether the detection frame is close to the edge of the image: top / H<δ1orΔx left / W<δ1, the lane line is at the upper or left edge; if Δy bottom / H<δ2 or Δx right / W<δ2, the lane line is at the bottom or right edge. S33: When the detection frame is judged to be close to the edge of the image, it means that the lane line may not be completely in the camera's field of view, and the integrity of the lane line needs to be further confirmed to avoid missed detection or misjudgment due to incomplete lane lines.

6. The lane line detection method based on YOLOv8 and LSD algorithm according to claim 5, characterized in that: The specific implementation of the LSD line detection algorithm includes: S41: cluster edge pixels according to their gradient directions to form line segment support regions. S42: Use the minimum rectangle or least square method to fit the support area of ​​each line segment to obtain the endpoints (x1, y1) and (x2, y2) of the line segment and the direction θ of the line segment. S43: Record the obtained line segment length, direction and other features to provide a basis for subsequent lane line integrity determination. The formulas for line segment length, line segment direction and line segment fitting are as follows: Segment length: Segment Direction: Line segment fitting: r = xCosθ ​​+ ysinθ Among them, θ represents the angle between the line segment and the horizontal direction. If |θ-θ road |Significantly deviate from the known road lane line directionθ road , you need to be alert to missed detection or false detection, r is the distance from the straight line to the origin. S44: By setting a length threshold, if a line segment is much shorter than the typical lane line length range or is close to the edge of the sub-image and cannot connect to other line segments, it is determined that the lane line has not yet fully appeared or is severely obscured; conversely, if the line segment remains continuous within the sub-image and has a reasonable interval with the edge, and its length and direction are consistent with the expected values ​​of common lane lines, it is considered that the lane line has completely entered the camera's field of view, providing a reliable basis for subsequent vehicle offset calculation and alarm. S45: Determine the closure of the lane line contour by detecting the perimeter P and area A of the lane line contour.

7. The lane line detection method based on YOL0v8 and LSD algorithm according to claim 6 is characterized in that The integrity of the lane line is determined by using closure and compactness. The specific steps of step S45 are as follows: For the closure of a contour, the ratio R of the contour's perimeter P to its area A is used to determine the degree of closure. A closed contour will usually have a smaller ratio, if R < R thresh , the contour is considered closed; if it is greater than this value, it may be non-closed. thresh To determine the threshold for contour closure, fine-tuning can be performed using data from different scenes. Next, according to the compactness C of the contour compactness Further help determine the closure of the contour. Check the closure of the contour by calculating whether the contour shape is tight and has no breaks. The judgment formula is as follows: Among them, C compactness When the value is close to 1, it means that the contour is tight and close to a closed shape. If it is significantly lower than 1, the contour is judged to be in an open state and the lane line has not completely entered the picture.

Images