A lane line detection method for strong light interference based on image restoration algorithm

Abstract

The present invention provides a method for detecting lane lines under strong light interference based on an image restoration algorithm, which belongs to the field of image processing and road safety driving technology. The method comprises the following steps: S1, extracting an image from a video frame; S2, performing ROI processing on the extracted image and calculating a discrete coefficient. If the coefficient is higher than a threshold, S3 is performed; otherwise, S4 is performed; S3, eliminating strong light interference using an inpaintCoherent algorithm; S4, converting the image from RGB channels to HSV channels and performing global equalization processing; S5, performing grayscale processing, then performing image edge detection using a LoG operator, and then performing mean filtering processing; S6, performing binarization processing based on a threshold constraint of the brightness mean and variance after mean filtering; S7, performing lane line detection using a Hough transform detection algorithm, performing outlier filtering, and least squares fitting to draw lane lines. The method can effectively improve the accuracy of lane line detection under external strong light interference, avoid identifying strip-shaped strong light as lane lines, and improve road driving safety.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and road safety driving, and in particular relates to a method for detecting lane lines interfered with by strong light based on an image restoration algorithm. Background Art

[0002] Lane line detection technology is one of the important components of current high-level automatic assisted driving of vehicles. In complex urban road environments or obstructed driving environments, strong sunlight shines through the gaps in the obstructions, forming light projections of various shapes on the road surface, interfering with lane line detection. In particular, sunlight shining through the gaps forms long strips of strong light on the road surface, which poses a challenge to lane line detection methods based on camera observation. When a vehicle passes through a scene with strong light interference, the strong light will interfere with lane line detection to a certain extent, reducing detection accuracy and affecting the safety of autonomous driving and road driving safety.

[0003] Existing lane line detection methods are designed for use in environments with ample and soft lighting, on wide, unobstructed roads, with clear contrast between lane markings and lanes. However, in complex road environments or obstructed driving environments, light passes through gaps in overhead obstructions, creating complex light projections on the lanes. Vehicles traveling under overpasses are particularly susceptible to the strong light that passes through the gaps, creating a straight line of strong light that is almost flush with the lane line. This presents a difficult challenge for lane line detection that relies on camera image detection technology. Existing lane line detection methods, affected by the aforementioned strong light interference, may mistakenly identify straight lines of strong light as lane lines, and the detected lane lines may deviate from the actual ones. This results in low lane line detection accuracy in environments with strong light interference. Existing lane line detection methods also ignore the image's illumination dispersion coefficient as a criterion for distinguishing images when identifying whether the current image is in a strong light environment. In addition, existing lane line detection methods enhance the entire image in the face of strong light interference, resulting in less than ideal lane line recognition in the presence of straight lines of strong light interference. Summary of the Invention

[0004] The purpose of the present invention is to provide a lane line detection method for strong light interference based on an image restoration algorithm, so as to solve the problem that existing methods do not consider the impact of strong light interference on lane line detection.

[0005] To achieve the above objectives, the specific technical solution of the method for detecting lane lines interfered with by strong light based on an image restoration algorithm of the present invention is as follows:

[0006] A method for detecting lane lines interfered with by strong light based on an image restoration algorithm includes the following steps, which are performed in sequence:

[0007] Step S1: Obtain a video image using a monocular vision camera at the front of the car, extract the image every other frame according to the real-time speed of the vehicle, and process it into PNG format;

[0008] Step S2: Perform ROI processing on the extracted current frame image, perform filtering operation with a binary mask, select the road part in the image as the region of interest, calculate the brightness mean and standard deviation of the ROI area, and then calculate the dispersion coefficient as the judgment condition of the strong light environment. If the calculated dispersion coefficient is higher than the threshold, proceed to step S3, if the calculated dispersion coefficient is less than or equal to the threshold, proceed to step S4; this method takes into account the brightness mean and standard deviation, calculates the brightness dispersion coefficient, and the dispersion coefficient can effectively reflect the brightness difference. Through test calculations, it was found that the dispersion coefficient of the case using this patent method is 0.97, and the dispersion coefficient of normal roads is 0.21. The larger the dispersion coefficient, the greater the strong light interference. The threshold set by this patent is 0.45, so the image of the case of this patent is interfered with by strong light. And this method is applicable to strong light of different shapes, and can effectively judge strong light interference;

[0009] Step S3: Demarcate the areas of strong light interference and generate a mask. Based on a fast, non-iterative image inpainting method, the image is inpainted using incoherent pixels in adjacent areas to eliminate strong light interference. This algorithm uses a fast advance method, traversing the inpainted area only once while transferring image values ​​along the coherent direction of the robustly estimated structure tensor, i.e., using the inpaintCoherent algorithm to eliminate strong light interference. By inpainting the areas of strong light interference, the image is normalized, allowing lane detection. Existing lane detection methods do not consider image inpainting. By inpainting the image, the vehicle's external perception sensors can be applied to a wider range of applications, improving system reliability.

[0010] Step S4: The image is converted from RGB channels to HSV channels, and global equalization is performed on each channel of the HSV channel to achieve global equalization of the color image. By changing the histogram of the image, the brightness of each pixel in the image is changed, and the contrast of the image with a small dynamic range is enhanced. The original image is not clear enough due to its uneven brightness distribution. By using global histogram equalization, the histogram of the original image can be transformed into a uniformly distributed form, and the color information of the image is retained, thereby achieving the effect of enhancing the overall contrast of the image; by converting the RGB image format to the HSV format and performing histogram equalization on each channel, global histogram equalization is achieved. The advantage is that the color information of the image is retained, the global contrast is improved, and the image is clearer. Some of the existing lane line detection methods perform histogram equalization and do not retain color information. This method can improve the reliability of the system;

[0011] Step S5: grayscale processing is performed on the enhanced image, and the Laplacian of Gaussian (LOG operator) is further used for image edge detection with a convolution kernel size of 5×5, and then mean filtering is performed to process the noise in the 3×3 range of the image;

[0012] Step S6: Calculate the brightness mean and variance after mean filtering, construct a threshold constraint to perform image binarization and select the ROI image of the area to be detected;

[0013] Step S7: Use the Hough transform detection algorithm to detect lane lines. Classify the detection results based on the positive and negative slopes of the lines, and filter outliers. In each category, find the endpoints of the line with the longest distance and perform line fitting to draw the lane lines.

[0014] Furthermore, in step S1, extracting images every other frame according to the real-time speed of the vehicle and processing them into PNG format includes the following steps, which are performed in sequence:

[0015] Step S11: Calculate and extract real-time frame interval

[0016] Define the frame rate of the monocular vision camera as f and the real-time displayed speed of the vehicle as v. To ensure the real-time requirements of lane line detection during vehicle driving, the calculated real-time frame interval is:

[0017]

[0018] Step S12: According to the calculated extraction real-time frame interval t, the current real-time image is extracted every t frames and processed into PNG format.

[0019] Furthermore, in step S2, the brightness mean and standard deviation of the ROI area are calculated and then the dispersion coefficient is calculated as the judgment condition of the strong light environment, which includes the following steps, and the following steps are performed in sequence:

[0020] Step S21: Calculate the brightness mean of the ROI area as mean_value and the standard deviation as std_value, and then calculate the dispersion coefficient:

[0021]

[0022] Step S22: Set the coefficient of dispersion threshold to 0.45. If c>0.45, proceed to step S3; if c≤0.45, proceed to step S4.

[0023] Furthermore, in step S4, the image is converted from RGB channels to HSV channels and each channel of the HSV channel is equalized. The global equalization of the color image includes the following steps:

[0024] Step S41: Convert the image from RGB channels to HSV channels, and process the V channel component to the range of [0, 255];

[0025] Step S42: Calculate the histogram distribution and frequency of V, calculate the cumulative distribution frequency of V, equalize the V channel component, and restore it to the range of [0,1]. The statistical frequency distribution formula and equalization formula are:

[0026]

[0027] where r k is the number of gray levels, M and N are the number of rows and columns of pixels respectively, n k is the number of times the current gray value appears, L is the maximum gray level, s k is the grayscale level after equalization;

[0028] Step S43: Replace the original unequalized V channel component with the equalized V channel component, convert the image from the HSV space back to the RGB space, and output the equalized image.

[0029] Furthermore, in step S5, the 5×5 discrete convolution kernel of the image edge detection operator using the Gaussian-Laplacian operator is:

[0030]

[0031] Since the Laplace operator is highly sensitive to image noise, better detection results can be achieved by first performing Gaussian blur processing on the image and then performing edge detection.

[0032] Furthermore, in step S6, the brightness mean and variance after mean filtering are calculated, a threshold constraint is constructed, the threshold is set as the sum of the mean and the variance, and then binarization is performed, and the ROI image of the area to be detected is selected.

[0033] Furthermore, the step S7 includes the following steps, which are performed in sequence:

[0034] Step S71: Detect and process lane lines using the Hough transform line detection algorithm to obtain a structure storing endpoint coordinate information. Calculate the slope of each line and store it in vector k. Remove outlier slopes using outlier filtering.

[0035] Step S72: The obtained slope is classified into positive and negative categories and stored in two vectors respectively. By traversing the longitudinal coordinates of the endpoints, the two endpoints with the farthest longitudinal distance are found, and a straight line is fitted to draw the lane line.

[0036] The method for detecting lane lines with strong light interference based on an image restoration algorithm of the present invention has the following advantages:

[0037] 1) The present invention proposes that by setting a region of interest (ROI) and calculating the corresponding brightness dispersion coefficient, it can effectively determine whether the current image is disturbed by strong light, and enter the image restoration process for images disturbed by strong light.

[0038] 2) The present invention proposes to obtain video images based on the monocular vision camera at the front of the car and extract images every other frame according to the real-time speed of the vehicle. This can reduce the number of images to be processed when the speed is slow, which is beneficial to reducing the computing power of the system.

[0039] 3) The present invention proposes using a Gaussian-Laplacian operator with a convolution kernel size of 5×5 for image edge detection, followed by mean filtering to process the noise in the 3×3 range of the image. This is conducive to more accurate edge detection of the image and better avoids the impact of noise on lane line detection.

[0040] 4) The present invention can effectively improve the lane line detection accuracy under the interference of external strong light, avoid misjudgment of lane line detection, and improve road driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Flowchart of the present invention.

[0042] Figure 2 Schematic diagram of lane line detection subject to strong light interference.

[0043] Figure 3 Schematic diagram of the ROI area selected for calculating the dispersion coefficient.

[0044] Figure 4 Schematic diagram of the selected mask area to be repaired.

[0045] Figure 5 Schematic diagram of image restoration after eliminating strong light interference.

[0046] Figure 6 This is a schematic diagram after HSV global equalization.

[0047] Figure 7 This is a schematic diagram of image edge detection using the LoG operator with a convolution kernel size of 5×5.

[0048] Figure 8 This is a schematic diagram after using mean filtering to eliminate noise.

[0049] Figure 9 Schematic diagram of the lane line detection area selected after binarization processing.

[0050] Figure 10Schematic diagram of lane line detection and fitting on the original image. DETAILED DESCRIPTION

[0051] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a strong light interference lane line detection method based on an image restoration algorithm of the present invention in conjunction with the accompanying drawings.

[0052] like Figure 1 As shown, the present invention first obtains a video image based on a monocular vision camera at the front of the car, and extracts images every other frame according to the real-time speed of the vehicle; by calculating the brightness dispersion coefficient of the area of ​​interest, it is determined whether the current image has strong light interference. If strong light interference exists, the image is repaired. By demarcating the area to be repaired and repairing the image, the strong light interference of the image can be significantly improved; then, image edge detection is performed based on the Gaussian-Laplacian operator, and the convolution kernel size is 3×3; then, mean filtering is performed, and a threshold-constrained binarization processing based on the brightness mean and variance after mean filtering is performed; finally, Hough transform detection and slope-based clustering processing are performed to mark lane lines. Through the above steps, lane line detection can be effectively achieved under strong light interference.

[0053] Example 1:

[0054] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting lane lines interfered by strong light based on an image restoration algorithm, comprising the following steps, which are performed in sequence:

[0055] Step S1: Obtain a video image using a monocular vision camera at the front of the car, extract the image every other frame according to the real-time speed of the vehicle, and process it into PNG format;

[0056] The specific steps include:

[0057] Step S11: Calculate and extract real-time frame interval

[0058] Define the frame rate of the monocular vision camera as f and the real-time displayed speed of the vehicle as v. To ensure the real-time requirements of lane line detection during vehicle driving, the calculated real-time frame interval is:

[0059]

[0060] Step S12: According to the calculated extraction real-time frame interval t, the current real-time image is extracted every t frames and processed into png format. The image interfered by strong light is obtained as shown in FIG. Figure 2 shown.

[0061] Step S2: Perform ROI processing on the extracted current frame image, use a binary mask to perform filtering operation, select the road part in the image as the region of interest, calculate the brightness mean and standard deviation of the ROI area, and then calculate the dispersion coefficient as the judgment condition of the strong light environment. If the calculated dispersion coefficient is higher than the threshold, proceed to step S3. If the calculated dispersion coefficient is less than or equal to the threshold, proceed to step S4 to obtain the ROI area as shown in the figure. Figure 3 As shown;

[0062] The specific steps include:

[0063] Step S21: Calculate the brightness mean of the ROI area as mean_value and the standard deviation as std_value, and then calculate the dispersion coefficient:

[0064]

[0065] Step S22: Set the coefficient of dispersion threshold to 0.45. If c>0.45, proceed to step S3; if c≤0.45, proceed to step S4.

[0066] Step S3: calibrate the strong light interference area to generate a mask. Based on a fast non-iterative image repair method, the incoherent pixels in the adjacent area are used to repair the image to eliminate the strong light interference. The algorithm uses a fast advancement method to traverse the repair area only once, and at the same time transmits the image value along the coherent direction of the robust estimation of the structure tensor, that is, the inpaintCoherent algorithm is used to eliminate the strong light interference, and the area to be repaired is obtained as follows: Figure 4 As shown in the figure, after image restoration to eliminate strong light interference Figure 5 As shown;

[0067] Step S4: The image is converted from RGB channels to HSV channels, and global equalization is performed on each channel of the HSV channel to achieve global equalization of the color image. By changing the histogram of the image, the brightness of each pixel in the image is changed, and the contrast of the image with a small dynamic range is enhanced. The original image is not clear enough due to its uneven brightness distribution. By using global histogram equalization, the histogram of the original image can be transformed into a uniformly distributed form and the color information of the image is retained, thereby achieving the effect of enhancing the overall contrast of the image;

[0068] The specific steps include:

[0069] Step S41: Convert the image from RGB channels to HSV channels, and process the V channel component to the range of [0, 255];

[0070] Step S42: Calculate the histogram distribution and frequency of V, calculate the cumulative distribution frequency of V, equalize the V channel component, and restore it to the range of [0,1]. The statistical frequency distribution formula and equalization formula are:

[0071]

[0072] where r k is the number of gray levels, M and N are the number of rows and columns of pixels respectively, n k is the number of times the current gray value appears, L is the maximum gray level, s k is the grayscale level after equalization;

[0073] Step S43: Replace the original V channel component with the equalized V channel component, and convert the image from HSV space back to RGB space, output the equalized image, and obtain the globally equalized image as shown in Figure 6 shown.

[0074] Step S5: grayscale processing is performed on the enhanced image, and the edge detection of the image is further performed using the Gaussian-Laplacian operator with a convolution kernel size of 5×5. Then, mean filtering is performed to process the noise in the 3×3 range of the image.

[0075] The 5×5 discrete convolution kernel of the image edge detection operator using the Gaussian-Laplacian operator is:

[0076]

[0077] Since the Laplace operator is highly sensitive to image noise, the image is first Gaussian blurred and then edge detected to achieve better detection results. The image after edge detection using the LoG operator is as follows: Figure 7 As shown, the image with mean filtering and noise elimination is obtained as Figure 8 shown.

[0078] Step S6: Calculate the brightness mean and variance after mean filtering, construct a threshold constraint to perform image binarization and select the ROI image of the area to be detected;

[0079] Calculate the brightness mean and variance after mean filtering, construct a threshold constraint, set the threshold as the sum of the mean and variance, then perform binarization processing, and select the ROI image of the area to be detected as follows Figure 9 shown.

[0080] Step S7: Use the Hough transform detection algorithm to detect lane lines. Classify the detection results based on the positive and negative slopes of the lines, and filter outliers. In each category, find the endpoints of the line with the longest distance and perform line fitting to draw the lane lines.

[0081] The specific steps include:

[0082] Step S71: Detect and process lane lines using the Hough transform line detection algorithm to obtain a structure storing endpoint coordinate information. Calculate the slope of each line and store it in vector k. Remove outlier slopes using outlier filtering.

[0083] Step S72: The slopes are classified into positive and negative categories and stored in two vectors respectively. By traversing the longitudinal coordinates of the endpoints, the two endpoints with the longest longitudinal distance are found, and a straight line is fitted to draw the lane line. The fitted lane line detection image is obtained as shown in the figure. Figure 10 shown.

[0084] It will be understood that the present invention is described by way of some embodiments, and it will be appreciated by those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are intended to be protected by the present invention.

Claims

1. A method for detecting lane lines interfered by strong light based on an image restoration algorithm, characterized in that: The following steps are included: And the following steps are carried out in sequence: Step S1: Obtain a video image using a monocular vision camera at the front of the car, extract the image every other frame according to the real-time speed of the vehicle, and process it into PNG format; Step S2: Perform ROI processing on the extracted current frame image, perform filtering operations using a binary mask, select the road portion in the image as the region of interest, calculate the brightness mean and standard deviation of the ROI region, and then calculate the dispersion coefficient as a judgment condition for the strong light environment. If the calculated dispersion coefficient is higher than the threshold, proceed to step S3; if the calculated dispersion coefficient is less than or equal to the threshold, proceed to step S4; Step S3: calibrate the strong light interference area to generate a mask, and use the incoherent pixels in the adjacent area to perform image repair to eliminate the strong light interference. The fast marching method is used to traverse the repair area only once, and at the same time transmit the image value along the coherent direction of the robust estimation of the structure tensor to eliminate the strong light interference; Step S4: the image is converted from RGB channels to HSV channels, and global equalization is performed on each channel of the HSV channel to achieve global equalization of the color image; Step S5: grayscale processing is performed on the enhanced image, and then the Gaussian-Laplacian operator is used for image edge detection with a convolution kernel size of 5×5, and then mean filtering is performed to process the noise in the 3×3 range of the image; Step S6: Calculate the brightness mean and variance after mean filtering, construct a threshold constraint to perform image binarization and select the ROI image of the area to be detected; Step S7: Use the Hough transform detection algorithm to detect lane lines. Classify the detection results based on the positive and negative slopes of the lines, and filter outliers. In each category, find the endpoints of the line with the longest distance and perform line fitting to draw the lane lines.

2. The method for detecting lane lines interfered with by strong light based on an image restoration algorithm according to claim 1, characterized in that: In the step S1, the image is extracted every other frame according to the real-time speed of the vehicle and processed into PNG format, including the following steps: And the following steps are carried out in sequence: Step S11: Calculate and extract real-time frame interval Define the frame rate of the monocular vision camera as f, the real-time displayed speed of the vehicle as v, and the calculated real-time frame interval as: Step S12: According to the calculated extraction real-time frame interval t, the current real-time image is extracted every t frames and processed into PNG format.

3. The method for detecting lane lines interfered with by strong light based on an image restoration algorithm according to claim 1, characterized in that: In step S2, the brightness mean and standard deviation of the ROI area are calculated and then the dispersion coefficient is calculated as the judgment condition of the strong light environment, which includes the following steps: And the following steps are carried out in sequence: Step S21: Calculate the brightness mean of the ROI area as mean_value and the standard deviation as std_value, and then calculate the dispersion coefficient: Step S22: Set the coefficient of dispersion threshold to 0.

45. If c>0.45, proceed to step S3; if c≤0.45, proceed to step S4.

4. The method for detecting lane lines interfered with by strong light based on an image restoration algorithm according to claim 1, wherein: The step S3 uses the inpaintCoherent algorithm to eliminate strong light interference.

5. The method for detecting lane lines interfered with by strong light based on an image restoration algorithm according to claim 1, characterized in that: In step S4, the image is converted from RGB channels to HSV channels and each channel of the HSV channels is equalized. The global equalization of the color image includes the following steps: Step S41: Convert the image from RGB channels to HSV channels, and process the V channel component to the range of [0, 255]; Step S42: Calculate the histogram distribution and frequency of V, calculate the cumulative distribution frequency of V, equalize the V channel component, and restore it to the range of [0,1]. The statistical frequency distribution formula and equalization formula are: where r k is the number of gray levels, M and N are the number of rows and columns of pixels respectively, n k is the number of times the current gray value appears, L is the maximum gray level, s k is the grayscale level after equalization; Step S43: Replace the unequalized V channel component with the equalized V channel component, convert the image from the HSV space back to the RGB space, and output the equalized image.

6. The method for detecting lane lines interfered with by strong light based on an image restoration algorithm according to claim 1, characterized in that: In step S5, the 5×5 discrete convolution kernel of the image edge detection operator using the Gaussian-Laplacian operator is:

7. The method for detecting lane lines interfered with by strong light based on an image restoration algorithm according to claim 1, characterized in that: The step S7 includes the following steps, which are performed in sequence: Step S71: Detect and process lane lines using the Hough transform line detection algorithm to obtain a structure storing endpoint coordinate information. Calculate the slope of each line and store it in vector k. Remove outlier slopes using outlier filtering. Step S72: The obtained slope is classified into positive and negative categories and stored in two vectors respectively. By traversing the longitudinal coordinates of the endpoints, the two endpoints with the farthest longitudinal distance are found, and a straight line is fitted to draw the lane line.

Images