Night lane line recognition method based on illumination compensation

Abstract

The application discloses a night lane line recognition method based on illumination compensation, which comprises the following steps: a weighted average value gray processing step based on lane line color features, an illumination compensation step of an image based on a parameter-adjusted logarithmic transformation method, and an illumination compensation step of a lane line image in a combined form. The application has the advantages of being able to accurately highlight and recognize lane lines under different brightness road low illumination.

Description

Technical Field

[0001] The present invention relates to the field of lane line recognition for intelligent connected vehicles, and in particular to a nighttime lane line recognition method based on illumination compensation. Background Art

[0002] As the automotive industry enters the 4.0 era, the intelligentization of vehicles is gaining increasing attention. Given the importance of lane recognition for autonomous vehicles, many scholars at home and abroad have conducted extensive research on lane recognition, but most of these studies have been conducted under normal lighting conditions.

[0003] Under normal lighting conditions, the road images collected by vehicles have high clarity, and the contrast between lane lines and background road surface is relatively obvious. The image can be processed using methods based on optical flow estimation and background modeling. However, if the lighting is poor at night, the contrast between lane lines and background is small, and the overall brightness of the image is low, the lane line recognition method under normal lighting conditions is used to process the image. The result cannot meet the needs of lane line recognition at night. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: in view of the problem that the lane line information collected by vehicles at night has poor lighting and low brightness, and thus the lane line recognition method under normal lighting cannot be used, a nighttime lane line recognition method based on illumination compensation is provided.

[0005] The present invention is achieved through the following technical solutions:

[0006] The nighttime lane line recognition method based on illumination compensation provided by the present invention comprises the following steps:

[0007] (1) Grayscale processing steps based on the weighted average value of lane line color features:

[0008] According to the color characteristics of lane lines in RGB space and the different sensitivity of human eyes to color space, a weighted average method based on lane line color characteristics is proposed to grayscale the image.

[0009] (2) The steps of performing illumination compensation on the image based on the logarithmic transformation method with parameter adjustment:

[0010] Select an intensity parameter based on the grayscale maximum value weighting and an adjustment parameter based on the overall brightness change of the image to adjust the image;

[0011] (3) Using a combination form to perform illumination compensation on the lane line image:

[0012] The local histogram equalization, logarithmic transformation based on parameter adjustment and global histogram equalization are combined to perform illumination compensation for lane line images under low illumination.

[0013] In step (1) of the above method, the following method can be used to grayscale the image:

[0014] 1) Based on the color characteristics of white and yellow lane markings in RGB space, only R and G spaces are considered in the subsequent weighted calculation formula;

[0015] 2) The specific weight distribution formula in the weighted average method based on human eye sensitivity is:

[0016] gray(i,j)=0.299*R(i,j)+0.587*G(i,j)+0.114*B(i,j)

[0017] Where gray(i,j) is the grayscale value of the point in the image after grayscale conversion; R(i,j) is the grayscale value of the point in the red channel; G(i,j) is the grayscale value of the point in the green channel; B(i,j) is the grayscale value of the point in the blue channel.

[0018] In the above method, the R and G space weights can be distributed according to the R and G weight ratio based on the sensitivity of the human eye to color:

[0019]

[0020] The above formula gives the weighted average grayscale formula based on lane line color features:

[0021] Gray(i,j)=0.337*R(i,j)+0.663*G(i,j)

[0022] Where: Gray(i,j) is the gray value of the point in the image obtained under the new image grayscale calculation formula.

[0023] The present invention is based on the fact that image brightness is not fixed, so the grayscale formula needs to have a range of variation for correction. Therefore, images at different brightness levels can be processed to obtain the R and G spatial weight distribution when the grayscale effect is best. Then, the data is fitted, and the obtained fitting straight line analytical formula is:

[0024] m=-0.003*B+0.504

[0025] Where m is the weight of the R space, and B is the average brightness of the image.

[0026] The present invention is based on the fact that the research environment is poorly illuminated. Therefore, the brightness range can be selected as 20 to 90. Substituting the linear analytical formula into the range of m is 0.234 to 0.444. Then, a fluctuation value of 0.1 is added to the grayscale formula to make it almost consistent with the range of m, that is:

[0027]

[0028] In step (2) of the above method, an intensity parameter c based on the grayscale maximum value weighting and an adjustment parameter A based on the overall optimal contrast of the image can be selected to adjust the image logarithmic transformation formula f(i,j)=c*log(A*g(i,j)+1), where f(i,j) and g(i,j) are the image output and input grayscale values, respectively;

[0029] The value of the intensity parameter c can be determined using the following formula:

[0030]

[0031] Where a and b are the maximum input grayscale values ​​g(i,j) max And the minimum input gray value g(i,j) min Weight coefficient, and a+b=1.

[0032] In the above method, while keeping A = 1, we can adjust the values ​​of a and b separately. The processed results show that the smaller a is, the more blurred the lane line becomes under illumination. Since the image contrast is constantly changing, the selection of the weight parameter for the grayscale maximum value also fluctuates. We selected 9 groups of images with different contrasts, ranging from 44.56 to 135.99. We found that the optimal values ​​of a for these 9 groups of contrast images are all between 0.98 and 1.00. Therefore, a is set to 0.99 and b is set to 0.01, with a fluctuation of 0.01. The value of the intensity parameter c is:

[0033]

[0034] In the above method, the following method can be used to determine the value of the optimal adjustment parameter A for the overall brightness change of the image:

[0035] Under the aforementioned conditions of a = 0.99 and b = 0.01, we determined the value of A. We found that the larger the value of A, the more likely the image would appear too bright. We processed nine sets of lane lines at different brightness levels and obtained the optimal A value for each brightness. We then fitted the data points, yielding the following analytical expression for the fitted line:

[0036] A=-0.0017*B+0.258.

[0037] In the above method, we can select an ambient brightness between 20 and 90 degrees, and obtain an A value of 0.105 to 0.224. Then, we use a baseline value of 0.16 for A, with a fluctuation of 0.5, for lane line recognition in poor lighting conditions:

[0038]

[0039] In step (3) of the above method, the image obtained after local histogram equalization and logarithmic transformation still has the phenomenon of foggy appearance as a whole. After canny edge detection, more redundant edge information will appear, affecting the subsequent lane line detection. Therefore, global histogram equalization is performed on the image obtained after the two image transformation processes. The three methods of local histogram equalization, logarithmic transformation based on parameter adjustment, and global histogram equalization are combined together to perform illumination compensation for the lane line image under low illumination.

[0040] Compared with the prior art, the present invention has the following main advantages:

[0041] 1. This paper proposes a weighted average grayscale processing method based on lane marking color characteristics. Based on the characteristics of lane marking colors in RGB space and the human eye's sensitivity to color, this grayscale method can better highlight lane marking information when processing road images.

[0042] 2. We propose a parameter-adjusted logarithmic transformation method for image illumination compensation. We select an intensity parameter weighted by the grayscale maximum value and an adjustment parameter based on the overall brightness of the image to adjust the image. A fluctuation range is proposed, and the results show that it effectively highlights the low-grayscale features of the image.

[0043] 3. Combining local histogram equalization, parameter-adjusted logarithmic transformation, and global histogram equalization to compensate for lane line images under low illumination can effectively filter out redundant information and accurately identify lane lines.

[0044] In summary, the present invention has the advantages of being able to accurately highlight and identify lane lines under low illumination on roads of different brightness. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic flow diagram of the present invention.

[0046] Figure 2 is the grayscale image of the road.

[0047] Figure 3 is the fitting image of the optimal m value under different brightness.

[0048] Figure 4 is the local histogram equalization image.

[0049] Figure 5 It is the logarithmic change image when A is 1, Figure 5 In the figure: (a) is the image when a=1, b=0; (b) is the image when a=0.8, b=0.2; (c) is the image when a=0.5, b=0.5; (d) is the image when a=0.2, b=0.8.

[0050] Figure 6 It is the logarithmic transformation image when a is 0.99 and b is 0.01. Figure 6 In the figure: (a) is the image when A=2; (b) is the image when A=1; (c) is the image when A=0.5; (d) is the image when A=0.2; (e) is the image when A=0.1; (f) is the image when A=0.05.

[0051] Figure 7 It is the fitting image of the optimal A value under different brightness.

[0052] Figure 8 It is a canny edge detection image, where: (a) is an image without global histogram equalization; (b) is an image with global histogram equalization.

[0053] Figure 9 It is the lane line recognition image after Hough transform. DETAILED DESCRIPTION

[0054] The present invention will be further described in detail below with reference to the accompanying drawings, which are intended to explain rather than limit the present invention.

[0055] The present invention discloses a nighttime lane line recognition method based on illumination compensation. First, bilateral filtering and denoising are performed on the acquired road image, which is obtained using an on-board camera. A weighted average grayscale processing method based on the color characteristics of the lane lines is then used. Local histogram equalization is first performed on the grayscale image. A logarithmic transformation method using an intensity parameter weighted by the grayscale maximum value and an adjustment parameter based on the overall optimal brightness change of the image is then used to perform illumination compensation on the image, highlighting the low-grayscale features of the image. Global histogram equalization is then performed on the image to achieve the purpose of illumination compensation. Canny edge detection and Hough transform are then performed on the image to identify lane lines. Finally, the identified lane lines are fitted. Results show that the illumination compensation algorithm, which uses local, then logarithmic, then global methods, can effectively identify lane lines under low illumination and accurately fit the lane lines, improving the ability to recognize lane lines under poor lighting conditions.

[0056] The nighttime lane line recognition method based on illumination compensation provided by the present invention includes image denoising, grayscale conversion, illumination compensation, edge detection, and Hough transform lane line fitting.

[0057] The image denoising provided by the present invention, whether it is mean filtering, median filtering or Gaussian filtering, will make the boundary of the entire image blurred. Therefore, in order to effectively protect the boundary information, a bilateral filter is used, which automatically determines whether the filter kernel is in a flat area or an edge area, and well preserves the boundary information.

[0058] The image denoising method uses the bilateralFilter() method in OpenCV to directly perform bilateral filtering on the image.

[0059] The most commonly used grayscale methods are the component method, the weighted average method, and the maximum method. Based on the characteristics of lane marking colors in RGB space and the varying sensitivity of the human eye to color space, a weighted average method is proposed for grayscaling lane marking images.

[0060] Since the lane lines are both white and yellow, and these two colors are (255, 255, 255) and (255, 255, 0) respectively in the RGB space, considering the grayscale value characteristics of the two lane line colors, only the R and G spaces are considered, which helps to highlight the lane lines after grayscale processing.

[0061] The weighted average method is based on the different sensitivity of the human eye to various colors, and different weights are assigned to the three components of RGB. The specific weight distribution in the weighted average method based on human eye sensitivity is shown in the formula:

[0062] gray(i,j)=0.299*R(i,j)+0.587*G(i,j)+0.114*B(i,j) (1)

[0063] Where gray(i,j) is the grayscale value of the point in the image after grayscale conversion; R(i,j) is the grayscale value of the point in the red channel; G(i,j) is the grayscale value of the point in the green channel; B(i,j) is the grayscale value of the point in the blue channel.

[0064] Considering only the R and G spaces, the weights of the R and G spaces are distributed according to the R and G weight ratio relationship based on the human eye's sensitivity to color in formula (1):

[0065]

[0066] Where Gray(i,j) is the gray value of the point in the image obtained by the new image grayscale calculation formula.

[0067] The weighted average grayscale formula based on lane line color features is obtained as follows:

[0068] Gray(i,j)=0.337*R(i,j)+0.663*G(i,j) (3)

[0069] Since the image brightness is not fixed, the grayscale formula also needs to have a range of variation for correction. The images at different brightness are processed to obtain the R and G space weight distribution when the grayscale effect is best. Then the above data is fitted, and the fitting straight line analytical formula is obtained:

[0070] m=-0.003*B+0.504 (4)

[0071] Where: m is the weight of the R space, and B is the average brightness of the image.

[0072] Since the research environment is poorly illuminated, the brightness range is selected as 20 to 90. Substituting the linear analytical formula into the above grayscale formula (3), the range of m is 0.234 to 0.444. In this way, a fluctuation value of 0.1 is added to the grayscale formula (3), that is:

[0073]

[0074] The illumination compensation described above first performs local histogram equalization on the grayscale image, then uses an improved logarithmic transformation algorithm to transform the original grayscale values ​​into grayscale values ​​in the logarithmic domain, replacing the original grayscale values, highlighting the low-grayscale features of the image, and then performs global histogram equalization on the transformed image to achieve the purpose of illumination compensation for the image.

[0075] First, local histogram equalization is used to transform the image so that the overall image brightness becomes more uniform.

[0076] On the basis of local histogram equalization, the logarithmic transformation method is used to transform the original grayscale value into the grayscale value in the logarithmic domain to replace the original grayscale value. By partially expanding the low grayscale value of the image and partially compressing the high grayscale value of the image, the low grayscale features of the image can be highlighted, thereby achieving the purpose of illumination compensation for the image.

[0077] The basic form of the logarithmic transformation is:

[0078] f(i,j)=log(g(i,j)+1) (6)

[0079] Where f(i,j) and g(i,j) are the image output and input grayscale values ​​respectively.

[0080] Multiplying the right side of equation (6) by a strength parameter c yields:

[0081] f(i,j)=c*log(g(i,j)+1) (7)

[0082] However, relying solely on the intensity parameter c for adjustment does not significantly improve the image contrast. The lane line recognition under illuminated conditions differs significantly from that under poorly illuminated conditions. Therefore, the adjustment parameter A can be pre-multiplied in formula (7) so that the compensation result can be adjusted by changing the value of A, as shown in the following formula:

[0083] f(i,j)=c*log(A*g(i,j)+1) (8)

[0084] The value of c is related to the maximum grayscale value g(i,j) max and the minimum grayscale value g(i,j) min Related, take Where a and b are the maximum input grayscale values ​​g(i,j) max And the minimum input gray value g(i,j) min The weight coefficient is a + b = 1. Keeping the adjustment parameter A = 1, we change the values ​​of a and b. The processed results show that the smaller a is, the more blurred the lane lines become under illumination. Since image contrast is constantly changing, the selection of the weight parameter for the grayscale maximum value also fluctuates. We selected 9 groups of images with different contrasts, ranging from 44.56 to 135.99. We found that the optimal a values ​​for these 9 groups of contrast images were all between 0.98 and 1.00. Therefore, we selected a value of 0.99, with a fluctuation of 0.01, that is:

[0085]

[0086] Under the above conditions of a = 0.99 and b = 0.01, we took the value of A. We found that the larger the value of A, the brighter the image would be. We processed 9 groups of lane lines at different brightness and obtained the optimal A value at each brightness. We then fitted the data points, as shown in the figure below. Figure 7 As shown, the analytical expression of the fitting straight line is:

[0087] A=-0.0017*B+0.258 (10)

[0088] Where B is the average brightness of the image, and A is the optimal value under this average brightness. Similarly, since lane line recognition is performed under poor lighting conditions, the ambient brightness is selected to be between 20 and 90. The resulting A value range is 0.105 to 0.224, so the baseline value of A is selected to be 0.16, with a fluctuation range of 0.5, that is:

[0089]

[0090] The image obtained after local histogram equalization and logarithmic transformation still has an overall foggy appearance. Canny edge detection also produces a lot of redundant edge information, which affects subsequent lane detection. Therefore, global histogram equalization is performed on the image obtained after the two image transformations. The results show that the image obtained after global histogram equalization has significantly reduced redundant information after Canny edge detection, and can effectively identify lane edges in poor lighting conditions.

[0091] To facilitate subsequent lane line positioning and fitting, edge detection must first be performed on the processed image. Common edge detection operators include the Sobel operator and the Canny operator. Comparing the detection results of the Canny operator and the Sobel operator, edge detection using the Canny operator is less susceptible to noise interference and can better detect true weak edges.

[0092] The Hough transform, described above, detects lane lines by leveraging the duality between points and lines. It transforms a given curve in the original image into a point in parameter space through a curve representation. Lines are represented using polar coordinates. Therefore, each point (r, θ) in the polar coordinate system represents a line in the rectangular coordinate system.

[0093] r represents the vertical distance from the origin to the line, and θ represents the angle between the line and the positive x-axis. θ ranges from (0 to 180°). This transforms the problem of detecting curves in the original image into finding peaks in parameter space, achieving dimensionality reduction. The HoughLines function in OpenCV can be used to directly detect lines using the Hough transform.

[0094] The present invention improves the ability to recognize lane lines under poor lighting conditions by proposing the following methods, namely, a new image grayscale calculation method, a new logarithmic transformation method, a new calculation formula for logarithmic transformation intensity parameters, a new calculation formula for logarithmic transformation adjustment parameters, and a combined illumination compensation method, thereby contributing to the intelligence of automobiles and the promotion of autonomous driving.

Claims

1. A nighttime lane recognition method based on illumination compensation, characterized in that: The following steps are involved: (1) Grayscale processing steps based on the weighted average value of lane line color features: According to the color characteristics of lane lines in RGB space and the different sensitivity of human eyes to color space, a weighted average method based on lane line color characteristics is proposed to grayscale the image. (2) Steps for image illumination compensation based on the logarithmic transformation method with parameter adjustment: Select an intensity parameter based on the grayscale maximum value weighting and an adjustment parameter based on the overall brightness change of the image to adjust the image; (3) Use a combination of methods to perform illumination compensation on the lane line image: The local histogram equalization, logarithmic transformation based on parameter adjustment and global histogram equalization are combined to perform illumination compensation for lane line images under low illumination.

2. The nighttime lane line recognition method based on illumination compensation according to claim 1 is characterized in that In step (1), the image is grayscaled using the following method: 1) Based on the color characteristics of white and yellow lane markings in RGB space, only R and G spaces are considered in the subsequent weighted calculation formula; 2) The specific weight distribution formula in the weighted average method based on human eye sensitivity is: ， in is the grayscale value of the image interior point after grayscale conversion; is the gray value of the point in the red channel; is the gray value of the point in the green channel; is the grayscale value of the point in the blue channel.

3. The nighttime lane line recognition method based on illumination compensation according to claim 2 is characterized in that The R and G space weights are distributed according to the R and G weight ratio based on the human eye's sensitivity to color: ， The above formula gives the weighted average grayscale formula based on lane line color features: ， in: It is the gray value of the point in the image obtained by the new image grayscale calculation formula.

4. The nighttime lane line recognition method based on illumination compensation according to claim 2 is characterized in that The images at different brightness levels are processed to obtain the R and G spatial weight distribution when the grayscale effect is best. Then the data is fitted and the analytical expression of the fitted straight line is obtained as follows: Where m is the weight of the R space, and B is the average brightness of the image.

5. The nighttime lane line recognition method based on illumination compensation according to claim 4 is characterized in that The brightness range is selected as 20-90. Substituting it into the linear analytical formula, the range of m is 0.234-0.

444. Then, a fluctuation value of 0.1 is added to the weighted average grayscale formula based on the lane color characteristics to make it almost consistent with the range of m, that is: 。 6. The nighttime lane line recognition method based on illumination compensation according to claim 1 is characterized in that In step (2), an intensity parameter c based on the grayscale maximum value weighting and an adjustment parameter A based on the overall optimal contrast of the image are selected to adjust the image logarithmic transformation formula Adjust, where 、 are the image output and input grayscale values ​​respectively; The value of the intensity parameter c is determined using the following formula: Where a and b are the maximum input grayscale values and the minimum input grayscale value weight coefficient, and .

7. The nighttime lane line recognition method based on illumination compensation according to claim 6 is characterized in that Keeping A=1 unchanged, select 9 groups of images with different contrasts, with contrast ranging from 44.56 to 135.

99. The optimal value of a is 0.98 to 1.00, so a is 0.99 and b is 0.01, with a fluctuation of 0.

01. The value of intensity parameter c is: 。 8. The nighttime lane line recognition method based on illumination compensation according to claim 7 is characterized in that The following method is used to determine the optimal adjustment parameter A for the overall brightness change of the image: Under the conditions of a=0.99 and b=0.01, we take the value of A and obtain the optimal A value for the lane line under different ambient brightness. Then, we fit the data points and obtain the analytical expression of the fitting line: 。 9. The nighttime lane line recognition method based on illumination compensation according to claim 8 is characterized in that Selecting an ambient brightness between 20 and 90, we obtain an A value of 0.105 to 0.

224. Then, we use a baseline value of 0.16 for A with a fluctuation of 0.5 for lane line recognition in poor lighting conditions: 。 10. The nighttime lane line recognition method based on illumination compensation according to claim 1 is characterized in that In step (3), the image obtained after local histogram equalization and logarithmic transformation still has the phenomenon of overall fogging. After canny edge detection, more redundant edge information will appear, affecting the subsequent lane line detection. Therefore, global histogram equalization is performed on the image obtained after the two image transformation processes. The three methods of local histogram equalization, logarithmic transformation based on parameter adjustment, and global histogram equalization are combined together to perform illumination compensation for the lane line image under low illumination.

Images