Nighttime road image enhancement method based on threshold partitioning and weighted brightness component

Abstract

The present invention provides a method for enhancing nighttime road images based on threshold partitioning and weighted brightness components, comprising the following steps: dividing the V component of the HSV color space into a dark area and a bright area according to a threshold value obtained by a maximum entropy method; for the dark area, using an improved cumulative distribution function of a hyperbolic secant distribution to improve its brightness; for the bright area, using an improved hyperbolic tangent S-type function to enhance its brightness; and converting the enhanced V component into a dark area and a bright area. new The (i, j) component is fused with the H and S components of the original input image and converted back to RGB color space. Illumination compensation is performed using the nonlinear transformation function LC. A dual-scale mean filter is used to enhance image detail, resulting in a final night-time enhanced image. This method improves the brightness of dark areas in nighttime images while preventing over-enhancement of bright areas. It also enables adaptive enhancement without manual parameter adjustment, ensuring the real-time performance of intelligent driving.

Description

Technical Field

[0001] The present invention relates to the technical field of nighttime image enhancement, in particular to a nighttime road image enhancement method based on threshold partitioning weighted brightness components. Background Art

[0002] Computer vision technology is a key component of intelligent driving systems. Especially in low-light conditions, such as at dusk or at night, captured images often lack object shape and high-dimensional texture information. They suffer from low brightness and contrast, blurred details, and high noise levels. This makes it difficult for image detection and recognition systems to accurately and clearly identify useful information. To address these issues, quality enhancement processing is required for acquired nighttime road images. The ultimate goal of nighttime image enhancement is to improve image brightness and contrast, suppress noise amplification, and restore more image detail.

[0003] Traditional methods include histogram equalization, Retinex theory-based methods, and mathematical model-based methods. In recent years, domestic and foreign scholars have conducted relevant research on visual image processing technology in nighttime environments. In the paper "Retinex Low-Illumination Image Enhancement Algorithm Based on Illumination Map Estimation", based on Retinex theory, it is proposed to use the L2 norm to initialize the illumination map, and improve the RTV model to refine the illumination map, thereby enhancing the contrast of the image. However, the model has poor robustness and the algorithm is time-consuming. In the paper "Low-Illumination Image Enhancement Algorithm Based on Illumination Component Correction and Compensation", the illumination map is obtained by processing the brightness component using multi-scale gradient domain guided filtering and two-dimensional adaptive gamma correction. This algorithm improves the overall brightness of the image, but the enhancement of the shadow area is not obvious.

[0004] In addition, deep learning has also been gradually applied to the field of computer vision and has achieved some results. Existing technologies use neural networks to train weight parameters and learn image features to achieve image enhancement effects.

[0005] However, when using histogram equalization to process low-light nighttime images, the histogram distribution of an image is transformed into a nearly uniform distribution. Since nighttime images are generally darker and have a higher prevalence of low pixel values, histogram equalization expands the dynamic range of pixel values, resulting in a reduction in the grayscale level of the transformed image and a loss of image detail. Since nighttime road images are often interfered with by artificial light sources (such as car lights, streetlights, and reflective building materials), the captured images can contain artifacts, resulting in peaks in the histogram and, after processing, over-enhancement or exposure.

[0006] When estimating illumination images, Retinex theory assumes that the initial illumination image is slowly changing, meaning it is smooth. However, at edges with large brightness differences, the illumination changes are non-smooth. In this case, the Retinex enhancement algorithm is prone to producing halos at edges with large brightness differences. Furthermore, both single-scale and multi-scale Retinex algorithms operate channel by channel. For RGB color images, this approach fails to account for the relationship between different color channels, resulting in some images experiencing color casts and distortion after processing, which has limitations for color images.

[0007] Methods based on mathematical models are more based on various prior conditions, and their models have limited practical conditions and poor robustness.

[0008] Although the image enhancement method based on deep learning is effective, the training model has high requirements for experimental equipment, and the collection and establishment of data sets is also a difficult problem.

[0009] In summary, the existing night image enhancement technology has the following main defects:

[0010] (1) Most existing methods use a global, indifferent approach to enhance nighttime images. However, due to the presence of light sources such as streetlights, vehicle lights, and building reflections on nighttime roads, this can easily lead to over-enhancement of light source areas while enhancing the brightness of dark areas of the image, resulting in exposure.

[0011] (2) Intelligent driving systems perform dynamic path, image recognition, and system operations in real time. However, some existing methods require manual parameter adjustment during operation, which does not meet the real-time performance requirements of intelligent driving. Summary of the Invention

[0012] To overcome the shortcomings of the existing technology, the present invention provides a nighttime road image enhancement method based on threshold partitioning and weighted brightness components. This method can not only improve the brightness of dark areas in nighttime images, but also avoid over-enhancement of bright areas (such as lights, building reflections, etc.). It can also achieve adaptive enhancement without manual parameter adjustment, and can meet the real-time performance requirements of intelligent driving.

[0013] The technical solution adopted by the present invention to solve the technical problem is: a nighttime road image enhancement method based on threshold partitioning weighted brightness component, comprising the following steps:

[0014] S1. Using the maximum entropy method to obtain a threshold value for the original input image, the original input image is converted from the RGB color space to the HSV color space, and the V component of the HSV color space is divided into dark and light areas according to the threshold value obtained by the maximum entropy method;

[0015] S2. For dark areas, the improved cumulative distribution function of the hyperbolic secant distribution is used to increase their brightness. The cumulative distribution function is defined as follows:

[0016]

[0017] Among them, (i, j) is the position index of the pixel point, V new-CDF-HSD (i, j) is the enhanced brightness component, and V(i, j) is the V component after conversion to HSV color space; is the weight matrix, and the formula is:

[0018]

[0019] For bright areas, an improved hyperbolic tangent sigmoid function is used to enhance their brightness. The hyperbolic tangent sigmoid function is defined as follows:

[0020]

[0021] Among them, V new-tan-S (i, j) is the result of mapping the V component using the improved hyperbolic tangent sigmoid function; λ is a scalar that controls the enhancement process, and the formula is:

[0022]

[0023] Where s(X) is the grayscale value variance of the original input image;

[0024] S3. The enhanced V new The (i, j) component is fused with the H and S components of the original input image and converted back to the RGB color space. The nonlinear transformation function LC is used for illumination compensation. The LC function is defined as follows:

[0025]

[0026] Where, I in The image is converted back to RGB color space after brightness enhancement;

[0027] S4. Use dual-scale mean filtering to enhance image details and obtain the final night-time enhanced image.

[0028] Furthermore, after performing illumination compensation on the image in step S3, the color cast of the image is adjusted using the grayscale world method.

[0029] Furthermore, the specific operation of adjusting the color deviation of the image by the gray world method is:

[0030] Calculate the mean of the RGB three channels The constant K can be expressed as:

[0031]

[0032] Gain coefficient of three channels:

[0033]

[0034] The adjusted RGB values ​​are R′, G′, B′:

[0035] R′=β R *R, G′=β G *G, B′=β B *B.

[0036] Furthermore, the specific operation of dividing the V component of the HSV color space into dark areas and bright areas according to the threshold value obtained by the maximum entropy method in step S1 is:

[0037] Image entropy definition:

[0038]

[0039] Where H(I) is the image entropy, (u, v) is the position index of the pixel, g = I(u, v), I is the input image, and p(g) is the probability of pixel (u, v). Given a threshold q, the cumulative probability P of the dark and bright areas segmented by the q threshold is calculated. L (q), P D (q), the sum of the two is 1; the entropy corresponding to the two regions is expressed as follows:

[0040]

[0041]

[0042] Where H L (q), H D (q) represents the entropy of the bright area and the dark area respectively;

[0043] Under the threshold q, the total entropy of the image is:

[0044] H(q)=H L (q)+H D (q)

[0045] H=max(H(q))

[0046] Calculate the total entropy H(q) of the image under all thresholds, and take the threshold corresponding to the maximum entropy H=max(H(q)) as the optimal threshold T;

[0047] In the HSV color space, the area where the brightness component V pixel value t≤T is divided into the dark area, and the area where the brightness component V pixel value t>T is divided into the dark area.

[0048] Furthermore, the specific operations of step S4 are:

[0049] Decompose the image into layers of different scales by mean filtering:

[0050] B=X*Z

[0051] Wherein, X is the enhanced image obtained after processing in step S3, Z is the mean filter, and B is the image after convolution with the mean filter, which represents the image base layer;

[0052] Subtracting the base layer from the enhanced image yields the detail layer D, i.e.:

[0053] D=XB

[0054] Add the detail layer D to the image obtained after processing in step S3, and finally output image I output :

[0055] I output =X+D

[0056] I output This is the final night-time enhanced image.

[0057] Positive effects of the present invention:

[0058] (1) The brightness component V is partitioned by the optimal threshold obtained by the maximum entropy method. At the same time, the improved cumulative distribution function of the hyperbolic secant distribution is used to improve the brightness of the dark area, and the improved hyperbolic tangent S-type function is used to enhance the brightness of the bright area. This method can not only improve the low-intensity and medium-intensity pixels, but also suppress the excessive increase of high-intensity pixels, thereby enhancing the overall brightness of the night image and improving the contrast.

[0059] (2) The present invention uses a nonlinear transformation function LC to perform illumination compensation on the image, thereby improving the color distortion problem of the night image.

[0060] (3) Adaptive parameters do not require manual adjustment, thus saving computing time. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 1 is a flow chart of the nighttime road image enhancement method of the present invention;

[0062] Figure 2a is the function image corresponding to different λ values;

[0063] Figure 2b This is a comparison of the function images before and after improvement;

[0064] Figure 3a is the original input image;

[0065] Figure 3bis the final output image;

[0066] Figure 3c is the histogram of the original input image;

[0067] Figure 3d is the histogram of the final output image. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0069] Reference Figure 1 The preferred embodiment of the present invention provides a method for enhancing nighttime road images based on threshold partitioning weighted brightness components, comprising the following steps:

[0070] S1. Use the maximum entropy method to obtain a threshold value for the original input image, convert the original input image from the RGB color space to the HSV color space, and divide the V component of the HSV color space into dark and light areas according to the threshold value obtained by the maximum entropy method. The specific operation is as follows:

[0071] Image entropy definition:

[0072]

[0073] Where H(I) is the image entropy, (u, v) is the position index of the pixel, g = I(u, v), I is the input RGB image, and p(g) is the probability of pixel (u, v). Given a threshold q, the cumulative probability P of the dark and bright areas segmented by the q threshold is calculated. L (q), P D (q), the sum of the two is 1; the entropy corresponding to the two regions is expressed as follows:

[0074]

[0075]

[0076] Where H L (q), H D (q) represents the entropy of the bright area and the dark area respectively;

[0077] Under the threshold q, the total entropy of the image is:

[0078] H(q)=H L (q)+H D (q)

[0079] H=max(H(q))

[0080] Calculate the total entropy H(q) of the image under all thresholds, and take the threshold corresponding to the maximum entropy H=max(H(q)) as the optimal threshold T;

[0081] In the HSV color space, the area where the brightness component V pixel value t≤T is divided into the dark area, and the area where the brightness component V pixel value t>T is divided into the bright area.

[0082] A distinct characteristic of nighttime images is their overall low brightness, so enhancing brightness is the primary task in nighttime image enhancement. Existing enhancement methods often rely on global enhancement, which can easily lead to over-enhancement of bright areas. During brightness enhancement, it is necessary to increase the intensity of dark areas while suppressing over-enhancement of bright areas to avoid overexposure. Therefore, this patent proposes using the maximum entropy principle to divide the image into two parts: dark and bright areas, and process them separately.

[0083] In addition to the maximum entropy method, other threshold segmentation methods exist, such as the global threshold method and the maximum inter-class variance method. The global threshold method works well only when the grayscale values ​​of the object and the background remain unchanged, but more often the lighting conditions are uneven. The maximum inter-class variance method is sensitive to image noise and can only segment a single target. When the size ratio of the target and background is very different, the inter-class variance function may be bimodal or multimodal, making it impossible to accurately design a threshold, resulting in unsatisfactory segmentation results. Both algorithms require manual threshold determination. The advantages of maximum entropy principle segmentation: automatic threshold determination, segmentation based on the amount of information contained, and the most accurate distinction between bright and dark areas.

[0084] S2. For dark areas, the cumulative distribution function of the hyperbolic secant distribution (CDF-HSD) is used to increase the brightness. This type of function is smooth and gradual, and maintains monotonous increase. The standard CDF-HSD is:

[0085]

[0086] In order to improve the function's ability to enhance brightness and contrast in dark areas, this patent improves it. The improved cumulative distribution function of the hyperbolic secant distribution is defined as follows:

[0087]

[0088] Among them, (i, j) is the position index of the pixel point, V new-CDF-HSD (i, j) is the enhanced brightness component, and V(i, j) is the V component after conversion to HSV color space; is the weight matrix, which controls the brightness enhancement degree. Its calculation formula is:

[0089]

[0090] Where V(i,j) is the V component after conversion to the HSV color space.

[0091] To avoid over-enhancement in bright areas, data smoothing is performed on them. The tan-sigmoid function (hyperbolic tangent S-shaped function) has the characteristic of smoothly increasing the image within a range greater than zero. After reaching a certain value, the increase slowly becomes constant, which can meet the smoothing processing requirements. The standard function is defined as follows:

[0092]

[0093] The range of f(x) is [-1,1]. Since the pixel value is non-zero and in order to avoid negative pixel value processing results, the function is improved. The improved hyperbolic tangent sigmoid function is defined as follows:

[0094]

[0095] Among them, V new-tan-S (i, j) is the result of mapping the V component using the improved hyperbolic tangent sigmoid function; (i, j) is the position index of the pixel point; V(i, j) is the V component after conversion to HSV space; λ is a scalar that controls the enhancement process, and the formula is:

[0096]

[0097] Where s(X) is the grayscale value variance of the original input image; λ is rounded to two decimal places and is given by Figure 2a It can be seen that the smaller the value, the greater the brightness, but it should not be too small. Excessive reduction of λ will cause the pixels to exceed the standard range and become invalid pixels. The optimal range of λ is 2 to 7.

[0098] The comparison of function images before and after improvement is as follows: Figure 2b As shown in the figure (λ is 2), in the high-intensity pixel area (such as the exposed area), the f(x) function value drops excessively, while the improved V new-tan-S (i,j) is processed slowly to avoid over-enhancement.

[0099] Finally, the enhanced V new The (i,j) components are as follows:

[0100]

[0101] S3. The enhanced V newAfter the (i, j) component is fused with the H and S components of the original input image and converted back to the RGB color space, the overall brightness of the resulting image is dim, so illumination compensation is required. This patent uses a nonlinear transformation function (Illumination compensation, LC) for illumination compensation (other illumination compensation methods include gamma correction, which requires the design of gamma values ​​and has poor adaptability. The illumination compensation method proposed in this patent is adaptive compensation illumination). The LC function is defined as follows:

[0102]

[0103] Where, I in is the image converted back to RGB color space after brightness enhancement; || represents the absolute value, 1.3×I in Adjust the image chromaticity. This mapping function can enhance the overall brightness of the image without affecting the image contrast.

[0104] Images processed with illumination compensation may have slight color casts and require color balancing. The grayscale world method assumes that for an image with color changes, the average values ​​of its three RGB channels tend to a constant value, K. This assumption is applied to the image to be processed to eliminate the influence of the lighting environment on the image color. This patent uses the grayscale world method to adjust the color cast, weakening the influence of ambient light in the image. The restored image is more consistent with the real scene. The specific operation is as follows:

[0105] Calculate the mean of the RGB three channels The constant K can be expressed as:

[0106]

[0107] Gain coefficient of three channels:

[0108]

[0109] The adjusted RGB values ​​are R′, G′, B′:

[0110] R′=β R *R, G′=β G *G, B′=β B *B.

[0111] Common color balancing methods include perfect reflection and color temperature estimation. The perfect reflection method performs poorly when the area with the highest brightness is not white, and RGB values ​​tend to overflow (greater than 255). The color temperature estimation method is algorithmically complex and computationally time-consuming, failing to meet the real-time requirements of nighttime road visual processing. The grayscale world method employed in this patent is simple, easy to implement, and fast.

[0112] S4. Use dual-scale mean filtering to enhance image details and obtain the final night-time enhanced image:

[0113] During the aforementioned enhancement process, nonlinear mapping between pixels can easily cause sudden gradient changes in pixels at lane edge locations in the road image, resulting in loss of lane edge information and texture details. Therefore, this patent uses a dual-scale mean filter to reduce noise in the image and restore more detailed information. The specific operations are as follows:

[0114] Decompose the image into layers of different scales by mean filtering:

[0115] B=X*Z

[0116] Where X is the enhanced image obtained after processing in step S3, Z is the mean filter. If the image size is small, select the filter radius 3x3. B is the image after convolution with the mean filter, which represents the image base layer;

[0117] Subtracting the base layer from the enhanced image yields the detail layer D, i.e.:

[0118] D=XB

[0119] The purpose of the dual-scale decomposition is to obtain the base layer B and the detail layer D, and to add the detail layer D to the image obtained after processing in step S3, and finally output the image I. output :

[0120] I output =X+D

[0121] I output This is the final night-time enhanced image.

[0122] After the night image is enhanced by the method described in this patent, the details of road traffic information in the low-light image are effectively improved, and the lane lines and landmark prompts in the road image are more obvious (such as Figure 3a 、 Figure 3b In addition, from the histogram of the image before and after enhancement ( Figure 3c 、 Figure 3d ) As can be seen from the image histogram after enhancement, the grayscale levels are expanded, the grayscale levels are shifted to the right as a whole, and the number of medium-intensity pixels is relatively large. This shows that the overall visual effect of the image has been improved.

[0123] The above are only preferred embodiments of the present invention. It should be understood that the description of the above embodiments is only used to help understand the method and core ideas of the present invention, and is not used to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, etc. made within the ideas and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A nighttime road image enhancement method based on threshold partitioning weighted brightness component, characterized in that: The steps include: S1. Using the maximum entropy method to obtain a threshold value for the original input image, the original input image is converted from the RGB color space to the HSV color space, and the V component of the HSV color space is divided into dark and light areas according to the threshold value obtained by the maximum entropy method; S2. For dark areas, the improved cumulative distribution function of the hyperbolic secant distribution is used to increase their brightness. The cumulative distribution function is defined as follows: Among them, (i, j) is the position index of the pixel point, V new-CDF-HSD (i, j) is the enhanced brightness component, and V(i, j) is the V component after conversion to HSV color space; is the weight matrix, and the formula is: For bright areas, an improved hyperbolic tangent sigmoid function is used to enhance their brightness. The hyperbolic tangent sigmoid function is defined as follows: Among them, V new-tan-S (i, j) is the result of mapping the V component using the improved hyperbolic tangent sigmoid function; λ is a scalar that controls the enhancement process, and the formula is: Where s(X) is the grayscale value variance of the original input image; S3. The enhanced V new The (i, j) component is fused with the H and S components of the original input image and converted back to the RGB color space. The nonlinear transformation function LC is used for illumination compensation. The LC function is defined as follows: Where, I in The image is converted back to RGB color space after brightness enhancement; S4. Use dual-scale mean filtering to enhance image details and obtain the final night-time enhanced image.

2. The method for nighttime road image enhancement based on threshold partitioning and weighted luminance components according to claim 1, characterized in that: After performing illumination compensation on the image in step S3, the color cast of the image is adjusted using the grayscale world method.

3. The method for nighttime road image enhancement based on threshold partitioning and weighted luminance components according to claim 2, characterized in that: The specific operation of adjusting the color deviation of an image using the grayscale world method is as follows: Calculate the mean of the RGB three channels The constant K can be expressed as: Gain coefficient of three channels: The adjusted RGB values ​​are R′, G′, B′: R′=β R *R, G′=β G *G, B′=β B *B.

4. The method for nighttime road image enhancement based on threshold partitioning and weighted luminance components according to claim 1, characterized in that: The specific operation of dividing the V component of the HSV color space into dark areas and bright areas according to the threshold value obtained by the maximum entropy method in step S1 is: Image entropy definition: Where H(I) is the image entropy, (u, v) is the position index of the pixel, g = I(u, v), I is the input image, and p(g) is the probability of pixel (u, v). Given a threshold q, the cumulative probability P of the dark and bright areas segmented by the q threshold is calculated. L (q), P D (q), the sum of the two is 1; the entropy corresponding to the two regions is expressed as follows: Where H L (q), H D (q) represents the entropy of the bright area and the dark area respectively; Under the threshold q, the total entropy of the image is: H(q)=H L (q)+H D (q) H=max(H(q)) Calculate the total entropy H(q) of the image under all thresholds, and take the threshold corresponding to the maximum entropy H=max(H(q)) as the optimal threshold T; In the HSV color space, the area where the brightness component V pixel value t≤T is divided into the dark area, and the area where the brightness component V pixel value t>T is divided into the dark area.

5. The method for nighttime road image enhancement based on threshold partitioning and weighted luminance components according to claim 1, characterized in that: The specific operations of step S4 are: Decompose the image into layers of different scales by mean filtering: B=X*Z Wherein, X is the enhanced image obtained after processing in step S3, Z is the mean filter, and B is the image after convolution with the mean filter, which represents the image base layer; Subtracting the base layer from the enhanced image yields the detail layer D, i.e.: D=XB Add the detail layer D to the image obtained after processing in step S3, and finally output image I output : I output =X+D I output This is the final night-time enhanced image.

Images