Traffic image enhancement method and device in low-light environment

Abstract

The present invention provides a method and device for enhancing traffic images in low-light environments. The method comprises the following steps: acquiring a traffic image in a low-light environment; performing enhancement processing on the traffic image; and adaptively fusing the traffic image data with the enhanced traffic image to obtain an output image. The enhancement method of the present invention can improve the image quality of low-light traffic environments. By adaptively fusing the initial image and the processed image, the image features are made clearer and more image details are preserved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for enhancing traffic images in a low-illumination environment and a device for enhancing traffic images in a low-illumination environment. Background Art

[0002] In low-light environments at night, road traffic images are of poor quality and low contrast. Using low-light images for road monitoring or detection yields limited results due to the lack of distinct features. To improve the visibility of low-light image information and enrich its details, image enhancement is required.

[0003] Current research on low-light imaging primarily focuses on hardware, but due to the high cost of hardware equipment used for nighttime low-light imaging, it is not applicable in daily life. Furthermore, software research is underway, such as the histogram equalization algorithm, which improves image quality by evenly distributing image pixels across each grayscale level. The Retinex algorithm, proposed by Edwin Land, also enhances low-light images by simulating the human visual system. While extensive research has been conducted on nighttime low-light imaging, achieving both real-time performance and high-quality results is difficult. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a traffic image enhancement method in a low-light environment, which can improve the image quality of the low-light traffic environment and perform adaptive image fusion of the initial image and the processed image to make the image features clearer and retain more image details.

[0005] The technical solution adopted in the present invention is as follows:

[0006] A method for enhancing traffic images in a low-light environment comprises the following steps: acquiring a traffic image in a low-light environment; performing enhancement processing on the traffic image; and adaptively fusing the traffic image data with the enhanced traffic image to obtain an output image.

[0007] According to one embodiment of the present invention, the traffic image is enhanced, including: performing R, G, and B channel enhancement on the traffic image through homomorphic filtering; performing reflection processing on the traffic image after the homomorphic filtering; performing defogging on the traffic image after the reflection processing; converting the color space of the defogging traffic image from RGB to HSL; processing the L channel image using a limited contrast adaptive histogram equalization algorithm and correcting it using a gamma transform; enhancing the S channel image using an adaptive nonlinear enhancement algorithm; merging the processed L channel image, S channel image, and H channel image, converting them into RGB color space, and performing reflection processing on the converted image.

[0008] According to one embodiment of the present invention, the following formula is used for homomorphic filtering:

[0009]

[0010] Among them, I(x,y) represents the image after homomorphic filtering, FFT and FFT -1 represent Fourier transform and its inverse transform respectively, M represents a frequency domain filter, and f(x, y) represents the traffic image.

[0011] According to one embodiment of the present invention, the following formula is used for reflection processing:

[0012] I=255-I in

[0013] Among them, I in represents the input traffic image, and I represents the reflected output traffic image.

[0014] According to one embodiment of the present invention, the following formula is used for defogging:

[0015]

[0016] Where t(x) represents the atmospheric transmittance, I c (x) represents the image before dehazing, A c represents the global atmospheric light component, w represents the correction factor, whose value range is [0,1], Ω(x) represents a window centered on pixel x, and J(x) represents the defogged image.

[0017] According to one embodiment of the present invention, the following formula is used for color space conversion:

[0018]

[0019] Among them, B, G, R represent the coordinates of blue, green, and red in the BGR color space respectively, max represents the maximum coordinate of B, G, and R, min represents the minimum coordinate of B, G, and R, and H, S, and L represent the hue, saturation, and brightness in the HSL color space respectively.

[0020] According to one embodiment of the present invention, the L channel is gamma transformed using the following formula:

[0021] S′=cr γ

[0022] Among them, S′ represents the output value after gamma transformation correction, c represents the scaling coefficient, r represents the input value of the original grayscale image, and γ represents the gamma factor coefficient, which is used to control the degree of correction.

[0023] According to one embodiment of the present invention, adaptive nonlinear enhancement processing is performed on the S channel using the following formula:

[0024]

[0025] Among them, S out It represents the saturation component after adaptive nonlinear stretching, S in Represents the initial saturation component before stretching, mean(R,G,B) represents the average value of the R, G, and B components corresponding to each pixel in the original image, max(R,G,B) represents the maximum value of the R, G, and B components corresponding to each pixel in the original image, and min(R,G,B) represents the minimum value of the R, G, and B components corresponding to each pixel in the original image.

[0026] According to one embodiment of the present invention, the following formula is used for adaptive fusion:

[0027] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.144*B(i,j)

[0028]

[0029] I(i,j)=αI in (i,j)+(1-α)I out (i,j)

[0030] Among them, R(i,j), G(i,j), B(i,j) represent the red, green, and blue channel values ​​of each pixel, respectively. Gray(i,j) represents the weighted average gray value of the pixel. α represents the ratio of each pixel in the traffic image to the enhanced traffic image. Gray(i,j) in Represents the value of a pixel in the traffic image, Gray(i,j)out represents the value of each pixel in the enhanced traffic image, I(i,j) represents each pixel after fusion, and I in (i, j) represents the value of each pixel point in the traffic image, I out (i, j) represents the value of each pixel point after enhancement processing.

[0031] The present invention also proposes a traffic image enhancement device in a low-light environment, comprising: an image acquisition module for acquiring traffic images in a low-light environment; an enhancement processing module for enhancing the traffic images; and a fusion module for adaptively fusing the traffic image data with the enhanced traffic image to obtain an output image.

[0032] Beneficial effects of the present invention:

[0033] The present invention first captures traffic images in low-light environments, enhances them, and adaptively fuses the traffic image data with the enhanced traffic image to produce an output image. This improves image quality in low-light traffic environments. Adaptive fusion of the initial and processed images enhances image features and preserves more image detail. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a method for enhancing traffic images in a low-light environment according to an embodiment of the present invention;

[0035] Figure 2a and Figure 2b This is a comparison diagram of traffic images before and after enhancement in a low-light environment according to an embodiment of the present invention;

[0036] Figure 3 4 is a block diagram of a traffic image enhancement device in a low-light environment according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] Figure 1 Flowchart of a traffic image enhancement method in a low-light environment according to an embodiment of the present invention.

[0039] like Figure 1 As shown, the traffic image enhancement method in a low-light environment according to an embodiment of the present invention may include the following steps:

[0040] S1, collects traffic images in low-light environments.

[0041] Generally, traffic images of the vehicle's current environment can be obtained through cameras or lidar installed on the vehicle.

[0042] S2, enhance the traffic image.

[0043] According to one embodiment of the present invention, traffic image enhancement processing includes: performing R, G, and B channel enhancement on the traffic image through homomorphic filtering; performing reflection processing on the traffic image after homomorphic filtering; performing defogging processing on the traffic image after reflection processing; converting the color space of the defogging traffic image from RGB to HSL; processing the L channel image using a limited contrast adaptive histogram equalization algorithm and correcting it using a gamma transform; enhancing the S channel image using an adaptive nonlinear enhancement algorithm; merging the processed L channel image, S channel image, and H channel image, and converting them into RGB color space, and performing reflection processing on the converted image.

[0044] In one embodiment of the present invention, when performing R, G, and B channel enhancement on a traffic image through homomorphic filtering, the following formula may be used for enhancement processing:

[0045]

[0046] Among them, I(x,y) represents the image after homomorphic filtering, FFT and FFT -1 They represent Fourier transform and its inverse transform respectively, M represents the frequency domain filter, and f(x,y) represents the traffic image.

[0047] After the traffic image is processed by homomorphic filtering, an inversion operation is performed on the processed traffic image, that is, the values ​​of the R, G, and B channels are inverted. For example, the inversion operation can be performed using the following formula:

[0048] I=255-I in

[0049] Among them, I in represents the input traffic image, and I represents the reflected output traffic image.

[0050] After the traffic image is reflected, the scattering effect of a large number of tiny water droplets and aerosols suspended in the atmosphere due to fog, haze and other weather conditions will cause the captured image to be severely degraded. Therefore, defogging is required. First, the atmospheric transmittance in the current environment needs to be obtained, and then defogging is performed based on the atmospheric transmittance. For example, the atmospheric transmittance and defogging can be obtained using the following formula:

[0051]

[0052] Where t(x) represents the atmospheric transmittance, I c (x) represents the image before dehazing, A c represents the global atmospheric light component, w represents the correction factor, which ranges from [0, 1], Ω(x) represents a window centered on pixel x, and J(x) represents the dehazed image. It should be noted that the image dehazing method can be based on the specific content described in the prior art and will not be detailed here.

[0053] After dehazing the traffic image, color conversion is performed, that is, converting the RGB color space to the HSL color space. For example, the hue H, saturation L, and brightness S in the HSL color space can be obtained by the following formula:

[0054]

[0055] Among them, B, G, R represent the coordinates of blue, green, and red in the BGR color space respectively, max represents the maximum coordinate of B, G, and R, min represents the minimum coordinate of B, G, and R, and H, S, and L represent the hue, saturation, and brightness in the HSL color space respectively.

[0056] Next, the L channel image after color conversion is processed using a limited contrast adaptive histogram equalization algorithm so that the image contrast is more natural when the histogram equalization image is trimmed. The processed L channel image is then gamma transformed to correct the image. For example, the correction can be performed using the following formula:

[0057] S′=cr γ

[0058] Among them, S′ represents the output value after gamma transformation correction, c represents the scaling coefficient, r represents the input value of the original grayscale image, and γ represents the gamma factor coefficient, which is used to control the degree of correction.

[0059] Next, in order to improve the positioning accuracy, the S channel is also subjected to adaptive nonlinear enhancement processing. For example, the enhancement processing can be performed using the following formula:

[0060]

[0061] Among them, S out It represents the saturation component after adaptive nonlinear stretching, S inRepresents the initial saturation component before stretching, mean(R,G,B) represents the average value of the R, G, and B components corresponding to each pixel in the original image, max(R,G,B) represents the maximum value of the R, G, and B components corresponding to each pixel in the original image, and min(R,G,B) represents the minimum value of the R, G, and B components corresponding to each pixel in the original image.

[0062] Finally, the three channels are merged to obtain the H, S, and L values ​​in the HSL color space, and the color space conversion is performed again, that is, the HSL color space is converted to the BGR color space, and then the reflection processing is performed to obtain the enhanced image.

[0063] S3, adaptively fusing the traffic image data with the enhanced traffic image to obtain an output image.

[0064] According to one embodiment of the present invention, the following formula is used for adaptive fusion:

[0065] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.144*B(i,j)

[0066]

[0067] I(i,j)=αI in (i,j)+(1-α)I out (i,j)

[0068] Among them, R(i,j), G(i,j), B(i,j) represent the red, green, and blue channel values ​​of each pixel, respectively. Gray(i,j) represents the weighted average gray value of the pixel. α represents the ratio of each pixel in the traffic image to the enhanced traffic image. Gray(i,j) in Represents the value of a pixel in the traffic image, Gray(i,j) out represents the value of each pixel in the enhanced traffic image, I(i,j) represents each pixel after fusion, and I in (i, j) represents the value of each pixel point in the traffic image, I out (i, j) represents the value of each pixel point after enhancement processing.

[0069] That is to say, the original image and the enhanced image are adaptively fused to obtain the final output image, such as Figure 2a and 2b As shown in the figure, it can be seen that the gap between the adaptive fused image and the original image can improve the image quality in low-light traffic environments, making the image features clearer and retaining more image details.

[0070] In summary, the present invention first captures traffic images in low-light environments, enhances them, and adaptively fuses the traffic image data with the enhanced traffic image to produce an output image. This improves image quality in low-light traffic environments, and adaptively fuses the initial and processed images to achieve clearer image features and preserve more image details.

[0071] Corresponding to the above method, the present invention also proposes a traffic image enhancement device in a low-light environment.

[0072] Figure 3 4 is a block diagram of a traffic image enhancement device in a low-light environment according to an embodiment of the present invention.

[0073] like Figure 3 As shown, the traffic image enhancement device in a low-light environment of the present invention may include: an image acquisition module 10 , an enhancement processing module 20 and a fusion module 30 .

[0074] The image acquisition module 10 is used to acquire traffic images in low-light environments. The enhancement processing module 20 is used to enhance the traffic images. The fusion module 30 is used to adaptively fuse the traffic image data with the enhanced traffic image to obtain an output image.

[0075] According to one embodiment of the present invention, the enhancement processing module 20 performs enhancement processing on the traffic image, specifically for performing R, G, and B channel enhancement on the traffic image through homomorphic filtering; performing reflection processing on the traffic image after homomorphic filtering; performing defogging processing on the traffic image after reflection processing; converting the color space of the defogging traffic image from RGB to HSL; processing the L channel image using a limited contrast adaptive histogram equalization algorithm and correcting it using a gamma transform; enhancing the S channel image using an adaptive nonlinear enhancement algorithm; merging the processed L channel image, S channel image, and H channel image, and converting them into RGB color space, and performing reflection processing on the converted image.

[0076] According to one embodiment of the present invention, the following formula is used for homomorphic filtering:

[0077]

[0078] Among them, I(x,y) represents the image after homomorphic filtering, FFT and FFT -1 They represent Fourier transform and its inverse transform respectively, M represents the frequency domain filter, and f(x,y) represents the traffic image.

[0079] According to one embodiment of the present invention, the following formula is used for reflection processing:

[0080] I=255-I in

[0081] Among them, I in represents the input traffic image, and I represents the reflected output traffic image.

[0082] According to one embodiment of the present invention, the following formula is used for defogging:

[0083]

[0084] Where t(x) represents the atmospheric transmittance, I c (x) represents the image before dehazing, A c represents the global atmospheric light component, w represents the correction factor, whose value range is [0,1], Ω(x) represents a window centered on pixel x, and J(x) represents the dehazed image.

[0085] According to one embodiment of the present invention, the following formula is used for color space conversion:

[0086]

[0087] Among them, B, G, R represent the coordinates of blue, green, and red in the BGR color space respectively, max represents the maximum coordinate of B, G, and R, min represents the minimum coordinate of B, G, and R, and H, L, and S represent the hue, saturation, and brightness in the HSL color space respectively.

[0088] According to one embodiment of the present invention, the L channel is gamma transformed using the following formula:

[0089] S′=cr γ

[0090] Among them, S′ represents the output value after gamma transformation correction, c represents the scaling coefficient, r represents the input value of the original grayscale image, and γ represents the gamma factor coefficient, which is used to control the degree of correction.

[0091] According to one embodiment of the present invention, adaptive nonlinear enhancement processing is performed on the S channel using the following formula:

[0092]

[0093] Among them, S out It represents the saturation component after adaptive nonlinear stretching, S inRepresents the initial saturation component before stretching, mean(R,G,B) represents the average value of the R, G, and B components corresponding to each pixel in the original image, max(R,G,B) represents the maximum value of the R, G, and B components corresponding to each pixel in the original image, and min(R,G,B) represents the minimum value of the R, G, and B components corresponding to each pixel in the original image.

[0094] According to one embodiment of the present invention, the following formula is used for adaptive fusion:

[0095] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.144*B(i,j)

[0096]

[0097] I(i,j)=αI in (i,j)+(1-α)I out (i,j)

[0098] Among them, R(i,j), G(i,j), B(i,j) represent the red, green, and blue channel values ​​of each pixel, respectively. Gray(i,j) represents the weighted average gray value of the pixel. α represents the ratio of each pixel in the traffic image to the enhanced traffic image. Gray(i,j) in Represents the value of a pixel in the traffic image, Gray(i,j) out represents the value of each pixel in the enhanced traffic image, I(i,j) represents each pixel after fusion, and I in (i, j) represents the value of each pixel point in the traffic image, I out (i, j) represents the value of each pixel point after enhancement processing.

[0099] It should be noted that for details not disclosed in the traffic image enhancement device under low illumination environment in an embodiment of the present invention, please refer to the details disclosed in the traffic image enhancement method under low illumination environment in an embodiment of the present invention, and the details will not be repeated here.

[0100] The present invention's low-light traffic image enhancement device first uses an image acquisition module to capture traffic images in low-light environments. The enhancement processing module then performs enhancement processing on the traffic images. A fusion module then adaptively fuses the traffic image data with the enhanced traffic image to produce an output image. This improves image quality in low-light traffic environments. Adaptive fusion of the initial and processed images enhances image features and preserves more image detail.

[0101] Corresponding to the above device, the present invention also proposes a vehicle.

[0102] The vehicle of the present invention includes the above-mentioned traffic image enhancement device in a low-light environment.

[0103] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. "Multiple" means two or more, unless otherwise specifically defined.

[0104] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0105] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0106] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0107] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0108] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. Traffic image enhancement method in low illumination environment, characterized by: The following steps are involved: Collect traffic images in low-light environments; performing enhancement processing on the traffic image; Adaptively fusing the traffic image data with the enhanced traffic image to obtain an output image; Performing enhancement processing on the traffic image, including: Performing R, G, and B channel enhancement on the traffic image through homomorphic filtering; Performing reflection processing on the traffic image after the homomorphic filtering processing; performing defogging processing on the traffic image after the reflection processing; Converting the color space of the dehazed traffic image from RGB to HSL; The L channel image is processed using the contrast-limited adaptive histogram equalization algorithm and corrected using gamma transformation; Use adaptive nonlinear enhancement algorithm to enhance the S channel image; The processed L channel image, S channel image and H channel image are merged, converted into RGB color space, and reflection processing is performed on the converted image.

2. The method for enhancing traffic images in a low-light environment according to claim 1, characterized in that: The following formula is used for homomorphic filtering: Among them, I(x,y) represents the image after homomorphic filtering, FFT and FFT -1 represent Fourier transform and its inverse transform respectively, M represents a frequency domain filter, and f(x, y) represents the traffic image.

3. The method for enhancing traffic images in a low-light environment according to claim 1, characterized in that: The following formula is used for reflection processing: I=255-I in Among them, I in represents the input traffic image, and I represents the reflected output traffic image.

4. The method for enhancing traffic images in a low-light environment according to claim 1, characterized in that: The following formula is used for defogging: Where t(x) represents the atmospheric transmittance, I c (x) represents the image before dehazing, A c represents the global atmospheric light component, w represents the correction factor, whose value range is [0,1], Ω(x) represents a window centered on pixel x, and J(x) represents the dehazed image.

5. The method for enhancing traffic images in a low-light environment according to claim 1, characterized in that: The following formula is used for color space conversion: Among them, B, G, R represent the coordinates of blue, green, and red in the BGR color space respectively, max represents the maximum coordinate of B, G, and R, min represents the minimum coordinate of B, G, and R, and H, S, and L represent the hue, saturation, and brightness in the HSL color space respectively.

6. The method for enhancing traffic images in a low-light environment according to claim 5, characterized in that: Perform gamma transformation on the L channel using the following formula: S′=cr γ Among them, S′ represents the output value after gamma transformation correction, c represents the scaling coefficient, r represents the input value of the original grayscale image, and γ represents the gamma factor coefficient, which is used to control the degree of correction.

7. The method for enhancing traffic images in a low-light environment according to claim 5, characterized in that: The S channel is adaptively enhanced nonlinearly using the following formula: Among them, S out It represents the saturation component after adaptive nonlinear stretching, S in Represents the initial saturation component before stretching, mean(R,G,B) represents the average value of the R, G, and B components corresponding to each pixel in the original image, max(R,G,B) represents the maximum value of the R, G, and B components corresponding to each pixel in the original image, and min(R,G,B) represents the minimum value of the R, G, and B components corresponding to each pixel in the original image.

8. The method for enhancing traffic images in a low-light environment according to claim 1, characterized in that: The following formula is used for adaptive fusion: Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.144*B(i,j) I(i,j)=αI in (i,j)+(1-α)I out (i,j) Among them, R(i,j), G(i,j), B(i,j) represent the red, green, and blue channel values ​​of each pixel, respectively. Gray(i,j) represents the weighted average gray value of the pixel. α represents the ratio of each pixel in the traffic image to the enhanced traffic image. Gray(i,j) in Represents the value of a pixel in the traffic image, Gray(i,j) out represents the value of each pixel in the enhanced traffic image, I(i,j) represents each pixel after fusion, and I in (i, j) represents the value of each pixel point in the traffic image, I out (i, j) represents the value of each pixel point after enhancement processing.

9. A traffic image enhancement device in a low-light environment, characterized in that: include: Image acquisition module, used to collect traffic images in low-light environments; An enhancement processing module, configured to perform enhancement processing on the traffic image; a fusion module, configured to adaptively fuse the traffic image data with the enhanced traffic image to obtain an output image; The enhancement process includes: Performing R, G, and B channel enhancement on the traffic image through homomorphic filtering; Performing reflection processing on the traffic image after the homomorphic filtering processing; performing defogging processing on the traffic image after the reflection processing; Converting the color space of the dehazed traffic image from RGB to HSL; The L channel image is processed using the contrast-limited adaptive histogram equalization algorithm and corrected using gamma transformation; Use adaptive nonlinear enhancement algorithm to enhance the S channel image; The processed L channel image, S channel image and H channel image are merged, converted into RGB color space, and reflection processing is performed on the converted image.

Images