A single night image defogging method based on multi-scale decomposition

Abstract

The application discloses a single night image defogging method based on multi-scale decomposition and relates to the technical field of image defogging. The steps comprise the following steps: using a relative smooth constraint to decompose an input single night foggy image into a glow image and a residual image, and removing the glow effect of multiple light sources in the night foggy image; using a gamma correction method to compensate the brightness of the residual image; then combining a total variation constraint to construct an energy optimization equation to decompose the brightness-compensated image into a structure image and two scale texture images; further using a local dark channel prior algorithm to perform a defogging operation on the structure residual image to obtain a structure image without fog; using a three-dimensional block matching denoising algorithm to remove noise components in the texture image T1; subsequently performing gradient domain enhancement on the two scale texture images to obtain texture layer images with clear details; and finally performing linear fusion on the defogged structure layer image and the two scale texture images after detail enhancement to obtain a night image without fog. The application has the advantages of removing fog interference in the image, compensating the brightness of the image, highlighting texture details, and achieving a noise suppression effect.

Description

Technical Field

[0001] This invention relates to the field of digital image processing, and in particular to a method for dehazing a single nighttime image based on multi-scale decomposition. Background Technology

[0002] In foggy nighttime conditions, factors such as low illumination, artificial light sources, and smog can cause uneven lighting, excessive brightness, high noise, and blurred texture details in outdoor images and videos. This severely restricts the normal operation of outdoor computer vision systems in nighttime scenes and significantly impacts people's daily lives and travel. For example, nighttime environments are typically characterized by weak human visual perception, high safety risks, and frequent accidents for vehicles traveling on roads. Furthermore, fog and smog further reduce visibility, making computer-aided vision systems crucial for improving drivers' perception of their surroundings and preventing accidents. Therefore, improving the quality of images acquired in foggy nighttime conditions has significant practical implications for the application of computer vision.

[0003] Images acquired under foggy nighttime conditions exhibit significant degradation compared to daytime scenes. During the day, atmospheric light primarily originates from sunlight and is uniformly distributed. However, in foggy nighttime scenes, the ambient light used for imaging mainly comes from artificial light sources such as streetlights and vehicle headlights, characterized by low illuminance, uneven distribution, and diverse colors. This results in images that are too dark, lack detail, and suffer from color distortion. Furthermore, darker areas in the image often conceal substantial noise, which is typically amplified during image restoration. Currently, mainstream image dehazing methods primarily focus on daytime scenes and are inadequate for addressing the dehazing problem in nighttime scenes. Therefore, this invention holds significant theoretical value for the field of digital image processing.

[0004] While existing daytime image dehazing methods cannot effectively restore nighttime foggy images, they have provided inspiration and solutions for nighttime image dehazing research. In recent years, researchers have proposed various image dehazing methods suitable for nighttime foggy scenes. In 2012, Soo-Chang Pei and Tzu-Yen Lee first proposed a nighttime image dehazing method based on color transfer technology. This method uses color transfer technology to convert nighttime foggy images into daytime foggy images, and then uses dark channel prior theory to restore the image. Zhang Jing et al. considered the influence of artificial light sources, modifying atmospheric light in the atmospheric scattering model to ambient light to better utilize nighttime foggy images. Based on the new model, they then performed illumination compensation and color correction on the input image to remove fog. This algorithm can significantly improve the brightness and contrast of degraded images, but it is prone to glow phenomena in areas with artificial light sources. Li Yu et al. analyzed the influence of artificial light sources on multiscattering during imaging, adding a new term to the atmospheric scattering model to describe the glow effect in nighttime foggy images. They then used a relatively smooth constraint image decomposition algorithm based on the new image degradation model to separate the glow layer, and finally utilized dark channel prior theory to achieve dehazing. This algorithm effectively removes the glow phenomenon generated during the restoration of nighttime foggy images, but its noise suppression is poor. Zhang Jing et al. proposed a maximum reflectance prior theory through statistical analysis of a large number of daytime fog-free natural image patches, and then used this prior theory to dehaze nighttime foggy images. The algorithm's results show a slight dimming effect and also fails to effectively address the problem of noise amplification in dark areas of the image. Tang Chunming et al. used an image decomposition algorithm to divide the input image into a structural layer and a texture layer, then dehazed the structural layer and optimized the texture layer for denoising separately, finally superimposing the two to obtain the final image. Although this algorithm can suppress noise to some extent, it does not consider the glow effect in nighttime foggy images, and the parameters set for optimizing the texture layer for denoising are not applicable to different input images. Yang Minmin et al. introduced the idea of ​​superpixel segmentation into nighttime image dehazing algorithms. When estimating ambient light and transmittance, they used irregular but content-aware image patches instead of classic rectangular image patches, making the resulting atmospheric light map and transmittance map more closely match the structure of the input image. However, this algorithm still did not consider the problem of noise amplification. Fang Shuai et al. proposed a nighttime image dehazing method based on illumination estimation, which removes the influence of uneven illumination by estimating the illumination map. Yu Teng et al. first divided the nighttime foggy image into light source regions and non-light source regions, then used different prior knowledge to process the light source and non-light source regions separately, and finally performed pixel-level fusion to obtain a clear image. However, the resulting image obtained by this method exhibits a darker appearance.

[0005] Based on this, the present invention provides a method for dehazing a single nighttime foggy image based on multi-scale decomposition. The proposed method mainly includes five processing steps: glow decomposition, brightness compensation, multi-scale image decomposition, structural layer dehazing, texture layer denoising, and gradient domain enhancement. First, the glow layer of the nighttime foggy image is removed using a relatively smooth constraint image decomposition method, and then gamma correction is used to compensate the brightness of the remaining residual image. Then, the brightness-compensated image is decomposed into a structural layer image and two texture layer images using total variation constraints; then, dehazing and gradient domain enhancement are performed on the structural layer image and the texture layer images respectively. Finally, the dehazed structural layer image and the enhanced texture layer images are linearly fused to obtain the final fog-free and clear image. Summary of the Invention

[0006] This invention proposes a single-image dehazing method based on multi-scale decomposition, which aims to simultaneously overcome the problems of glow effect, excessive brightness, haze interference, blurred details, and noise amplification in nighttime foggy images.

[0007] The technical solution adopted in this invention is as follows.

[0008] A method for dehazing a single nighttime image based on multi-scale decomposition, characterized by the following steps:

[0009] (1) Decompose the acquired single nighttime foggy image into a glow layer image and a residual image;

[0010] (2) Brightness compensation is performed on the residual image after glow removal using gamma correction;

[0011] (3) Decompose the brightness-compensated image into a structural layer image Base and two texture layer images T1 and T2;

[0012] (4) Dehaze the structural layer residual image Base;

[0013] (5) Denoise and enhance the texture layer image T1 sequentially;

[0014] (6) Perform gradient domain enhancement processing on texture layer T2;

[0015] (7) The dehazed structure layer image and the two enhanced texture layer images are fused to obtain the result image.

[0016] In step (1), the gradient histogram features of the glow layer image in terms of color, shape and direction are obtained by statistical analysis. Then, an energy equation is constructed and minimized to separate the glow layer in the input single night fog image.

[0017] In step (2), a gamma correction function is used to perform brightness compensation, so that the brightness of the night fog image is compensated.

[0018] In step (3), the total variation constraint is used to decompose the brightness-compensated night fog image into multiple structural layer images step by step. Then, the difference between adjacent structural layer images is used as the texture image. Typically, three scales of structural layer images are selected to generate, and then two scales of texture layer images are obtained.

[0019] In step (4), the transmittance of the structural layer residual image is estimated by using the maximum reflection prior and the dark channel prior, and then the structural layer image is restored by using the non-uniform atmospheric light scattering model.

[0020] In step (5), the texture layer image T1 is first denoised using a three-dimensional block matching algorithm, and then the denoised image is converted to the gradient domain to enhance the details.

[0021] In step (6), since the noise intensity in the texture layer image T2 is not high, only gradient domain enhancement processing is performed to highlight the detailed areas.

[0022] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following advantages.

[0023] (1) This invention utilizes a multi-scale decomposition strategy with total variation constraints to decompose the input single night fog image into a structural layer residual image and multiple scale texture residual images, which facilitates the corresponding processing of residual images with different features. This can effectively solve the degradation problem caused by different factors, improve the image quality, and greatly enhance the clarity of night fog images.

[0024] (2) The present invention can simultaneously solve the problems of glow effect, low brightness, haze interference, blurred details and noise amplification in single night fog images, and has the advantages of low noise and rich texture details. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the process framework of the present invention.

[0026] Figure 2 (a) is a magnified image of a foggy night scene and a local area thereof.

[0027] Figure 2 (b) is a restored image and magnified local area of ​​an embodiment of the present invention. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0029] like Figure 1 As shown in the figure, the present invention is a method for dehazing a single nighttime image based on multi-scale decomposition, characterized by the following main steps:

[0030] (1) Decompose the acquired single nighttime foggy image into a glow layer image and a residual image;

[0031] (2) Brightness compensation is performed on the residual image after glow removal using gamma correction;

[0032] (3) Decompose the brightness-compensated image into one structural layer image and two texture layer images;

[0033] (4) Dehaze the structural layer residual image;

[0034] (5) Denoise and enhance the texture layer image T1 sequentially;

[0035] (6) Perform image enhancement processing on texture layer T2;

[0036] (7) The dehazed structure layer image and the two enhanced texture layer images are fused to obtain the result image.

[0037] In step (1), the gradient histogram features of the glow layer image in terms of color, shape, and direction are obtained by using statistical analysis. Then, an energy equation is constructed and minimized to separate the glow layer in the input single nighttime foggy image. The specific energy optimization equation is as follows:

[0038]

[0039] st0≤H(x)≤I(x),

[0040]

[0041] Where H(x) represents the residual image after glow decomposition, f 1,2 f1 is a first-order differential filter in both the vertical and horizontal directions, f2 is a second-order Laplace filter, and the symbol "*" represents convolution. The first term in the above equation uses the robustness function ρ(s) = min(s). 2The first term, τ), ensures that larger gradients in the input image I(x) are retained in H(x). The second term uses the L2 norm to constrain the glow image, making the resulting glow image as smooth as possible, where λ controls the smoothness of the glow layer image. Furthermore, an inequality constraint is added to the above equation to ensure that the obtained solution is within a suitable range. However, since this constraint is used individually in each color channel of the RGB, it can easily lead to color shifts in the separated H(x). Therefore, a color constancy constraint is added to address the color shift problem.

[0042] In step (2), a gamma correction function is used to perform brightness compensation, specifically as follows:

[0043] H(x)=H(x) γ

[0044] Where γ is the brightness compensation parameter.

[0045] In step (3), the brightness-compensated image H(x) is decomposed into a structural layer residual image and two texture layer residual images using total variation constraints. Specifically, the following energy optimization equation is constructed using total variation constraints, and the structural layer residual image S is obtained by minimizing the energy optimization equation.

[0046]

[0047] Where S is the structural layer residual image, The gradient operator is α, and the regularization parameter is used to balance the data term and the regularization term. Iteratively solving the above energy optimization equation using the alternating direction minimization method yields the structural layer residual image S, and subsequently the corresponding texture layer image T = HS. To further obtain texture images at different scales, we use a multi-scale representation to describe image H. Specifically, we use the first obtained structural layer residual image S as a coarse representation of image H, and then sequentially apply total variation constraints to the obtained structural layer images to obtain the structural layer residual image SH. i The corresponding texture layer residual image T i T is the difference between the residual images of adjacent structural layers. i =S i -S i-1 (i = 1, 2, ..., k). Usually, we set k = 2, which means decomposing the image H into a structural layer residual image S and two texture layer residual images T1 and T2.

[0048] In step (4), the residual image S of the structural layer is dehazed. Specifically, the transmittance of the residual image S is estimated by utilizing the maximum reflection prior and the dark channel prior.

[0049]

[0050] Where t is transmittance and L is non-uniform atmospheric light, we use the maximum pixel intensity within a local block as the estimate of atmospheric light. Then, we inversely solve the atmospheric scattering model to restore the structural layer residual image:

[0051]

[0052] It should be noted that since haze has a significant impact on the low-frequency components of the image, this invention only performs dehazing on the structural layer residual image.

[0053] In step (5), the residual texture layer image T1 contains significant noise and details. Therefore, this invention first uses a three-dimensional block matching denoising algorithm to denoise the residual texture layer image T1, and then converts it into the gradient domain and divides it by the transmittance to achieve texture layer image enhancement. Specifically:

[0054]

[0055] Then, gradient domain reconstruction operation is performed to obtain Restore represents the gradient domain reconstruction operation. The processed texture layer residual image T1, after gradient domain enhancement and reconstruction, not only suppresses noise but also becomes richer in detail.

[0056] In step (6), since the texture layer T2 contains almost no noise, only gradient domain enhancement is performed to highlight the details of interest.

[0057] In step (7), the final clear image is obtained by linearly fusing the dehazed structural layer residual image and the gradient domain enhanced texture layer residual image.

[0058]

[0059] Among them, I dehazed This is a nighttime foggy image after the final defogging process.

[0060] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A method for dehazing a single nighttime image based on multi-scale decomposition, characterized in that, Includes the following steps: (1) Decompose the nighttime foggy image into a glow layer image and a residual image; (2) Brightness compensation is performed on the residual image after glow removal using gamma correction; (3) Decompose the brightness-compensated image into one structural layer image and two texture layer images; (4) Dehaze the structural layer residual image; (5) Denoise and enhance the texture layer image T1 sequentially; (6) Perform gradient domain enhancement processing on texture layer T2; (7) Fuse the dehazed structure layer image and the two enhanced texture layer images to obtain the result image; In step (3), the brightness-compensated image is first smoothed step by step using total variation constraints to generate a series of structural layer images S. i Then, the difference between the structural layer images is used as the texture layer image, specifically: T i =S i -S i-1 (i=1,2,…,k) Select texture layer images at two scales; In step (3), the brightness-compensated image H(x) is decomposed into a structural layer residual image and two texture layer residual images using total variation constraints. Specifically, the following energy optimization equation is constructed using total variation constraints, and the structural layer residual image S is obtained by minimizing the energy optimization equation: ; Where S is the structural layer residual image, For gradient operators; This is a regularization parameter used to balance data items and regularization terms.

2. The method for dehazing a single nighttime image based on multi-scale decomposition according to claim 1, characterized in that, In step (4), the prior knowledge of the dark channel is applied to local blocks of the structural layer residual image to perform dehazing.

3. The method for dehazing a single nighttime image based on multi-scale decomposition according to claim 1, characterized in that, In steps (5) and (6), the texture layer image T1 is first denoised using a three-dimensional block matching algorithm, and then the denoised image is enhanced using a gradient domain enhancement method; only gradient domain enhancement is performed on the texture layer image T2.

Images