Single image dehazing method based on joint optimization of edge and tone fidelity constraints

Abstract

The invention discloses a single image defogging method for joint constraint optimization of edge and hue fidelity. The invention firstly divides a foggy image into a series of image blocks by using SLIC superpixel segmentation; secondly, by using normal distribution standardization and Sigmoid nonlinear mapping, a normalized dark channel map and a normalized detail map are constructed to generate a feature map of weighted fusion of brightness and structural details, so as to adaptively select image blocks reflecting atmospheric light and estimate ideal atmospheric light values; then, with the scattering coefficient as an independent variable, an objective function for joint constraint optimization of edge and hue fidelity is constructed, and then a one-dimensional optimization search based on the Fibonacci method is adopted to calculate the adaptive scattering coefficient and the transmittance map under the premise of ensuring the hue fidelity, so as to finally realize image defogging based on the atmospheric scattering model.

Description

Technical Field

[0001] The present invention relates to the field of outdoor monitoring image processing, and in particular to a single image defogging method with good defogging effect, high tone fidelity in smooth areas, strong halo suppression capability, good edge preservation quality, and joint constraint optimization of edge and tone fidelity. Background Art

[0002] For video surveillance systems in outdoor scenes, the shooting quality depends largely on the weather environment. Under weather conditions such as fog and haze, a large number of tiny water droplets and aerosols suspended in the atmosphere have a scattering effect, resulting in color distortion, contrast and visibility reduction of the monitored images, which will undoubtedly affect the effectiveness and practicality of the outdoor video surveillance system. Therefore, removing or suppressing the interference of dust, haze particles and impurities is one of the key technical links to improve the performance of outdoor video surveillance systems. The fundamental purpose of image defogging is to restore the contrast and clarity of foggy images, restore their true colors and real scenes, and achieve the shooting effect under ideal weather conditions, thereby helping managers and departments make appropriate decisions.

[0003] The defogging method based on multiple images is an earlier type of method. It takes into account the randomness of dust and haze particle impurities and uses the fusion or weighted average of multiple images to achieve defogging. Grewe et al. fused multiple foggy images in the wavelet domain to obtain high-contrast images. Schechner et al. estimated the depth of field information based on the filter images and different degrees of polarization of multiple images, and then restored the image without fog. Narasimhan et al. compared the images of the same scene in different weather conditions and estimated the depth of field information and scene structure based on the changes between scene points, thereby enhancing the image contrast. Galdran et al. fused multiple images with different exposures through gamma correction and multi-scale mixed Laplace, reducing the dependence and complexity of image defogging. Although this type of method has achieved a certain defogging effect, it takes a lot of manpower and time to obtain the input image group of the same scene at different times, and sometimes it is even impossible. Therefore, researchers gradually shifted their attention to the defogging research based on a single image.

[0004] At present, typical single image dehazing methods are mainly divided into three categories: image dehazing methods based on fusion, image dehazing methods based on physical models, and image dehazing methods based on convolutional neural networks.

[0005] 1) Image dehazing methods based on fusion use more than two traditional techniques to generate multiple derivative images, and then fuse the multiple derivative images to improve the quality of image dehazing. Zhu et al. first obtained four underexposed images from the blurred image through gamma correction, then decomposed each underexposed image into overall and local components, constructed a pixel-level weight map for image fusion, and finally balanced the brightness and color saturation of the image to obtain the best haze-free image. This method can solve the saturation problem well, but the pixel-by-pixel estimation of the transmission map is prone to cause oversaturation of the dehazed image, while the block-by-block estimation of the transmission map will produce edge artifacts. Zhao et al. used a pixel-by-pixel estimated transmission map at the edge of the hazy image, selected multiple blocks of different scales in the non-edge area to estimate the transmission map block by block, and fused the final transmission map through weights, thereby proposing a MOF model. Although this method effectively solves the edge artifact phenomenon, the running complexity is not very ideal. Ancuti et al. used white balance and contrast enhancement techniques to obtain two derivative images from the original image respectively, and then retained the important features of the derivative image by calculating the weight map obtained by weighting the brightness, chromaticity and saliency, and finally used multi-scale mixing to obtain a clear image. Although this type of method increases the visible details of the blurred area by fusion and improves the visual quality of the image, the selection of adaptive weights remains a technical difficulty. When the foggy image is darker or the fog concentration is higher, inappropriate fusion weights will significantly affect the dehazing performance and produce edge artifacts and halos.

[0006] 2) Image dehazing methods based on physical models model the physical effects of light attenuation on images during atmospheric scattering, and use the inverse process to achieve image dehazing. He Kaiming proposed a dark channel prior model based on the atmospheric scattering model, and then obtained the atmospheric light value and rough transmittance, and then refined the transmittance using guided filtering or soft cutout methods. Although this method can retain image details as much as possible, the estimation of atmospheric light value and transmittance is often not accurate enough, which will cause the dehazed image to lose some texture details, and it is easy to have obvious color distortion and halo phenomenon when restoring large areas of sky. In order to solve this problem, Zhang et al. found that the maximum intensity point of each color channel in the local pixel block corresponds to the scene point with the maximum reflectivity, so as to directly estimate the ambient illumination and transmittance based on the maximum reflectivity prior. However, when the scene objects have different colors, this method often produces color distortion. Zhu et al. used the color attenuation prior model to create a linear model related to scene depth and used a supervised method to learn the model parameters. However, in order to reduce the complexity of the algorithm, this method fixed the scattering coefficient to a constant of 1, which easily underestimated the transmittance, resulting in excessive pixel changes before and after defogging, causing color cast distortion. Berman et al. found that the color distribution of haze-free images would form a tight color cluster in the RGB space, and because the distances between the pixels in the cluster and the camera were different, they would form a line in the RGB space of the haze image, called the "fog line". Based on this finding, Berman et al. proposed a color-based non-local strategy to estimate transmittance and atmospheric light values. However, the defogging image still has the phenomenon of oversaturation of color. In general, for images with high fog concentration, the processing results of this type of defogging method in the sky or smooth area are inevitably noisy and color cast distortion, and because the transmittance estimation of the atmospheric scattering model is inaccurate, the noise in the image will be amplified by the defogging process, resulting in poor subjective effect of the defogging image.

[0007] 3) Image dehazing methods based on convolutional neural networks use the powerful learning ability of neural networks to establish a complex nonlinear mapping from foggy images to fog-free images, thereby achieving the purpose of image dehazing. Chen et al. designed a PMS-Net network that can adaptively select appropriate pixel blocks for each pixel, and then solve the atmospheric light value and transmittance based on the dark channel map of the pixel block. Huang et al. proposed an end-to-end neural network to learn the residual map between foggy and fog-free images, and then use the transition convolution layer to enhance the structural information to complete dehazing. Cai et al. also constructed an end-to-end deep neural network DehazeNet, which can effectively eliminate fog evenly distributed in the image. Wu et al. proposed a dehazing network with autoencoder and contrast regularization, which fully utilizes the information of blurry and clear images to achieve dehazing. Dong et al. used frequency information as prior information and proposed an end-to-end generative adversarial network with a fusion discriminator to obtain a more natural dehazed image. Because it is difficult to capture fog-free and foggy images with the same background in nature, most of the foggy image datasets currently used by convolutional neural networks are artificially synthesized based on the atmospheric scattering model, which inevitably affects the robustness and stability of the neural network, resulting in its defogging performance on real foggy images not always being satisfactory. Moreover, this type of defogging method still cannot handle dense fog images, the training time is too long, and it lacks practicality.

[0008] In general, the existing image defogging methods based on fusion and physical models use prior knowledge to simplify physical models and estimate defogging parameters. They cannot effectively process foggy images with large smooth areas such as the sky. They are prone to inaccurate transmittance estimation and large differences in transmittance between sky and non-sky areas. They have never been able to solve the color cast and halo problems in smooth areas. Image defogging methods based on convolutional neural networks still have problems such as insufficient robustness and stability, limited defogging capabilities, and difficulty in obtaining real foggy image training data sets. Therefore, there is currently no single image defogging method that can handle both light fog and dense fog images, and has high tonal fidelity in smooth areas, strong halo suppression capabilities, and good edge retention quality. Summary of the invention

[0009] The present invention aims to solve the above-mentioned technical problems existing in the prior art and provides a single image defogging method with good defogging effect, high hue fidelity in smooth areas, strong halo suppression capability, good edge preservation quality, and joint constraint optimization of edge and hue fidelity.

[0010] The technical solution of the present invention is: a single image defogging method for joint constraint optimization of edge and hue fidelity, which is performed in the following steps: using SLIC superpixel segmentation to divide the foggy image into a series of image blocks; using normal distribution standardization and Sigmoid nonlinear mapping to construct a normalized dark channel image and a normalized detail image, generating a feature map of weighted fusion of brightness and structural details, adaptively selecting image blocks reflecting atmospheric light and estimating ideal atmospheric light values; using the scattering coefficient as an independent variable, constructing an objective function for joint constraint optimization of edge and hue fidelity, and then using a one-dimensional optimization search based on the Fibonacci method to calculate the adaptive scattering coefficient and transmittance map under the premise of ensuring hue fidelity, and finally realizing image defogging based on the atmospheric scattering model.

[0011] The specific steps are as follows:

[0012] Step 1. Input a foggy image I, let its height and width be H and W respectively;

[0013] Step 2: Use the SLIC superpixel algorithm to segment the foggy image I and obtain the pixel coordinate set corresponding to each superpixel block The Ω i represents the pixel coordinate set of the i-th superpixel block and 1≤i≤N sp , N sp is a preset constant used to represent the total number of superpixel blocks;

[0014] Step 3. Use minimum filtering to calculate the dark channel image J of the foggy image I dark ;

[0015] Step 4. Convert the color space of the foggy image I to YCbCr and set its brightness component to I Y ;

[0016] Step 5. Calculate I Y Multi-scale fusion details

[0017] Step 6. Use a square structure element of size s to fusion the multi-scale detail map Perform morphological erosion operation to obtain

[0018] Step 7. Use morphological region filling operation to Perform 8-connected region filling to obtain

[0019]

[0020] Step 8. Calculate the normalized dark channel image of the foggy image I

[0021] Step 9. Calculate the brightness component I Y Normalized detail map of

[0022] Step 10. Calculate the weighted fusion feature value F(i) of the foggy image I by using the weighted fusion method of brightness and structural details;

[0023] Step 11. Calculate the atmospheric light value A of the foggy image I;

[0024] Step 12. Calculate the scene depth map D;

[0025] Step 13: Defogging the foggy image I based on the atmospheric scattering model and constraint optimization;

[0026] Step 14. Output the dehazed image ρ.

[0027] The step 5 is calculated according to the following formula:

[0028]

[0029] Said express The value at coordinate x in I Y (x) indicates I Y The brightness value at coordinate x in G δ represents a Gaussian smoothing kernel with a standard deviation of δ, “*” represents a convolution operation, (I Y *G δ )(x) indicates I Y (x) and G δ The value after performing the convolution operation, δ1, δ2 and δ3 are all preset constants;

[0030] The step 8 is calculated according to the following formula:

[0031]

[0032] Said express The value at coordinate x in μ dark Represents the dark channel graph J dark The mathematical expectation of all pixel values ​​in , σ dark Indicates J dark The standard deviation of all pixel values ​​in .

[0033] The step 9 is calculated according to the following formula:

[0034]

[0035] Said express The value at coordinate x in express The value at coordinate x in μ blur express The mathematical expectation of all pixel values ​​in , σ blur express The standard deviation of all pixel values ​​in .

[0036] The step 10 is performed as follows:

[0037] Step 10.1 Let i←1;

[0038] Step 10.2 calculates the brightness feature value f1(i) of the i-th superpixel block;

[0039]

[0040] The |Ω i | represents the number of pixels in the i-th superpixel block;

[0041] Step 10.3 calculates the structural detail feature value f2(i) of the i-th superpixel block;

[0042]

[0043] Step 10.4 calculates the weighted fusion feature value F(i) of the brightness and structural details of the i-th superpixel block;

[0044] F(i)=f1(i)+f2(i)

[0045] Step 10.5 Let i←i+1, if i≤N sp , then return to step 10.2, otherwise go to step 11.

[0046] The step 11 is calculated according to the following formula:

[0047]

[0048] The A c represents the color channel c of A and c∈{R,G,B}, Ω k represents the pixel coordinate set of the kth superpixel block, |Ω k | represents the number of pixels in the kth superpixel block.

[0049] The step 13 is performed as follows:

[0050] Step 13: Defogging the foggy image I based on the atmospheric scattering model and constraint optimization;

[0051] Step 13.1 establish an objective evaluation model vif for edge and tone fidelity;

[0052]

[0053] In represents the reference image of vif, Ot represents the image to be evaluated of vif, y represents the coordinates of any pixel in In and Ot, In c (y) represents the value at coordinate y in the color channel c of the image In, Ot Y represents the brightness component of image Ot, represents the gradient operator, var(·) represents the operator for calculating the variance, MSCN(·) represents the mean contrast normalization MSCN operator, λ and ξ are both preset constants, θ(·) represents the hue angle estimation function, which is defined as follows;

[0054]

[0055] The p represents the pixel value, p R represents the red component of p, p G represents the green component of p, p B represents the blue component of p;

[0056] Step 13.2 Establish a defogging model based on the atmospheric scattering model

[0057]

[0058] The β represents the scattering coefficient, express Dehazing result at coordinate x;

[0059] Step 13.3 Establish the objective function β for joint optimization of edge and tone fidelity constraints * ;

[0060]

[0061] Step 13.4 uses a one-dimensional optimization search based on the Fibonacci method to solve the objective function and obtain the optimal scattering coefficient β * ;

[0062] Step 13.5 uses the defogging model based on the atmospheric scattering model and the optimal scattering coefficient given by the following formula to process the foggy image I to obtain the defogging image ρ;

[0063]

[0064] The c (x) represents the value at coordinate x in the color channel c of the image ρ and c∈{R,G,B}.

[0065] Compared with the existing technology, the present invention guarantees the subjective and objective quality of image defogging from three aspects: first, the foggy image is divided into a series of image blocks that are more in line with the topological relationship of the spatial region of the object in the real scene by using superpixel segmentation, so that the pixel distribution inside each image block has a higher consistency, which can fundamentally avoid the problem of inaccurate atmospheric light value estimation caused by the traditional regular block method due to the wrong division of pixels of different objects into the same image block, and more accurately locate the area reflecting the actual atmospheric light. Second, according to the characteristics of pixel value distribution and image texture detail distribution under multi-scale, the dark channel map and detail map are normalized and Sigmoid nonlinear mapping is performed, and a normalized dark channel map and a normalized detail map are constructed, and then a novel feature map of weighted fusion of brightness and structural details is generated, which can make the atmospheric light area and the large area of ​​white smooth area show significant state differences on the feature map, and the best candidate area reflecting the atmospheric light is selected and the atmospheric light value is estimated, which effectively suppresses the interference of the large area of ​​white smooth area on the atmospheric light value estimation process, and avoids the dark phenomenon of the defogging image caused by the excessive estimation of the atmospheric light value in the prior art. Third, an objective function with scattering coefficient as independent variable and edge and hue fidelity jointly constrained optimization is designed. Under the premise of ensuring the hue fidelity before and after defogging, the adaptive scattering coefficient and transmittance map can be calculated, which can effectively avoid the serious halo and hue distortion caused by the defogging operation in large smooth areas such as the sky. Therefore, the present invention has the advantages of good defogging effect, high hue fidelity in smooth areas, strong halo suppression ability, and good edge retention quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a comparison diagram of the defogging results of the foggy image 1 by the present invention and the prior art.

[0067] Figure 2 It is a comparison diagram of the defogging results of the foggy image 2 by the present invention and the prior art.

[0068] Figure 3 It is a comparison diagram of the defogging results of the foggy image 3 by the present invention and the prior art.

[0069] Figure 4 It is a comparison diagram of the defogging results of the foggy image 4 by the present invention and the prior art.

[0070] Figure 5 It is a comparison diagram of the defogging results of the foggy image 5 by the present invention and the prior art. DETAILED DESCRIPTION

[0071] The present invention provides a single image defogging method for joint optimization of edge and tone fidelity constraints, which is characterized by following the steps:

[0072] Step 1. Input a foggy image I, let its height and width be H and W respectively;

[0073] Step 2: Use the SLIC superpixel algorithm to segment the foggy image I and obtain the pixel coordinate set corresponding to each superpixel block The Ω i represents the pixel coordinate set of the i-th superpixel block and 1≤i≤N sp , N sp is a preset constant used to represent the total number of super pixel blocks. In this embodiment, N sp =500;

[0074] Step 3. According to formula (1), use minimum value filtering to calculate the dark channel image J of the foggy image I dark ;

[0075]

[0076] The J dark (x) represents J dark The value at coordinate x in I c represents the color channel c of I and c∈{R,G,B}, I c (x) indicates I c The pixel value at the coordinate x in , min means taking the minimum function;

[0077] Step 4. Convert the color space of the foggy image I to YCbCr and set its brightness component to I Y ;

[0078] Step 5. Calculate I according to formula (2) Y Multi-scale fusion details

[0079]

[0080] Said express The value at coordinate x in I Y (x) indicates I Y The brightness value at coordinate x in G δ represents a Gaussian smoothing kernel with a standard deviation of δ, “*” represents a convolution operation, (I Y *G δ )(x) indicates I Y (x) and G δ The values ​​after the convolution operation, δ1, δ2 and δ3 are all preset constants. In this embodiment, δ1=9, δ2=17, δ3=33;

[0081] Step 6. Use a square structure element of size s to fusion the multi-scale detail map Perform morphological erosion operation to obtain The s is a preset constant, and in this embodiment, s=7;

[0082] Step 7. Use morphological region filling operation to Perform 8-connected region filling to obtain

[0083]

[0084] Step 8. According to formula (3), calculate the normalized dark channel image of the foggy image I

[0085]

[0086] Said express The value at coordinate x in μ dark Represents the dark channel graph J dark The mathematical expectation of all pixel values ​​in , σ dark Indicates J dark The standard deviation of all pixel values ​​in ;

[0087] Step 9. Calculate the brightness component I according to formula (4): Y Normalized detail map of

[0088]

[0089] Said express The value at coordinate x in express The value at coordinate x in μ blur express The mathematical expectation of all pixel values ​​in , σ blur express The standard deviation of all pixel values ​​in ;

[0090] Step 10. Estimate the atmospheric light value A of the foggy image I by using the weighted fusion method of brightness and structural details;

[0091] Step 10.1 Let i←1;

[0092] Step 10.2: Calculate the brightness feature value f1(i) of the i-th superpixel block according to formula (5);

[0093]

[0094] The |Ω i | represents the number of pixels in the i-th superpixel block;

[0095] Step 10.3: Calculate the structural detail feature value f2(i) of the i-th superpixel block according to formula (6);

[0096]

[0097] Step 10.4: Calculate the weighted fusion feature value F(i) of the brightness and structural details of the i-th superpixel block according to formula (7);

[0098] F(i)=f1(i)+f2(i) (7)

[0099] Step 10.5 Let i←i+1, if i≤N sp , then return to step 10.2, otherwise go to step 11;

[0100] Step 11. Calculate the atmospheric light value A of the foggy image I according to formula (8) and formula (9);

[0101]

[0102] The A c represents the color channel c of A and c∈{R,G,B}, Ω k represents the pixel coordinate set of the kth superpixel block, |Ω k | represents the number of pixels in the kth superpixel block;

[0103] Step 12. Calculate the scene depth map D according to formula (10);

[0104]

[0105] D(x) represents the value at coordinate x in D, and max represents the maximum value function;

[0106] Step 13: Defogging the foggy image I based on the atmospheric scattering model and constraint optimization;

[0107] Step 13.1 According to formula (11), establish an objective evaluation model vif for edge and tone fidelity;

[0108]

[0109] In represents the reference image of vif, Ot represents the image to be evaluated of vif, y represents the coordinates of any pixel in In and Ot, In c (y) represents the value at coordinate y in the color channel c of the image In, Ot Y represents the brightness component of image Ot, represents the gradient operator, var(·) represents the operator for calculating the variance, MSCN(·) represents the mean contrast normalization MSCN operator, λ and ξ are both preset constants, in this example, λ=0.5, ξ=-0.0001, θ(·) represents the hue angle estimation function, which is defined by formula (12);

[0110]

[0111] The p represents the pixel value, p R represents the red component of p, p G represents the green component of p, p B represents the blue component of p;

[0112] Step 13.2 According to formula (13), establish a defogging model based on the atmospheric scattering model

[0113]

[0114] The β represents the scattering coefficient, express Dehazing result at coordinate x;

[0115] Step 13.3 I c and Substitute into formula (11), and then establish the objective function of joint optimization of edge and tone fidelity constraints given by formula (14);

[0116]

[0117] Step 13.4 uses a one-dimensional optimization search based on the Fibonacci method to solve the objective function of formula (14) and obtain the optimal scattering coefficient β * ;

[0118] Step 13.5: β * Substitute into formula (13), and then use the defogging model based on the atmospheric scattering model and the optimal scattering coefficient given by formula (15) to process the foggy image I to obtain the defogging image ρ;

[0119]

[0120] The c (x) represents the value at coordinate x in the color channel c of the image ρ and c∈{R,G,B};

[0121] Step 14. Output the dehazed image ρ.

[0122] The defogging results of the foggy image 1 using the present invention and the prior art are compared. Figure 1As shown in the figure, (a) is the original foggy image; (b) is the dehazing result of the IDGCP method; (c) is the dehazing result of the IDE method; (d) is the dehazing result of the Haze-Lines method; (e) is the dehazing result of the DehazeNet method; and (f) is the dehazing result of the present invention.

[0123] The defogging results of the foggy image 2 using the present invention and the prior art are compared. Figure 2 As shown in the figure, (a) is the original foggy image; (b) is the dehazing result of the IDGCP method; (c) is the dehazing result of the IDE method; (d) is the dehazing result of the Haze-Lines method; (e) is the dehazing result of the DehazeNet method; and (f) is the dehazing result of the present invention.

[0124] The defogging results of the foggy image 3 using the present invention and the prior art are compared. Figure 3 As shown in the figure, (a) is the original foggy image; (b) is the dehazing result of the IDGCP method; (c) is the dehazing result of the IDE method; (d) is the dehazing result of the Haze-Lines method; (e) is the dehazing result of the DehazeNet method; and (f) is the dehazing result of the present invention.

[0125] The defogging results of the foggy image 4 using the present invention and the prior art are compared. Figure 4 As shown in the figure, (a) is the original foggy image; (b) is the dehazing result of the IDGCP method; (c) is the dehazing result of the IDE method; (d) is the dehazing result of the Haze-Lines method; (e) is the dehazing result of the DehazeNet method; and (f) is the dehazing result of the present invention.

[0126] The defogging results of the foggy image 5 using the present invention and the prior art are compared. Figure 5 As shown in the figure, (a) is the original foggy image; (b) is the dehazing result of the IDGCP method; (c) is the dehazing result of the IDE method; (d) is the dehazing result of the Haze-Lines method; (e) is the dehazing result of the DehazeNet method; and (f) is the dehazing result of the present invention.

[0127] The RESIDE RTTS standard foggy image dataset is processed by the present invention and the prior art, and the quality of the dehazed image is objectively evaluated using the FADE, UCIQE, HRAM, R, NR-FQA, MSE, SSIM, and BRISQUE indicators. The results are shown in Table 1.

[0128] Depend on Figure 1 to Figure 5It can be seen that the dehazing results of the IDGCP method, the IDE method and the Haze-Lines method are not ideal, and will cause significant color cast and halo phenomena in the sky area; although the dehazing result of the DehazeNet method does not produce obvious color cast and halo, its dehazing ability is weak and is only effective for light fog, and its processing efficiency for thick fog is limited; in comparison, the dehazing result of the present invention is the most ideal, the image contrast is significantly improved, the picture is smooth and natural, and there is no halo and edge distortion in the smooth area.

[0129] As can be seen from Table 1, in terms of the UCIQE index, the evaluation results of the present invention are higher than those of the prior art, indicating that the overall contrast and edge retention of the image after dehazing of the present invention are higher, and the uneven color cast phenomenon is less; in terms of the HRAM index, the average score of the present invention is higher than that of IDE, Haze-Lines, and Dehaze-Net, and is second only to IDGCP, indicating that the present invention can effectively restore ambient light and has a certain ability to suppress halo; in terms of the FADE index, the image haze density estimation value of the present invention is second only to HL, indicating that the present invention has achieved an ideal dehazing effect and a low residual haze concentration. It should be pointed out that HL enhances its dehazing ability at the expense of hue fidelity, resulting in the most significant halo and color cast phenomena; in terms of NR-FQA indicators, the average scores of the present invention are lower than those of other existing methods, indicating that the dehazed images of the present invention have more prominent details, are clear and natural as a whole, and the image quality is significantly improved; in terms of R indicators, the scores of the present invention are higher than those of other existing technologies, indicating that the present invention can effectively restore pseudo edges and has high image edge fidelity; in terms of SSIM, MSE and BRISQUE indicators, the present invention has achieved evaluation results higher than those of other existing technologies, indicating that the dehazed images of the present invention have less distortion and higher subjective and objective quality.

[0130] Table 1 Comparison of objective evaluation results of defogging image quality between the present invention and the prior art

[0131]

[0132] From the above comparison results, it can be seen that for foggy images containing large smooth areas such as the sky, the present invention has achieved defogging results that are significantly superior to other existing technologies in terms of edge detail preservation and color fidelity. It can not only overcome the halo, false edge and color cast distortion existing in the existing technology, but also has a strong defogging ability.

Claims

1. A single image dehazing method with joint optimization of edge and tone fidelity constraints, characterized in that The steps are as follows: SLIC superpixel segmentation is used to divide the foggy image into a series of image blocks; normal distribution standardization and Sigmoid nonlinear mapping are used to construct a normalized dark channel image and a normalized detail image, generate a feature map of weighted fusion of brightness and structural details, adaptively select image blocks reflecting atmospheric light and estimate the ideal atmospheric light value; using the scattering coefficient as the independent variable, construct an objective function for joint constraint optimization of edge and hue fidelity, and then use a one-dimensional optimization search based on the Fibonacci method to calculate the adaptive scattering coefficient and transmittance map under the premise of ensuring hue fidelity, and finally realize image defogging based on the atmospheric scattering model; Proceed as follows: Step 1. Input a foggy image I, let its height and width be H and W respectively; Step 2: Use the SLIC superpixel algorithm to segment the foggy image I and obtain the pixel coordinate set corresponding to each superpixel block The Ω i represents the pixel coordinate set of the i-th superpixel block and 1≤i≤N sp , N sp is a preset constant used to represent the total number of superpixel blocks; Step 3. Use minimum filtering to calculate the dark channel image J of the foggy image I dark ; Step 4. Convert the color space of the foggy image I to YCbCr and set its brightness component to I Y ; Step 5. Calculate I Y Multi-scale fusion details Step 6. Use a square structure element of size s to fusion the multi-scale detail map Perform morphological erosion operation to obtain Step 7. Use morphological region filling operation to Perform 8-connected region filling to obtain Step 8. Calculate the normalized dark channel image of the foggy image I Step 9. Calculate the brightness component I Y Normalized detail map of Step 10. Calculate the weighted fusion feature value F(i) of the foggy image I by using the weighted fusion method of brightness and structural details; Step 11. Calculate the atmospheric light value A of the foggy image I; Step 12. Calculate the scene depth map D; Step 13: Defogging the foggy image I based on the atmospheric scattering model and constraint optimization; Step 13.1 establish an objective evaluation model vif for edge and tone fidelity; In represents the reference image of vif, Ot represents the image to be evaluated of vif, y represents the coordinates of any pixel in In and Ot, In c (y) represents the value at coordinate y in the color channel c of the image In, Ot Y represents the brightness component of image Ot, represents the gradient operator, var(·) represents the operator for calculating variance, MSCN(·) represents the mean contrast normalization MSCN operator, λ and ξ are both preset constants, θ(·) represents the hue angle estimation function, which is defined as follows; The p represents the pixel value, p R represents the red component of p, p G represents the green component of p, p B represents the blue component of p; Step 13.2 Establish a defogging model based on the atmospheric scattering model The β represents the scattering coefficient, express Dehazing result at coordinate x; Step 13.3 Establish the objective function β for joint optimization of edge and tone fidelity constraints * ; Step 13.4 uses a one-dimensional optimization search based on the Fibonacci method to solve the objective function and obtain the optimal scattering coefficient β * ; Step 13.5 uses the defogging model based on the atmospheric scattering model and the optimal scattering coefficient given by the following formula to process the foggy image I to obtain the defogging image ρ; The c (x) represents the value at coordinate x in the color channel c of the image ρ and c∈{R,G,B}; Step 14. Output the dehazed image ρ.

2. A single image dehazing method for joint optimization of edge and tone fidelity constraints according to claim 1, characterized in that The step 5 is calculated according to the following formula: Said express The value at coordinate x in I Y (x) indicates I Y The brightness value at coordinate x, G δ represents a Gaussian smoothing kernel with a standard deviation of δ, "*" represents a convolution operation, (I Y *G δ )(x) indicates I Y (x) and G δ The value after performing the convolution operation, δ1, δ2 and δ3 are all preset constants; The step 8 is calculated according to the following formula: Said express The value at coordinate x in μ dark Represents the dark channel graph J dark The mathematical expectation of all pixel values ​​in , σ dark Indicates J dark The standard deviation of all pixel values ​​in ; The step 9 is calculated according to the following formula: Said express The value at coordinate x in express The value at coordinate x in μ blur express The mathematical expectation of all pixel values ​​in , σ blur express The standard deviation of all pixel values ​​in .

3. A single image dehazing method for joint optimization of edge and tone fidelity constraints according to claim 2, characterized in that The step 10 is performed as follows: Step 10.1 Let i←1; Step 10.2 calculates the brightness feature value f1(i) of the i-th superpixel block; The |Ω i | represents the number of pixels in the i-th superpixel block; Step 10.3 calculates the structural detail feature value f2(i) of the i-th superpixel block; Step 10.4 calculates the weighted fusion feature value F(i) of the brightness and structural details of the i-th superpixel block; F(i)=f1(i)+f2(i) Step 10.5 Let i←i+1, if i≤N sp , then return to step 10.2, otherwise go to step 11.

4. A single image dehazing method for joint optimization of edge and tone fidelity constraints according to claim 3, characterized in that The step 11 is calculated according to the following formula: The A c represents the color channel c of A and c∈{R,G,B}, Ω k represents the pixel coordinate set of the kth superpixel block, |Ω k | represents the number of pixels in the kth superpixel block.

Images