Zero sample image defogging algorithm based on transmission graph invariance

Abstract

The present application relates to the technical field of computer vision image enhancement, and particularly discloses a zero sample image defogging algorithm based on transmission graph invariance, which firstly inputs a foggy image I1 into a defogging network, the defogging network estimates a corresponding feature transmission graph t1 and an atmospheric light value A1, obtains a defogging image J through an atmospheric scattering model, then inputs J into a degradation (fogging) network to estimate a corresponding feature transmission graph t2 and an atmospheric light value A2, obtains a degraded image I2 through the atmospheric scattering model, uses a loss function to make I2 equal to I1, and combines prior knowledge to ensure the accuracy of the defogging image, the present application uses the consistency of the feature transmission graph in the image restoration and degradation process, regards the image degradation as the inverse process of the image restoration, excavates the potential contribution of the image degradation to the image restoration, avoids the domain barrier problem caused by the data set, and in addition, the fog generated by the degradation (fogging) network is more close to the real fog distribution, and the method can reduce the gap between the synthetic data set and the real data set.

Description

Technical Field

[0001] This invention relates to the field of computer vision image enhancement technology, and specifically to a zero-sample image dehazing algorithm based on transport graph invariance. Background Technology

[0002] Image dehazing is an image processing technique used to improve the blurriness and low contrast of images caused by haze or atmospheric humidity. The earliest image dehazing methods were mainly based on physical models, using atmospheric scattering models and color attenuation principles to restore image details. With the development of computer vision and machine learning technologies, data-driven methods have also been widely applied in the field of image dehazing.

[0003] These methods predict and recover images by learning statistical features from a large amount of training data containing both foggy and fog-free image pairs. Foggy For detail analysis in images, common data-driven methods include those based on convolutional neural networks (CNNs) and generative adversarial networks (GANs). Algorithm Several methods based on image priors and statistical properties have also been extensively studied. For example, the dark channel prior posits that in most natural images, there exists at least one color channel whose pixel value is close to zero in haze-free regions. By utilizing this prior, the haze transmission rate can be estimated, and image details can be further recovered. ; color attenuation prior discovery In hazy images, the difference between saturation and brightness is larger, while in clear images, the difference is very small. 。 However, prior-based methods have significant limitations, as the preconditions cannot be applied to all scenarios. With the development of neural networks, learning-based dehazing methods have emerged, but obtaining paired hazy images remains difficult. Learning-based dehazing algorithm Dehazing performance of networks Training on synthetic images Degradation (fogging) On real images Degradation (fogging) The results were significantly reduced, leading to the development of self-supervised, semi-supervised, and weakly supervised methods. While these methods avoided the need for paired images, they still required a large number of images for training. This resulted in higher requirements for the network structure, the model remembering the data features from training, leading to poor performance on other types of images (with different features), long training time, difficulty in model convergence, and weak generalization ability of the trained model. Summary of the Invention

[0004] The purpose of this invention is to provide a zero-sample image dehazing algorithm based on transport graph invariance to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A zero sample image defogging algorithm based on transmission map invariance, comprising the following steps:

[0007] Step one: input a single foggy image I1 into the defogging network N1 to estimate its corresponding feature transmission map t1 and atmospheric light value A1, and obtain a defogged image J using an atmospheric scattering model, as follows:

[0008]

[0009] Step two: input the defogged image J into the Figure 1 network N2 to estimate its corresponding feature transmission map t2 and atmospheric light value A2, and obtain a degraded (fogged) image I2 using an atmospheric scattering model, as follows:

[0010] I2 = Jt2(x) + A2(1-t2(x));

[0011] Step three: a combination function L of several no-reference loss functions and a hand-crafted prior loss function is used to constrain the network, and each loss function is as follows:

[0012] L1 = ||I1(x)-I2(x)||1;

[0013] In order to ensure the cycle consistency in the image processing process, a cycle consistency loss function L1 is used to make I2 infinitely close to I1, wherein || ||1 represents L1 regularization;

[0014] L2 = ||A1(x)-A2(x)||1;

[0015] wherein A1 and A2 represent the atmospheric light values estimated by the defogging network N1 and the degradation (fogging) network N2 respectively, wherein ||||1 represents L1 regularization, which ensures that A1 is equal to A2;

[0016] L3 = ||t1(x)-t2(x)||1;

[0017] wherein t1 and t2 represent the feature transmission maps estimated by the defogging network N1 and the degradation (fogging) network N2 respectively;

[0018]

[0019] wherein x represents an image pixel, c represents an original color channel, J is a defogged image, max() represents taking a larger value, min() represents taking a smaller value, and the loss function can constrain the pixel value of the image to be between [0, 1], ensuring the correctness of the defogged image;

[0020]

[0021] wherein J c() represents a color channel, J represents a defogging image, the loss function can help the network to correct the accuracy of the output, and improve the defogging performance of the network;

[0022] L = 1.0 x L1 + 1.0 x L2 + 1.0 x L3 + 0.001 x L4 + 0.01 x L5;

[0023] Step four: degradation (fogging) network N2 After convergence, input a single clear image J2 to be fogged into Figure 1 the network N2 to obtain its corresponding feature transmission map t3 and atmospheric light value A3, and then synthesize a hazy image hazy based on the atmospheric scattering model formula syn :

[0024] hazy syn = J2t3(x) + A3(1-t3(x));

[0025] Compared with the prior art, the beneficial effects of the present application are:

[0026] (1) The zero sample defogging method of the present application can perform the defogging task only with one hazy image, does not need a large number of images for training, and does not have the problem of weak generalization ability for different types of images, has no regional obstacles, has a simple network structure requirement, does not have the problem of weak generalization ability, can use the image degradation (fogging) process to transfer the real fog to the clear image, generate a more real fog distribution, in order to narrow the gap between the synthetic domain and the real domain, improve the image contrast, improve the image visual effect, and generate a more real hazy image. BRIEF DESCRIPTION OF DRAWINGS

[0027] Degradation (fogging) The algorithm flowchart of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0029] Embodiment one:

[0030] Please refer to Degradation (fogging) ,

[0031] The zero sample image defogging algorithm based on transmission map invariance specifically includes the following steps:

[0032] Step one: input a single foggy image I1 into the dehazing network N1 to estimate its corresponding feature transmission map t1 and the atmospheric light value A1, and obtain the dehazing image J by using the atmospheric scattering model, as follows:

[0033]

[0034] Step two: input the dehazing image J into the network N2 to estimate its corresponding feature transmission map t2 and the atmospheric light value A2, and obtain the degradation (fogging) image I2 by using the atmospheric scattering model, as follows: Dehazing

[0035] I2 = Jt2(x) + A2(1-t2(x));

[0036] Step three: the combination function L of a plurality of referenceless loss functions and a manual prior loss function is used to constrain the network, and each loss function is as follows:

[0037] L1 = ||I1(x)-I2(x)||1;

[0038] In order to ensure the loop consistency in the image processing process, the loop consistency loss function L1 is used to make I2 infinitely close to I1, wherein || ||1 represents L1 regularization;

[0039] L2 = ||A1(x)-A2(x)||1;

[0040] wherein A1 and A2 represent the atmospheric light values estimated by the dehazing network N1 and the degradation (fogging) network N2 respectively, wherein ||||1 represents L1 regularization, which ensures that A1 is equal to A2;

[0041] L3 = ||t1(x)-t2(x)||1;

[0042] wherein t1 and t2 represent the feature transmission maps estimated by the dehazing network N1 and the degradation (fogging) network N2 respectively;

[0043]

[0044] wherein x represents an image pixel, c represents an original color channel, J is a dehazing image, max() represents taking a larger value, min() represents taking a smaller value, and the loss function can constrain the pixel value of the image between [0, 1], ensuring the correctness of the dehazing image;

[0045]

[0046] wherein J c () represents a color channel, and J represents a dehazing image. The loss function can help the network to correct the accuracy of the output and improve the dehazing performance of the network; ​

[0047] L = 1.0 x L1 + 1.0 x L2 + 1.0 x L3 + 0.001 x L4 + 0.01 x L5;

[0048] Step four: after the convergence of the degradation (fogging) network, input the single clear image J2 to be fogged into the network N2 to obtain its corresponding feature transmission map t3 and atmospheric light value A3, and then synthesize the hazy image hazy based on the atmospheric scattering model formula. Degradation (fogging) syn

[0049] hazy syn = J2t3(x) + A3(1 - t3(x)).

[0050] As can be seen from the above, the algorithm model uses a natural image enhancement network as a basic framework, estimates the feature transmission map and atmospheric light value corresponding to the hazy image by maintaining the relationship between the feature transmission map during image restoration and degradation, and obtains the corresponding clear image to achieve the purpose of natural image defogging.

[0051] First, input the hazy image I1 to the network N1 to obtain the feature transmission map t1 and the atmospheric light value A1, and obtain the haze-free image J based on the atmospheric scattering model formula. Input the haze-free image J to the network N2 to obtain the feature transmission map t2 and the atmospheric light value A2, and obtain the reconstructed image I2 through the atmospheric scattering model formula. Degradation Degradation (fogging) ​ ​ syn

[0052] (1) Introduce the image degradation (fogging) process, and regard the image degradation (fogging) process as the inverse process of image restoration (defogging). In this process, the parameters (feature transmission map and atmospheric light value) output by the network are kept consistent, and only one hazy image is needed to realize the zero-sample image defogging task;

[0053] (2) A combination of a series of no-reference loss functions and manual prior loss functions helps the network to obtain the correct expected image;

[0054] (3) Use the network trained in the image degradation (fogging) process to transfer the real-world fog to the clear image, synthesize a hazy image closer to the real fog distribution, and reduce the gap between the synthetic domain and the real domain.

[0055] ​​​​​​​While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. A zero sample image defogging algorithm based on transmission graph invariance, characterized in that, Comprising the following steps: Step one: input a single foggy image I1 into the dehazing network N1 to estimate its corresponding feature transmission map t1 and atmospheric light value A1, and obtain the dehazed image J using the atmospheric scattering model, as follows: Step two: input the dehazed image J into the degradation network N2 to estimate its corresponding feature transmission map t2 and atmospheric light value A2, and obtain the degraded image I2 using the atmospheric scattering model, as follows: I2 = Jt2(x) + A2(1-t2(x)); Step three: a combination function L of several no-reference loss functions and a manual prior loss function is used to constrain the network, and each loss function is as follows: L1 = ||I1(x)-I2(x)||1; In order to ensure the consistency of the image processing process, a cycle consistency loss function L1 is used to make I2 infinitely close to I1, wherein ||·||1 represents L1 regularization; L2 = ||A1(x)-A2(x)||1; Wherein A1 and A2 represent the atmospheric light values estimated by the dehazing network N1 and the degradation network N2 respectively, wherein ||·||1 represents L1 regularization, to ensure that A1 is equal to A2; L3 = ||t1(x)-t2(x)||1; Wherein t1 and t2 represent the feature transmission maps estimated by the dehazing network N1 and the degradation network N2 respectively; Wherein x represents the image pixel, c represents the original color channel, J is the dehazed image, max(·) represents the larger value, min(·) represents the smaller value, the loss function can constrain the pixel value of the image between [0, 1], and ensure the correctness of the dehazed image; wherein J c (·) represents a color channel, J represents a defogging image, and the loss function can help the network correct the accuracy of the output and improve the defogging performance of the network; L = 1.0xL1 + 1.0xL2 + 1.0xL3 + 0.001xL4 + 0.01xL5; Step four: after the convergence of the degradation network, input the single clear image J2 to be atomized into the degradation network N2 to obtain its corresponding feature transmission diagram t3 and atmospheric light value A3, and then synthesize the hazy image hazy based on the atmospheric scattering model formula syn : hazy syn = J2t3(x) + A3(1 - t3(x)).

2. The zero sample image defogging algorithm based on transmission graph invariance according to claim 1, characterized in that: In the step one, step two and step four: Input a single foggy image I1 into the dehazing network N1 to estimate its corresponding feature transmission map t1 and atmospheric light value A1, input the dehazed image J into the degradation network N2 to estimate its corresponding feature transmission map t2 and atmospheric light value A2, and obtain the degraded image I2 using the atmospheric scattering model, complete the image processing cycle of foggy image to dehazed image to foggy image, as follows: I2 = Jt2(x) + A2(1-t2(x)) Use a cycle consistency loss function L1 to make I2 infinitely close to I1, and the expression of L1 is as follows: L1 = ||I1(x)-I2(x)||1 After the degradation network converges, a single clear image J2 to be atomized is input into the degradation network N2 to obtain its corresponding feature transmission map t3 and atmospheric light value A3, and then a hazy image hazy is synthesized based on the atmospheric scattering model formula syn : hazy syn = J2t3(x) + A3(1 - t3(x)).

3. The zero sample image defogging algorithm based on transmission graph invariance according to claim 1, characterized in that: Comprising the following algorithm steps: Introduce a combination function L to constrain the network, as follows: L = 1.0xL1 + 1.0xL2 + 1.0xL3 + 0.001xL4 + 0.01xL5 I. L2 is an atmospheric light consistency loss function, as follows: L2 = ||A1(x)-A2(x)||1 Wherein A1 and A2 represent the atmospheric light values estimated by the dehazing network N1 and the degradation network N2 respectively; II. L3 is a transmission map consistency loss function, as follows: L3 = ||t1(x)-t2(x)||1 Wherein t1 and t2 represent the feature transmission maps estimated by the dehazing network N1 and the degradation network N2 respectively; III. L4 is the pixel overflow penalty loss function, the formula is as follows: Where x represents the image pixel, c represents the original color channel, J is the defogging image, max(·) represents taking the larger value, min(·) represents taking the smaller value, the loss function can constrain the pixel value of the image between [0, 1], and ensure the correctness of the defogging image; IV. L5 is the dark channel prior loss function, the formula is as follows: where r, g, b represent red, green and blue color respectively, J c (·) represents the color channel, J represents the defogged image.

Images