A single image defogging method based on physical prior and deep learning

Abstract

The application relates to the technical field of image processing, and particularly discloses a single-image defogging method based on physical priori and deep learning, which is divided into two stages. In the first stage, residual Swin Transformer based on two parallel branches is used for supervised training, and transmittance and a clear image are estimated respectively. In the second stage, the network is fine-tuned in a self-supervised training manner. The second stage combines an atmospheric scattering model and a dark channel priori, combines priori knowledge to constrain network feature learning, enables the network to be trained under a physical mechanism, and improves the defogging performance of the network in a real scene. The method is trained through supervised learning and self-supervised learning, combines the advantages of physical priori and Transformer, learns features under the guidance of physical priori, improves the generalization ability of the network, and enables the network to have good recovery effects on both synthetic foggy images and real-world foggy images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a single-image dehazing method based on physical priors and deep learning. Background Technology

[0002] Fog, haze, and other complex weather conditions not only reduce landscape visibility but also affect the image quality of natural scenes captured by image capture devices, such as color loss, low saturation, and blurred texture details. This severely impacts computer vision tasks, leading to reduced accuracy in object detection and image classification. Therefore, image dehazing plays a crucial role in computer vision tasks.

[0003] Image dehazing methods are mainly divided into three types: image enhancement-based methods, prior-based methods, and deep learning-based methods. Image enhancement-based dehazing methods, such as the Retinex method, histogram equalization, and wavelet transform, do not rely on atmospheric scattering models and improve image quality by increasing contrast and saturation. However, these methods are not very effective at dehazing and are prone to oversaturation. Prior-based dehazing methods estimate the transmission map and atmospheric light, and then use an atmospheric scattering model to derive a clear image. For example, the dark channel prior method is used to evaluate the transmission map, which assumes that at least one color channel in the haze-free image has a low intensity value close to zero. Meng et al. proposed a regularized dehazing method (BCCR), which, although it imposes new constraints on the transmission map, improves the dehazing speed. Huang Wenjun et al. proposed a low-rank and dictionary representation decomposition method for dehazing dense fog and haze images. This method obtains a low-rank "fog" map through low-rank and dictionary representation decomposition, combines bicubic interpolation to generalize the local "fog" to the global image, and finally subtracts the "fog" map to recover the haze-free clear image. However, this method still suffers from distortion and unclear details. While these prior-based methods have achieved some success, they rely on the estimation of transmission maps and atmospheric light and are prone to distortion.

[0004] In recent years, deep learning technology has developed rapidly, and more and more researchers are exploring the use of Convolutional Neural Networks (CNNs) for image dehazing. Early image dehazing methods focused on recovering hazy images by estimating transmission maps and atmospheric light. For example, Li et al. proposed AOD-Net, which reformulated the atmospheric scattering model, placing the transmission map and atmospheric light into a new variable before recovering the image. Although these CNN-based algorithms can estimate the transmission map more accurately, these methods still require accurate estimation of atmospheric light and transmission maps to perfectly recover hazy images, essentially still relying on the atmospheric scattering model. To address the problem of inaccurate estimation of transmission maps and atmospheric light, some CNN methods do not rely on the estimation of transmission maps and atmospheric light, directly restoring hazy images to clear images. For example, Zheng et al. proposed a multi-guided bilateral learning framework for 4K resolution image dehazing (4K resolution image dehazing network, RIDN), which can quickly process high-resolution hazy images. Liu et al. proposed an end-to-end densely connected dilated convolutional neural network, which uses densely connected dilated convolutions to enhance the network's ability to aggregate large-scale feature information. Wang et al. proposed the DRHNet dehazing network, which first obtains a negative residual image, and then subtracts the negative residual image from the blurred image to obtain a hazy image. Although these methods have improved the dehazing effect compared with previous image dehazing methods, some of them use too many convolution operations and focus too much on detailed information, thus ignoring global information, resulting in less than ideal dehazing results.

[0005] In recent years, several studies have applied Transformer to various computer vision tasks, such as image enhancement, image segmentation, object detection, human pose estimation, and image classification. Transformer has not only achieved excellent results in these tasks but has also been applied in single-image dehazing. For example, Li et al. designed a two-stage single-image dehazing network, improving the encoder by introducing a hierarchical Transformer, effectively combining Transformer and CNN to achieve global and local feature extraction. Gao et al. designed a Transformer-based channel spatial attention module in their dehazing network and used a multi-scale parallel residual network as the backbone to extract feature information at different scales, achieving feature fusion. Li et al. proposed a hybrid dehazing network combining convolutional neural networks and Vision Transformer. This algorithm first uses a preprocessing module to extract shallow features from hazy images and then uses a symmetric network structure combining convolutional neural networks and Vision Transformer to capture local and global features of the hazy images respectively. Wang et al. proposed Uformer, which can reduce the computational complexity of high-resolution feature maps and enhance the ability to recover image details. Jiao et al. proposed a novel dehazing network based on fractional derivatives and data-driven regularization. To achieve better dehazing results, this network introduces a two-stream network combining convolutional neural networks and Transformers to construct data-driven regularization. While these methods address the problem of missing image details, they are still not thorough in dehazing areas with high fog density. Furthermore, purely prior-based methods are prone to artifacts, and purely deep learning methods are susceptible to overfitting. Summary of the Invention

[0006] This invention provides a single-image dehazing method based on physical priors and deep learning. The technical problem it solves is that although existing image dehazing methods have solved the problem of missing image details, the dehazing effect is still not thorough in areas with high fog concentration in the image. Moreover, pure prior methods are prone to artifacts, and pure deep learning methods are prone to overfitting.

[0007] To address the above technical problems, this invention provides a single-image dehazing method based on physical priors and deep learning, comprising the following steps:

[0008] S1, First Stage: Estimate the transmittance T(x) of the RGB haze image I(x) and the sharp image J in the RAW domain based on the first and second SwingIR branches, respectively. RAW (x); Based on the dark channel prior model, the global atmospheric light A and transmittance DCP_T(x) of the RGB fog map I(x) are obtained;

[0009] S2, Second Stage: Utilizing the clear image J obtained in the first stage (T(x)) RAW (x), global atmospheric light A, and transmittance DCP_T(x), are used to fine-tune the network parameters trained in the first stage and the network parameters trained in this stage through the atmospheric scattering model and the first and second generators.

[0010] Further, step S1 specifically includes the following steps:

[0011] S11. Convert the RGB fog image I(x) into a RAW fog image I in the RAW domain. RAW (x);

[0012] S12. Estimate the RAW fog map I based on the first SwinIR branch and the second SwinIR branch respectively. RAW Transmittance T(x) and clear image J RAW (x); Based on the dark channel prior model, the global atmospheric light A and transmittance DCP_T(x) of the RGB fog map I(x) are obtained;

[0013] S13, Based on the fog-free image J corresponding to the RGB fog map I(x) GT (x), transmittance T(x), global atmospheric light A, and the reconstructed RGB fog map I obtained through an atmospheric scattering model. REC_1 (x)=J GT (x)·T(x)+A·(1-T(x)), and estimate the sharp image J RAW (x) Image J in the RGB domain RAW (x)γ;

[0014] S14, Based on Image I REC_1 (x) and I(x) are trained on the first SwinIR branch with the objective of minimizing the first loss function, and on the image J aT (x), J RAW (x)γ is trained on the second SwinIR branch with the goal of minimizing the second loss function.

[0015] Furthermore, the first loss function is expressed as:

[0016] ζ SwinIR1 =Charbonnier_loss(I REC_1 (x), I(x))+SSIM_loss(I REC_1 (x), I(x)),

[0017] Among them, Charbonnier-loss(I REC_1 (x), I(x)) represent the image I REC_1Charbonnier loss between I(x) and I(x); SSIM-loss(I REC_1 (x), I(x)) represent the image I REC_1 The SSIM loss between I(x) and I(x). Further, the second loss function is expressed as:

[0018] ζ swinIR2 =Charbonnier_loss(J RAW (x) γ J GT (x))+SSIM_loss(J RAW (x) γ J GT (x)),

[0019] Charbonnier_loss(J RAW (x) γ J GT (x) represents the image J RAW (x) γ With J GT Charbonnier loss between (x), SSIM_loss(J RAW (x) γ J GT (x) represents the image J RAW (x) γ With J GT SSIM loss between (x).

[0020] Furthermore, Charbonnier_loss(J RAW (x) γ J GT The loss (x) is specifically calculated using the following formula:

[0021]

[0022] Where eps is a constant with a value of 10. -7 .

[0023] Furthermore, both the first SwinIR branch and the second SwinIR branch are based on the SwinIR architecture;

[0024] The first SwinIR branch includes a shallow feature extraction module, a deep feature extraction module, and a high-quality image reconstruction module;

[0025] The shallow feature extraction module is a 3×3 convolutional layer;

[0026] The deep feature extraction module includes two cascaded RSTB modules and a 3×3 convolutional layer; the RSTB module includes six cascaded Swing Transformer layers and a convolutional layer, and the Swing Transformer layer is equipped with 8 attention heads.

[0027] The high-quality image reconstruction module is a 3×3 convolutional layer;

[0028] The first SwinIR branch has an additional residual connection compared to the second SwinIR branch.

[0029] Furthermore, in step S2, the network parameters trained in the first stage are fine-tuned by minimizing the following loss function:

[0030] ζ Stage_2 =Charbonnier_loss(I RBC_2 (x), I(x))+SSIM_loss(I RREC_2 (x), I(x))+Charbonnier_loss(DCP_T(x), T(x)),

[0031] Charbonnier_loss(I REC_2 (x), I(x)) represent the image I REC_2 The Charbonnier loss between I(x) and I(x), I REC_2 (x) represents J obtained based on the first stage. RAW The RGB fog map reconstructed from the atmospheric scattering model and the first generator, consisting of (x)γ, transmittance T(x), and global atmospheric light A, is expressed by the following formula:

[0032] I REC_2 (x)=Generator_1(J RAW (x) γ ·T(x)+A·(1-T(x))),

[0033] Generator_1() is the function representation of the first generator;

[0034] SSIM_loss(I REC_2 (x), I(x)) represent the image I REC_2 SSIM loss between (x) and I(x);

[0035] Charbonnier_loss(DCP_T(x), T(x)) represents the Charbonnier loss between DCP_T(x) and T(x).

[0036] Furthermore, in step S2, the network parameters of the second generator are optimized by minimizing the following loss function:

[0037] ζ Generator_2 =Charbonnier_loss(J′(x),J GT (x))+SSIM_loss(J′(x), J GT (x)),

[0038] Where Charbonnier_loss(J′(x), J GT (x) represents the image J′(x) obtained by the second generator and J′(x) and J′(x) ( ... GT Charbonnier loss between (x), SSIM_loss(J′(x), J GT (x) represents the image J′(x) obtained by the second generator and J′(x) and J′(x) ( ... GT The SSIM loss between (x) is further described. The first generator employs a generator from CycleGAN; the second generator has one more skip connection from input to output than the first generator.

[0039] This invention provides a single-image dehazing method based on physical priors and deep learning, consisting of two stages. In the first stage, a SwinTransformer based on the residuals of two parallel branches is used for supervised training to estimate transmittance and the clear image, respectively. In the second stage, the network is fine-tuned using a self-supervised training approach. The second stage combines an atmospheric scattering model and a dark channel prior, using prior knowledge to constrain network feature learning, enabling the network to be trained under physical mechanisms and improving dehazing performance in real-world scenes. In the second stage, global atmospheric light is obtained through the dark channel prior. The network uses the transmittance and clear image obtained in the first stage to reconstruct the foggy image using an atmospheric scattering model, and performs self-supervised training using the original foggy image as a label. This method trains through supervised and self-supervised learning, combining the advantages of physical priors and the Transformer. Learning features under the guidance of physical priors improves the network's generalization ability, enabling the network to achieve good restoration results for both synthetic and real-world foggy images. Attached Figure Description

[0040] Figure 1 This is a network architecture diagram of SwinTD-Net provided in an embodiment of the present invention;

[0041] Figure 2 This is a function curve diagram of gamma correction and inverse gamma correction provided in the embodiments of the present invention;

[0042] Figure 3This is a network architecture diagram of SwinIR branch 1 provided in an embodiment of the present invention;

[0043] Figure 4 This is a network architecture diagram of SwinIR branch 2 provided in an embodiment of the present invention;

[0044] Figure 5 This is a network architecture diagram of the RSTB module and STL module provided in an embodiment of the present invention;

[0045] Figure 6 This is a network architecture diagram of generator 1 and generator 2 provided in an embodiment of the present invention;

[0046] Figure 7 These are comparison images of indoor synthetic foggy day image defogging results provided in embodiments of the present invention;

[0047] Figure 8 These are comparison images of defogging results for real outdoor foggy images provided in embodiments of the present invention. Detailed Implementation

[0048] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments are given for illustrative purposes only and should not be construed as limiting the present invention. The accompanying drawings are for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0049] Dehazing a single image is a challenging task for computer vision applications. Methods based on physical models and prior knowledge fail under certain conditions, leading to defects such as color distortion. Transformer-based methods possess a self-attention mechanism that effectively captures global information and exhibits strong representational capabilities. However, they are computationally expensive, and their weak inductive bias increases the risk of overfitting on small sample datasets. To address these issues, this paper proposes a novel physically-prior-guided image dehazing Transformer, named SwinTD-Net, whose network architecture is as follows: Figure 1 As shown, it is trained through supervised and self-supervised learning, combining the advantages of physical priors and the Transformer. The algorithm learns features under the guidance of physical priors, improving the network's generalization ability and enabling it to achieve good reconstruction results for both synthetic and real-world foggy images.

[0050] Based on the SwinTD-Net, this invention provides a single-image dehazing method based on physical priors and deep learning, specifically including the following steps:

[0051] S1, First Stage (Stage 1): Estimate the transmittance T(x) of the RGB haze image I(x) and the sharp image J in the RAW domain based on the first and second SwingIR branches, respectively. RAW (x); Based on the dark channel prior model (DCP), obtain the global atmospheric light A and transmittance DCP_T(x) of the RGB fog map I(x);

[0052] S2, Second Stage (Stage 2): Utilizing the T(x) and clear image J obtained in the first stage RAW (x), global atmospheric light A, and transmittance DCP_T(x), are used to fine-tune the network parameters trained in the first stage and the network parameters trained in this stage through the atmospheric scattering model (ASM) and the first and second generators.

[0053] The Atmospheric Scattering Model (ASM) is expressed as follows:

[0054] I(x)=J(x)t(x)+A(1-t(x)) (1)

[0055] Where x represents the pixel coordinates in the image, I(x) is the RGB fog image captured in foggy weather, J(x) represents the scene radiance (i.e., the clear image to be restored), A represents global atmospheric light, and T(x) represents transmittance. The atmospheric scattering model is a mathematical model of the image imaging process under foggy weather. The process of defogging using the atmospheric scattering model involves solving or estimating J(x) using the known I(x). This is an ill-posed problem and cannot be solved directly, so many methods utilize prior knowledge for estimation, such as the well-known Dark Channel Prior (DCP) model.

[0056] For decades, single-frame image dehazing algorithms based on ASM and DCP have primarily operated on RGB images. However, images captured by electronic devices such as digital cameras undergo a series of processes via the ISP Pipeline, converting them from the original RAW domain to RGB domain. Gamma correction in the ISP Pipeline is a non-linear operation that alters the linear relationship of light intensity in the original scene, such as... Figure 2 As shown. This affects the use of physical priors and the extraction of haze features by neural networks. Therefore, we restore the RGB image to a RAW image as input for ASM and DCP, which is more consistent with the atmospheric scattering mechanism. Numerically, the RGB image can be restored to a RAW image by performing inverse gamma correction:

[0057]

[0058] γ is the gamma correction value in the ISP Pipeline, typically 1 / 2.2.RAW (x) represents the RAW image that restores the linear relationship of the scene's radiant light intensity. In our SwinTD-Net, we first preprocess the RGB image into a RAW image through inverse gamma correction, and finally convert the restored RAW image back into an RGB image through gamma correction.

[0059] We divided the defogging task into two phases, such as Figure 1 As shown, this involves estimating transmittance and sharp images through supervised learning, and improving dehazing performance in the real world based on self-supervised learning. Prior knowledge-based methods achieve better dehazing results but introduce artifacts. Deep learning-based methods produce visually better results but may not fully dehaze the image and are prone to overfitting, leading to poor generalization. Therefore, our network architecture combines the advantages of both methods.

[0060] Further, see Figure 1 Step S1 specifically includes the following steps:

[0061] S11. Convert the RGB fog image I(x) into a RAW fog image I in the RAW domain. RAW (x);

[0062] S12. Estimate the RAW fog map I based on the first SwinIR branch and the second SwinIR branch respectively. RAW Transmittance T(x) and clear image J RAW (x); Based on the dark channel prior model, obtain the global atmospheric light A and transmittance DCP_T(x) of the RGB fog map I(x);

[0063] S13, Based on the fog-free image J corresponding to the RGB fog map I(x) GT (x), transmittance T(x), global atmospheric light A, and the reconstructed RGB fog map I obtained through an atmospheric scattering model. REC_1 (x)=J GT (x)·T(x)+A·(1-T(x)), and estimate the sharp image J RAW (x) Image J in the RGB domain RAW (x) γ ;

[0064] S14, Based on Image I REC_1 I(x) and I(x) are trained to minimize the first loss function, and the first SwingIR branch is trained based on the image J. GT (x), Image J RAW (x)γ is trained to minimize the second loss function.

[0065] In the first stage, the RGB blurred image is converted to a RAW image using equation (2), and then estimated using two parallel simplified SwinIR branches (the first SwinIR branch, i.e., SwinIR branch 1, and the second SwinIR branch, i.e., SwinIR branch 2). Figure 1 Clear images of transmittance T(x) and the RAW domain J RAW (x).

[0066] We employed two parallel SwinIR architectures. To reduce model parameters, we simplified the original SwinIR architecture. The original SwinIR model contained six RSTB modules for image restoration. In our image dehazing task, we reduced the number of RSTB modules to two while achieving good results. Furthermore, we increased the number of attention heads in the STL to eight. Moreover, the simplified SwinIR is more suitable for image dehazing tasks and also provides value for the research of lightweight dehazing networks. The structures of the first and second SwinIR branches are as follows: Figure 3 and Figure 4 As shown in the diagram, shallow feature extraction uses a 3×3 convolutional layer, and the Conv function following the RSTB module is also a 3×3 convolutional layer. High-quality (HQ) image reconstruction aggregates the shallow and deep features from the Swin transform convolution. The reconstruction function also uses a 3×3 convolutional layer in our dehazing task. Shallow features mainly contain low-frequency information, while deep features mainly contain high-frequency information; these are ultimately aggregated through the reconstruction module.

[0067] The structure of the RSTB module is as follows: Figure 5 As shown in (a), the RSTB module consists of six cascaded Swin Transformer layers (STL) and one convolutional layer. The Swin Transformer layers contain eight attention heads. The structure of the Swin Transformer layer is as follows: Figure 5 As shown in (b), the convolutional layers in the RSTB module enable the Swin Transformer to incorporate biases and provide the basis for the final feature aggregation.

[0068] To make training more stable and converge faster, we added a residual connection to the simplified SwinIR branch 2. Under our experimental machine and conditions, we measured the inference speed of the simplified SwinIR and the original SwinIR. Each time we input an image (resized to 128*128), we calculated the average 100 times and repeated the operation 10 times. After testing, the original SwinIR achieved 4.8 FPS, while our simplified SwinIR achieved 7.2 FPS, representing a significant improvement in inference speed.

[0069] SwinIR branch 2 is used to estimate scene radiation J in the RAW domain. RAW (x), and then restore it to the RGB space image through gamma correction (i.e. Figure 1 In We used Charbonnier loss and SSIM loss. The parameters of SwinIR branch 2 were optimized by minimizing the loss function.

[0070] ζ swinIR2 =Charbonnier_loss(J RAW (x) γ J GT (x))+SSIM_loss(J RAW (x) γ J GT (x)) (3)

[0071] Among them, J GT (x) represents the ground truth (the real reconstructed image) corresponding to the network input. Charbonnier-loss (J) RAW (x) γ J GT (x) represents the image J RAW (x) γ With J GT Charbonnier loss between (x), SSIM_loss(J RAW (x) γ J GT (x) represents the image J RAW (x) γ With J GT SSIM loss between (x).

[0072] The Charbonnier loss can be expressed as:

[0073]

[0074] Where eps is a constant with a value of 10. -7 The loss of SSIM can be expressed as:

[0075]

[0076] Where Num is the total number of pixels.

[0077] The purpose of SwinIR branch 1 is to estimate the transmittance T(x). Since the transmittance of outdoor fog maps has almost no ground truth, we use J... GT(x) Reconstruct the fog map to train the SwinIR branch 1. According to ASM, the reconstructed RGB fog map can be represented as:

[0078] I REC_1 (x)=J GT (x)·T(x)+A·(1-T(x)) (6)

[0079] A is estimated by DCP. We optimize the parameters of SwinIR branch 1 by minimizing the following loss function:

[0080] ζ SwinIR1 =Charbonnier_loss(I REC_1 (x), I(x))+SSIM_loss(I REC_1 (x), I(x)) (7)

[0081] Among them, Charbonnier-loss(I REC_1 (x), I(x)) represent the image I REC_1 The Charbonnier loss between I(x) and I(x); SSIM_loss(I REC_1 (x), I(x)) represent the image I REC_1 The SSIM loss between (x) and I(x) is calculated in the same way as in equations (4) and (5).

[0082] The second stage focuses on improving the network's dehazing performance on real-world foggy images. In this stage, we use only the fog image, without the corresponding ground truth, to fine-tune the network through self-supervised learning. This approach is highly effective for real-world foggy images, as it is difficult to obtain a clear image from a real-world fog image. We also combine DCP and ASM in this stage to allow the network to further learn physical priors. However, we encountered two problems during our experiments. First, we found that when using A (estimated by DCP), T(x) (estimated by SwinIR branch 1), and... (Estimated by SwinIR branch 2) When performing self-supervised learning to reconstruct fog images using ASM, the restored clear images obtained by fine-tuning the network do not show significant improvement. Secondly, we found that the proposed network model exhibits dark patches in local regions of the restored images in synthetic blur datasets, which may be due to the uniform haze distribution in synthetic fog images. Regarding the first issue, we argue that the process of reconstructing fog images based on ASM is not accurate enough. This is relatively easy to understand, as the input images at this stage are primarily blurry images from the real world. These images have different haze concentrations and uneven haze distributions, but ASM is an idealized model and not entirely applicable to complex haze scenes. Therefore, after reconstructing the blurry images based on ASM, we use Generator 1 (the first generator) to further fit the imaging process of real-world fog images to compensate for the shortcomings of ASM and DCP. For the second issue, we train Generator 2 (the second generator) as an image intensifier to further enhance scene restoration in dark patches of the image. The two generators we use are relatively simple generators in CycleGAN, mainly composed of 9 residual blocks. The structures of Generator 1 and Generator 2 are as follows: Figure 6 (a) and Figure 6 As shown in (b), generator 2 has one more skip connection from input to output than generator 1.

[0083] ASM, DCP, and Generator 1 all participate in self-supervised learning, fine-tuning the network parameters pre-trained in the first stage. The reconstructed blurred image I in this stage... REC_2 (x) can be represented as:

[0084] I REC_2 (x)=Generator_1(J RAw (x) γ ·T(x)+A·(1-T(x))) (8)

[0085] Generator_1() is the function representation of the first generator, SSIM-loss(I REC_2 (x), I(x)) represent the image I REC_2 The SSIM loss between I(x) and I(x), Charbonnier_loss(DCP_T(x), T(x)) represents the Charbonnier loss between DCP_T(x) and T(x), and is calculated in the same way as in equations (4) and (5).

[0086] We fine-tune the network parameters in the first stage by minimizing the following loss function:

[0087] ζ Stage_2 =Charbonnier_loss(I REC_2(x), I(x))+SSIM_loss(I REC_2 (x), I(x))+Charbonnier_loss(DCP_T(x), T(x)) (9)

[0088] Where DCP_T(x) is the transmittance estimated by DCP. In this loss function, we add a DCP prior constraint to T(x) to make T(x) estimated by SwinIR branch 1 more consistent with the physical prior. Through experiments, we found that without a prior constraint on T(x), the fine-tuning process will be very unstable and difficult to converge. For generator 2, we optimize the network parameters of generator 2 by minimizing the following loss function:

[0089] ζ Generator_2 =Charbonnier_loss(J′(x),J GT (x))+SSIM_loss(J′(x), J GT (x)) (10)

[0090] Where Charbonnier_loss(J′(x), J GT (x) represents the image J′(x) obtained by the second generator and J′(x) and J′(x) ( ... GT Charbonnier loss between (x), SSIM_loss(J′(x), J GT (x) represents the image J′(x) obtained by the second generator and J′(x) and J′(x) ( ... GT The SSIM loss between (x) is calculated in the same way as in equations (4) and (5).

[0091] In summary, this invention provides a single-image dehazing method based on physical priors and deep learning, consisting of two stages. In the first stage, a Swin Transformer based on the residuals of two parallel branches is used for supervised training to estimate transmittance and the clear image, respectively. In the second stage, the network is fine-tuned using a self-supervised training approach. The second stage combines an atmospheric scattering model and a dark channel prior, using prior knowledge to constrain network feature learning, enabling the network to be trained under physical mechanisms and improving dehazing performance in real-world scenes. In the second stage, global atmospheric light is obtained through the dark channel prior. The network uses the transmittance and clear image obtained in the first stage to reconstruct the foggy image using an atmospheric scattering model, and performs self-supervised training using the original input foggy image as a label. This method trains through supervised and self-supervised learning, combining the advantages of physical priors and the Transformer. Learning features under the guidance of physical priors improves the network's generalization ability, enabling the network to achieve good restoration results for both synthetic and real-world foggy images.

[0092] The effectiveness of this method will be verified by experiments below.

[0093] We experimentally compared our method with several state-of-the-art image dehazing techniques, including those based on prior knowledge and those based on deep learning, such as DCP, DehazeNet, AODNet, GridDehazeNet, MSBDN, FFANet, RefineDNet, PSD, and D... 4 For quantitative evaluation, we used PSNR and SSIM metrics for comparison. To ensure the fairness of the experiment, we retested all datasets used for experimental comparison, including visualizations and quantitative evaluations, using publicly available code and models employing these methods.

[0094] For synthesized foggy images Figure 7 This section compares the visualization results of different dehazing methods used to synthesize foggy images indoors. It can be seen that the image restored by DCP is darker, while the images restored by DehazeNet, AODNet, PSD, and D... 4 The restored image still retains residual haze. The image restored by GridDehazeNet is distorted. The image restored by MSBDN has insufficient contrast and detail recovery, as shown in the magnified portion of the image. The FFANet method is somewhat over-deblurred, excessively increasing the brightness of dark areas in the magnified portion of the image. The image restored by RefineDNet has obvious noise, as shown by the significant color deviation of the table in the image. The image restored by our method is closest to the ground truth (GT) in color and detail, and our method also has the highest PSNR and SSIM scores in the image, further validating the performance of our method.

[0095] The main purpose of image dehazing is to preprocess advanced computer vision tasks in realistic foggy environments. For outdoor images in real foggy conditions, such as... Figure 8 As shown, this further validates the performance of our method on real-world fog maps. The figures reveal that DCP and AODNet recover less fog, but the colors are too dark. Images recovered using DehazeNet and PSD methods have higher brightness, but poor dehazing results. Using GridDehazeNet, MSBDN, FFANet, RefeneDNet, and D... 4 Images restored by other methods all have residual haze, while the haze restored by our method is minimal and closest to the ground truth image. The PSNR and SSIM scores in the figure are also the highest for our method, further validating its performance.

[0096] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A single image defogging method based on physical prior and deep learning, characterized in that, The method comprises the steps of: S1, First Stage: Estimate RGB fog map based on the first and second SwinIR branches respectively. transmittance and clear images in the RAW domain The RGB fog map is obtained based on the dark channel prior model. Global atmospheric light A and transmittance ; The step S1 specifically comprises the steps of: S11, converting the RGB fog map into a RAW fog map converting the RAW fog map into a RAW domain ; S12, estimate the RAW fog map based on the first SwinIR branch and the second SwinIR branch respectively transmittance T(x) and a clear image ; obtain global atmospheric light A and transmittance of the RGB fog map based on a dark channel prior model ;​ S13, reconstructing an RGB haze map based on the RGB haze map the corresponding haze-free image , transmittance , global atmospheric light A by atmospheric scattering model , and estimating a clear image image in the RGB domain ; S14, based on the image , training the first SwinIR branch targeting at minimizing a first loss function, and based on the image , training the second SwinIR branch targeting at minimizing a second loss function; S2, second stage: using the first stage obtained , clear image , global atmospheric light A and transmittance , through the atmospheric scattering model and the first generator, the second generator, the network parameters trained in the first stage and the network parameters trained in this stage are fine-tuned; In the step S2, the network parameters of the first-stage trained network are fine-tuned by minimizing the following loss function: ， representing an image with between loss, is based on the first stage , transmittance , global atmospheric light A is reconstructed by atmospheric scattering model and the first generator, expressed by formula: ， is a function of the first generator; representing an image with between losses; denotes with between losses; In the step S2, the network parameters of the second generator are optimized by minimizing the following loss function: ， wherein, denotes the image obtained by the second generator and between losses, denotes the loss between the image and between losses.

2. The single image defogging method based on physical priors and deep learning according to claim 1, characterized in that, The first loss function is represented as: ， wherein represents the image and between losses; represents the image and between losses.

3. The single image defogging method based on physical priors and deep learning according to claim 2, characterized in that, The second loss function is represented as: ， Representing an image and Between loss, Representing an image and Between loss.

4. The single image defogging method based on physical priors and deep learning according to claim 3, characterized in that, The loss is specifically calculated by the following formula: ， wherein is a constant having a value of 10 −7 .

5. The single image defogging method based on physical priors and deep learning according to claim 4, characterized in that, The first SwinIR branch and the second SwinIR branch are both based on a SwinIR architecture; The first SwinIR branch comprises a shallow feature extraction module, a deep feature extraction module and a high-quality image reconstruction module; The shallow feature extraction module is a 3*3 convolution layer; The deep feature extraction module comprises two cascaded RSTB modules and a 3*3 convolution layer; the RSTB module comprises six cascaded Swin Transformer layers and a convolution layer, and eight attention heads are arranged in the Swin Transformer layer; The high-quality image reconstruction module is a 3*3 convolution layer; The first SwinIR branch is increased with a residual connection compared with the second SwinIR branch.

6. The single image defogging method based on physical priors and deep learning according to claim 1, characterized in that: The first generator adopts a generator in CycleGan; the second generator is additionally provided with a skip connection from input to output compared with the first generator.

Images