Image defogging method based on atmospheric scattering model

Abstract

The application belongs to the technical field of computer vision, and particularly relates to an image defogging method based on an atmospheric scattering model; the method comprises the following steps: obtaining an original spatial fog image and inputting the original spatial fog image into a trained image defogging model for processing to obtain a clear image after defogging; the processing of the image defogging model on the original spatial fog image comprises the following steps: processing the original spatial fog image by using a feature extraction module to obtain a first, a second and a third feature spatial fog image; inputting the feature spatial fog image into a haze extraction unit for processing to obtain a haze feature map; processing the haze feature map according to the first, the second and the third feature spatial fog image by using a feature decoding module to obtain a feature fusion map; performing inverse operation on the feature fusion map according to the atmospheric scattering model to obtain the clear image after defogging; the method significantly improves the defogging precision of the model, achieves good defogging effect, and effectively improves the defogging precision of the image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to an image defogging method based on an atmospheric scattering model. BACKGROUND

[0002] Image defogging technology is an important means to solve the problems of image blurring, contrast reduction and color distortion caused by suspended particulate matter in the atmosphere. In the fields of traffic monitoring, unmanned driving, smart home, medical imaging, etc., the clarity and quality of images are crucial. However, environmental factors such as fog and haze often cause the quality of images obtained by imaging systems to decrease significantly, which not only affects the visual effect, but also reduces the accuracy of subsequent computer vision tasks.

[0003] Traditional image defogging algorithms are mostly based on the atmospheric scattering model, which assumes that light scattering and attenuation caused by fog and haze are the main reasons for image quality degradation. Early defogging methods mainly rely on image enhancement techniques such as histogram equalization, frequency domain transformation, and defogging methods based on prior knowledge (such as dark channel prior). Although these methods are simple and effective, they often fail in thick fog conditions and are prone to color distortion and over-enhancement, making them difficult to apply to complex real-world scenarios. In recent years, the rapid development of deep learning technology has provided a new solution for image defogging. Defogging methods based on convolutional neural networks can learn nonlinear mappings through large-scale data and directly generate clear images from foggy images.

[0004] Therefore, how to combine the physical principles of the atmospheric scattering model with deep learning networks to develop defogging algorithms with physical interpretation has become an important research direction in the field of image defogging. This method not only improves the defogging effect, but also reduces error accumulation, improving the robustness and wide applicability of the model. SUMMARY

[0005] To address the deficiencies of the prior art, the application provides an image defogging method based on an atmospheric scattering model, which comprises: obtaining an original spatial fog image and inputting it into a trained image defogging model for processing to obtain a clear image after defogging.

[0006] The processing process of the image defogging model on the original spatial fog image comprises:

[0007] S1: using a feature extraction module to process the original spatial fog image to obtain a first feature spatial fog image, a second feature spatial fog image, and a third feature spatial fog image;

[0008] S2: inputting the feature spatial fog image into a haze extraction unit for processing to obtain a haze feature map;

[0009] S3: The feature decoding module processes the haze feature map according to the first feature space fog map, the second feature space fog map and the third feature space fog map to obtain a feature fusion map.

[0010] S4: The feature fusion map is subjected to inverse operation of the atmospheric scattering model to obtain a clear image after haze removal.

[0011] Preferably, the feature extraction module comprises a basic convolution block, a first MSG convolution module, a first down-sampling module, a second MSG convolution module, a second down-sampling module and a third MSG convolution module.

[0012] Further, the three MSG convolution modules have the same structure, and the data processing of the MSG convolution module comprises: after the input feature map is subjected to batch normalization processing, the input feature map is respectively input into two branches for processing; the first branch is composed of an adaptive convolution layer, an instance normalization layer, a ReLU layer and a deep convolution layer; the second branch is obtained by replacing the adaptive convolution layer of the first branch with a point convolution layer; the outputs of the two branches are fused, and then a point convolution layer is used to obtain the output of the MSG convolution module, i.e., the feature space fog map.

[0013] Preferably, the haze extraction unit comprises an atmospheric light extraction branch and a transmittance extraction branch; the third feature space fog map is input into the atmospheric light extraction branch and the transmittance extraction branch for processing to obtain the outputs of the two branches; pixel-level operations are performed on the outputs of the two branches to obtain the output of the haze extraction unit, i.e., the haze feature map.

[0014] Further, the atmospheric light extraction branch is composed of a global average pooling, a first ordinary convolution block, a first nonlinear activation function, a second ordinary convolution block and a second nonlinear activation function; the transmittance extraction branch is composed of a third ordinary convolution block, a fourth ordinary convolution block, a third nonlinear activation function, a fifth ordinary convolution block and a fourth nonlinear activation function.

[0015] Preferably, the feature decoding module comprises a first up-sampling module, a first SK Fusion module, a second up-sampling module, a second SK Fusion module, a third up-sampling module, a third SK Fusion module and a sixth ordinary convolution block; wherein the input of the first SK Fusion module is the third feature space fog map and the output of the first up-sampling module, the input of the second SK Fusion module is the second feature space fog map and the output of the second up-sampling module, and the input of the third SK Fusion module is the first feature space fog map and the output of the third up-sampling module.

[0016] Further, the three SK Fusion modules have the same structure, and the data processing process of the SK Fusion module includes: inputting two input feature maps into a global average pooling layer, a point convolution layer and a nonlinear function layer after adding the two input feature maps; inputting the output of the nonlinear function layer into two point convolution layers respectively, and processing the outputs of the two point convolution layers through a nonlinear function layer to obtain the output of the SK Fusion module.

[0017] Preferably, the loss function in the image defogging model training process is the sum of L1 loss, mean square error loss and structural similarity loss.

[0018] The beneficial effects of the present application are:

[0019] 1. The defogging method provided by the present application significantly improves the model defogging precision through three-stage image processing, achieves good defogging effect, effectively improves the image defogging precision, and is more generalized in cross-dataset testing.

[0020] 2. The MSG module used in the first part of the network model improves the performance and adaptability of the model, reduces information loss and maintains low computational cost through multi-scale feature extraction and information fusion.

[0021] 3. The SK Fusion module used in the third part of the network model provides an effective strategy to enhance the expression ability and robustness of the model through adaptive feature selection and multi-scale fusion, so that the model performs better in various visual tasks. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 Figure 1 is a total network structure diagram of the image defogging model in the present application;

[0023] Figure 2 Figure 2 is a structure diagram of the feature extraction module in the present application;

[0024] Figure 3 Figure 3 is a network structure diagram of the MSG module in the present application;

[0025] Figure 4 Figure 4 is a structure diagram of the haze extraction unit in the present application;

[0026] Figure 5 Figure 5 is a structure diagram of the feature decoding module in the present application;

[0027] Figure 6 Figure 6 is a network structure diagram of the SKFusion module in the present application;

[0028] Figure 7 Figure 7 is a comparison diagram of the image defogging effect in the present application;

[0029] Figure 8The comparison results of the application and the comparison method on different data sets are compared. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the application will be apparently and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative work belong to the protection scope of the application.

[0031] The application provides an image defogging method based on an atmospheric scattering model. The method is based on a learning method of the atmospheric scattering model, predicts features corresponding to atmospheric light and a transmission map, considers physical features of the two factors, and improves defogging accuracy. The method further improves the image defogging performance of the network by using a multi-scale gated convolution idea, and adaptively fuses different feature maps using an SK fusion block. The method comprises the following steps: obtaining an original spatial fog map and inputting the original spatial fog map into a trained image defogging model for processing to obtain a clear image after defogging.

[0032] As shown in Figure 1 , the processing process of the image defogging model on the original spatial fog map comprises the following steps:

[0033] S1: processing the original spatial fog map by using a feature extraction module to obtain a first feature spatial fog map, a second feature spatial fog map, and a third feature spatial fog map.

[0034] After the original spatial fog map is cropped into an image with a length and a width of 256 pixels, the image is input into the feature extraction module for processing. The image defogging model provided by the application comprises a feature extraction module, a haze extraction unit, a feature decoding module, and an atmospheric scattering model inverse operation unit. As shown in Figure 2 , the feature extraction module comprises, in sequence between an input layer and an output layer, a basic convolution block, a first MSG convolution module, a first down-sampling module, a second MSG convolution module, a second down-sampling module, and a third MSG convolution module.

[0035] The three MSG convolution modules have the same structure, and the outputs of the three MSG convolution modules are the first feature spatial fog map, the second feature spatial fog map, and the third feature spatial fog map, respectively. As shown in Figure 3 , the data processing of the MSG convolution module comprises the following steps: after the input feature map is subjected to batch normalization processing, the input feature map is input into two branches for processing; the first branch is composed of an adaptive convolution layer, an instance normalization layer, a ReLU layer, and a deep convolution layer; the second branch is obtained by replacing the adaptive convolution layer of the first branch with a point convolution layer; the outputs of the two branches are fused, and then a point convolution layer is used to obtain the output of the MSG convolution module, i.e., the feature spatial fog map.

[0036] S2: input the feature space fog map into the haze extraction unit for processing to obtain a haze feature map.

[0037] As shown in Figure 4 , the haze extraction unit includes an atmospheric light extraction branch and a transmittance extraction branch; the third feature space fog map is input into the atmospheric light extraction branch and the transmittance extraction branch respectively for processing to obtain two branch outputs; pixel-level operations are performed on the two branch outputs to obtain the output of the haze extraction unit, i.e., the haze feature map.

[0038] The atmospheric light extraction branch is composed of a global average pooling, a first ordinary convolution block, a first nonlinear activation function, a second ordinary convolution block and a second nonlinear activation function; the transmittance extraction branch is composed of a third ordinary convolution block, a fourth ordinary convolution block, a third nonlinear activation function, a fifth ordinary convolution block and a fourth nonlinear activation function.

[0039] S3: using a feature decoding module to process the haze feature map according to the first feature space fog map, the second feature space fog map and the third feature space fog map to obtain a feature fusion map.

[0040] As shown in Figure 5 , between the input layer and the output layer of the feature decoding module, there are in sequence: a first up-sampling module, a first SKFusion module, a second up-sampling module, a second SKFusion module, a third up-sampling module, a third SKFusion module and a sixth ordinary convolution block.

[0041] The three SKFusion modules have the same structure, wherein the input of the first SKFusion module is the third feature space fog map and the output of the first up-sampling module, the input of the second SKFusion module is the second feature space fog map and the output of the second up-sampling module, and the input of the third SKFusion module is the first feature space fog map and the output of the third up-sampling module.

[0042] As shown in Figure 6 , the data processing process of the SKFusion module includes: adding the two input feature maps and inputting them into a global average pooling layer, a point convolution layer and a nonlinear function layer; the output of the nonlinear function layer, i.e., the ReLU activation function, is input into two point convolution layers respectively, and the outputs of the two point convolution layers are further processed by a nonlinear function layer, i.e., a Softmax function, to obtain the output of the SKFusion module.

[0043] S4: performing inverse operation of the atmospheric scattering model on the feature fusion map to obtain a clear image after haze removal.

[0044] The atmospheric scattering model is represented as:

[0045] I(x) = t(x)J(x) + (1-t(x))A

[0046] where I(x) represents the pixel intensity at the x position of the foggy image, J(x) represents the pixel intensity at the x position of the clear image, t(x) represents the corresponding transmission map at the x position of the foggy image, and A represents the global atmospheric light.

[0047] Therefore, the inverse operation of the atmospheric scattering model is:

[0048]

[0049] The feature fusion image is substituted into the above formula to obtain a clear image after defogging.

[0050] During training of the image defogging model, the RESIDE public dataset can be used, and part of the ITS dataset in the RESIDE public dataset is randomly divided for training, and another part is used for verification. Part of the OTS dataset is randomly divided for training, and another part is used for verification. The loss function used in the training process of the present application is the sum of L1 loss, mean square error loss and structural similarity loss.

[0051] L1 loss: used to calculate the L1 loss between the model output and the target image.

[0052]

[0053] where n is the number of training samples, y i is the true value (target value), is the predicted value of the model.

[0054] Mean square error loss: measures the difference between the predicted value and the true value. The smaller the MSE value, the more accurate the prediction of the model.

[0055]

[0056] Structural similarity loss: the value of SSIM is between-1 and 1. The closer the value is to 1, the more similar the two images are.

[0057]

[0058] where μ x and μ y represent the pixel sample mean of x and y, and represent the variance of x and y, μ xy represents the covariance of x and y, and c1 and c2 represent two constants.

[0059] Evaluation of the present application:

[0060] like Figure 7 As shown, the fog map ( Figure 7 (Left) Input the trained model, and the network model generates a dehazed image. Figure 7 (Right). As can be seen, the dehazing method based on the atmospheric scattering model of this invention achieves excellent dehazing results. Furthermore, the MSG module used in this invention improves the model's performance and adaptability through multi-scale feature extraction and information fusion, reduces information loss, and maintains low computational costs. The SK Fusion module used provides an effective strategy to enhance the model's expressive power and robustness through adaptive feature selection and multi-scale fusion, making it perform better in various visual tasks.

[0061] The dehazing method of this invention is compared with two existing model structures, Dehazeformer and gUNet. For details of the Dehazeformer model, please refer to the literature Y. Song, Z. He, H. Qian, X. Du, Vision transformers for single image dehazing, IEEE Trans. Image Process. 32(2023) 1927–1941. For details of the gUNet model, please refer to the literature Y. Song, Y. Zhou, H. Qian, X. Du, Rethinking performance gains in image dehazing networks, 2022. The comparison results are shown below. Figure 8 The model structure, consisting of two networks, Dehazeformer and gUNet, utilizes separate transmission map estimation networks and atmospheric light estimation networks to obtain estimates of intermediate variables, and then uses the inverse transform of the atmospheric scattering model to obtain the dehazed image. The difference between this invention and the present invention is that the haze image is extracted separately. PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) are used to compare the dehazing accuracy of the model; higher values ​​for these two metrics indicate better dehazing accuracy. Figure 8 As shown, the results for different datasets (SOTS-indoor, SOTS-outdoor) demonstrate that the experimental performance of this invention is better, highlighting that the two-stage model structure effectively improves the model's accuracy.

[0062] In summary, this invention significantly improves the dehazing accuracy of the model through three stages of image processing, achieving a good dehazing effect and effectively enhancing the accuracy of image dehazing.

[0063] The above examples further illustrate the objects, technical solutions and advantages of the present application. It should be understood that the above examples are only preferred embodiments of the present application and are not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made to the present application within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image defogging method based on an atmospheric scattering model, characterized in that, The method comprises the following steps: An original spatial fog image is acquired and input into a trained image defogging model for processing to obtain a clear image after defogging; The processing of the original spatial fog image by the image defogging model comprises the following steps: S1: A feature extraction module is used to process the original spatial fog image to obtain a first feature spatial fog image, a second feature spatial fog image and a third feature spatial fog image; the feature extraction module comprises a basic convolution block, a first MSG convolution module, a first down-sampling module, a second MSG convolution module, a second down-sampling module and a third MSG convolution module; the three MSG convolution modules have the same structure, and the data processing of the MSG convolution module comprises the following steps: after the input feature map is subjected to batch normalization processing, the input feature map is input into two branches for processing; the first branch is composed of an adaptive convolution layer, an instance normalization layer, a ReLU layer and a deep convolution layer; the second branch is obtained by replacing the adaptive convolution layer of the first branch with a point convolution layer; the outputs of the two branches are fused, and then the fused outputs are subjected to a point convolution layer to obtain the output of the MSG convolution module, i.e., a feature spatial fog image; S2: The feature spatial fog image is input into a haze extraction unit for processing to obtain a haze feature map; S3: A feature decoding module is used to process the haze feature map according to the first feature spatial fog image, the second feature spatial fog image and the third feature spatial fog image to obtain a feature fusion image; S4: The feature fusion image is subjected to inverse operation of an atmospheric scattering model to obtain a clear image after defogging.

2. The image defogging method based on atmospheric scattering model according to claim 1, characterized in that, The haze extraction unit comprises an atmospheric light extraction branch and a transmittance extraction branch; The third feature spatial fog image is input into the atmospheric light extraction branch and the transmittance extraction branch for processing to obtain the outputs of the two branches; pixel-level operations are performed on the outputs of the two branches to obtain the output of the haze extraction unit, i.e., a haze feature map.

3. The image defogging method based on atmospheric scattering model according to claim 2, characterized in that, The atmospheric light extraction branch is composed of a global average pooling, a first ordinary convolution block, a first nonlinear activation function, a second ordinary convolution block and a second nonlinear activation function; and the transmittance extraction branch is composed of a third ordinary convolution block, a fourth ordinary convolution block, a third nonlinear activation function, a fifth ordinary convolution block and a fourth nonlinear activation function.

4. The image defogging method based on atmospheric scattering model according to claim 1, characterized in that, The feature decoding module comprises a first up-sampling module, a first SK Fusion module, a second up-sampling module, a second SK Fusion module, a third up-sampling module, a third SK Fusion module and a sixth ordinary convolution block; wherein the input of the first SK Fusion module is the third feature spatial fog image and the output of the first up-sampling module, the input of the second SK Fusion module is the second feature spatial fog image and the output of the second up-sampling module, and the input of the third SK Fusion module is the first feature spatial fog image and the output of the third up-sampling module.

5. The image defogging method based on atmospheric scattering model according to claim 4, characterized in that, The data processing process of the SK Fusion module includes: inputting two input feature maps into a global average pooling layer, a point convolution layer and a nonlinear function layer after adding the two input feature maps; inputting outputs of the nonlinear function layer into two point convolution layers respectively, and processing outputs of the two point convolution layers through a nonlinear function layer to obtain outputs of the SK Fusion module.

6. The image defogging method based on atmospheric scattering model according to claim 1, characterized in that, The loss function in the image defogging model training process is the sum of L1 loss, mean square error loss and structural similarity loss.

Images