Method for Removing Fog from Images of Land, Sea and Air Battlefields Based on Parameter Modeling and Gradient Guidance

Abstract

The present invention relates to a method for removing haze from land, sea and air battlefield images based on parametric modeling and gradient guidance. Natural battlefield images and remote sensing battlefield images are collected, and fog synthesis strategies are used to generate natural battlefield haze-removing datasets and remote sensing battlefield haze-removing datasets, and an underwater image enhancement dataset is obtained. A battlefield image haze-removing network is constructed, in which the fog concentration estimation network combines the dark channel prior and the Swin Transformer model to generate fog concentration parameters, the medium transmission estimation network estimates the medium transmission parameters based on the encoder-decoder model, and the ambient light estimation network collaborates with the encoder and local minimum filtering to estimate the ambient light parameters. The estimated parameters are substituted into the imaging model, and the imaging model is inversely solved to generate the battlefield haze-removing result. A gradient guidance module is used to further enhance the image details, and clear and detailed battlefield images are obtained. The present invention integrates the complementary advantages of physical model methods and deep learning methods, effectively improves the visual effect of battlefield fog images, and can achieve multi-scene haze removal in a single framework, which is convenient for popularization and use.

Description

Technical Field

[0001] The present invention belongs to a method for removing haze from battlefield images, and relates to a method for removing haze from land, sea and air battlefield images based on parameter modeling and gradient guidance. Background Art

[0002] Currently, image dehazing methods are mainly divided into three categories: the first category is the method based on physical models, the second category is the non-physical model method, and the third category is the method based on deep learning. The method based on physical models requires accurate estimation of model parameters and is difficult to apply to challenging foggy scenarios. The non-physical model method highly depends on the observable information in the image and cannot achieve good results for the dehazing task. With the booming development of deep learning technology, the dehazing method based on deep learning has become the mainstream. The literature "B. Li, Y. Gou, J. Z. Liu, H. Zhu, J. T. Zhou, and X. Peng. Zero-Shot Image Dehazing. IEEE Transactions on Image Processing, vol. 29, pp. 8457-8466, Aug. 2020." discloses a zero-shot learning dehazing network, which uses an encoder-decoder model to estimate medium transmission and ambient light, and inversely solves the atmospheric scattering imaging model to remove the fogging effect in natural images. Some remote sensing image dehazing methods and underwater image dehazing methods also incorporate the atmospheric scattering imaging model into the deep network and achieve good visual effects. However, the above methods can only improve the fogging phenomenon in a single scene. In addition, the literature "Q. Guo, H.-M. Hu, and B. Li. Haze and Thin Cloud Removal Using Elliptical Boundary Prior for Remote Sensing Image. IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 11, pp. 9124-9137, Nov. 2019." proposes that the atmospheric scattering imaging model ignores the correlation between medium transmission and fog concentration and is difficult to improve the contrast of remote sensing images. How to reasonably combine the complementary advantages of the imaging model and the deep network still needs to be explored, and designing a single framework to remove the fogging phenomenon in multi-source sensor images is an even more difficult problem to be solved. Summary of the Invention

[0003] Technical Problems to be Solved

[0004] In order to avoid the deficiencies of the prior art, the present invention proposes a method for removing haze from land, sea and air battlefield images based on parameter modeling and gradient guidance, overcoming the deficiency that the existing image dehazing methods are difficult to apply to the scenarios captured by multi-source sensors.

[0005] This method uses the dark channel prior theory to obtain the dark channel of the remote sensing battlefield image, and the Swin Transformer module is used to learn the features of dark elements in the form of non-overlapping local windows to generate the fog concentration parameter. Secondly, an encoder-decoder network is established to generate the medium transmission parameters of multi-source images, namely the medium transmission parameters of remote sensing battlefield images, natural battlefield images, and underwater images, and the guided filter is used to retain the edge information of the parameters. At the same time, the interference of high-brightness pixels on the environmental light estimation is excluded through local minimum filtering, and more attention is focused on the fogged area by combining the attention mechanism and deformable convolution to reasonably estimate the environmental light parameter. Then, the relevant parameters of the remote sensing battlefield image, namely the fog concentration parameter, medium transmission parameter, and environmental light parameter, are input into the remote sensing fog image imaging model, and the relevant parameters of the natural battlefield image and underwater image, namely the medium transmission parameter and environmental light parameter, are input into the atmospheric scattering imaging model, and the imaging model is inversely solved to generate multi-source defogged images. Finally, a gradient guidance module is constructed to adaptively enhance the image texture details, and a white balance algorithm is used to compensate for the color attenuation of underwater images, realizing the removal of the fogging phenomenon of multi-source battlefield images in a single framework.

[0006] Technical solution

[0007] A method for removing fog from land, sea, and air battlefield images based on parameter modeling and gradient guidance, characterized by the following steps:

[0008] Step 1: Collect natural battlefield image and remote sensing battlefield image samples, construct a natural battlefield defogging data benchmark and a remote sensing battlefield defogging data benchmark using a fog synthesis strategy based on the imaging model, collect existing underwater image defogging data benchmarks, and divide the above data benchmarks into remote sensing battlefield defogging, natural battlefield defogging, and underwater image defogging training sample sets and test sample sets at a ratio of 10:1;

[0009] Step 2: Perform data augmentation on the training sample set, crop and mirror flip according to the scale to generate training samples with a dimension of H×W×3;

[0010] Step 3: Input the remote sensing battlefield defogging training samples into the fog concentration estimation network (S-Net), and use the dark channel prior theory to obtain the dark channel of the training samples:

[0011]

[0012] where J dark is the dark channel, c is any one of the RGB color channels, and Ω(x) is the local area centered on pixel x;

[0013] Step 4: Input the dark channel in Step 3 into the Swin Transformer model to generate the fog concentration parameter s r ;

[0014] Step 5: Input the remote sensing battlefield defogging training samples, natural battlefield defogging training samples, and underwater image defogging training samples into the medium transmission estimation network (T-Net), and use the encoder-decoder model to generate the medium transmission parameter t with dimensions of H×W×1 r , t h , t u , with ReLU and Sigmoid as the activation functions;

[0015] Step 6: Refine the medium transmission parameter in Step 5 using guided filtering to retain edge information;

[0016] Step 7: Input the remote sensing battlefield defogging training samples, natural battlefield defogging training samples, and underwater image defogging training samples into the ambient light estimation network (A-Net) to generate the ambient light parameter A with dimensions of H×W×3 r , A h , A u ;

[0017] Input the fog concentration parameter s r , the medium transmission parameter t r , and the ambient light parameter A r in Steps 4, 6, and 7 into the remote sensing foggy image formation model:

[0018] I r (x) = J r (x)t r (x) + A r s r (x)

[0019] where I r is the remote sensing foggy image, J r is the clear remote sensing image, and r represents the remote sensing defogging task;

[0020] Step 9: Inversely solve the remote sensing foggy image formation model in Step 8 to obtain the clear remote sensing battlefield image:

[0021]

[0022] where t0 is the critical value, set to 0.1, to avoid the brightness imbalance of the clear remote sensing image;

[0023] Step 10: Input the medium transmission parameter t h and the ambient light parameter A h in Steps 6 and 7 into the atmospheric scattering imaging model:

[0024] I h (x) = J h (x)t h (x) + A h (1 - th (x))

[0025] Among them, h represents the natural image defogging task;

[0026] Step 11: Inversely solve the atmospheric scattering imaging model in Step 10 to obtain a clear natural battlefield image:

[0027]

[0028] Step 12: Input the medium transmission parameter t u and the ambient light parameter A u into the atmospheric scattering imaging model and inversely solve to obtain a clear underwater image;

[0029] Step 13: Use the white balance algorithm to compensate for the color attenuation of the underwater image;

[0030] Step 14: Input the clear remote sensing battlefield image, clear natural battlefield image, and clear underwater image into the gradient guidance module to enhance the image texture details;

[0031] Step 15: Train the land-sea-air battlefield image defogging network to obtain a trained land-sea-air battlefield image defogging network, where the optimizer is selected as Adam, and the loss function uses a hybrid loss function composed of the L1 loss function L l1 、gradient loss function L G 、MS-SSIM loss function L M :

[0032] L = λ1L l1 + λ2L G + λ3L M

[0033] Among them, λ1, λ2, and λ3 are trade-off parameters, set to 1, 1, and 2. The calculation formula of the L1 loss function is as follows:

[0034]

[0035] Among them, J is the true fog-free image, and J' is the defogged image output by the network. The calculation formula of the gradient loss function is as follows:

[0036] L G = E G'～Q(r),G～Q(g) ||G' - G||1

[0037] Among them, E(·) is the mathematical expectation, G' is the gradient map of J', G is the gradient map of J, Q(r) is the distribution of G', and Q(g) is the distribution of G. The calculation formula of the MS-SSIM loss function is as follows:

[0038] L M= 1 - MS - SSIM(J', J)

[0039] Use the obtained land - sea - air battlefield image de - fogging network to process remote - sensing battlefield fog images, natural battlefield fog images, and underwater fog images to obtain de - fogged images.

[0040] The environmental light estimation network in step 7 specifically includes the following steps:

[0041] Step 7 - 1: Use local minimum filtering on the input image to reduce the interference of high - light pixels;

[0042] Step 7 - 2: Use a 3×3 convolution on the image in step 7 - 1 to generate shallow features;

[0043] Step 7 - 3: Use a channel attention mechanism and a pixel attention mechanism on the shallow features in step 7 - 2 to generate attention features, treating the foggy areas and non - foggy areas in the image unequally;

[0044] Step 7 - 4: After performing a combination of 4 times of max - pooling, deformable convolution, batch normalization, and ReLU activation on the attention features in step 7 - 3, use global max - pooling and a fully - connected layer to generate the environmental light value;

[0045] Step 7 - 5: Use a dimension expansion operation on the environmental light value in step 7 - 4 to generate environmental light parameters with dimensions of H×W×3.

[0046] The gradient guidance module in step 14 specifically includes the following steps:

[0047] Step 14 - 1: Use the sobel operator on the input image to generate a gradient map G I ;

[0048] Step 14 - 2: Multiply the gradient map in step 14 - 1 with the network - output de - fogged image pixel - by - pixel to generate detail features f d :

[0049] f d = J'×G I

[0050] Step 14 - 3: Add the detail features in step 14 - 2 to the network - output de - fogged image element - by - element to generate a detail - clear de - fogged image J c :

[0051] J c = J'+f d .

[0052] Beneficial effects

[0053] A method for removing haze from land, sea and air battlefield images based on parametric modeling and gradient guidance proposed by the present invention solves the problem that a single model is difficult to remove the fogging phenomenon in multiple battlefield scenarios. The method mainly includes the following steps: collecting natural battlefield images and remote sensing battlefield images, generating a natural battlefield haze removal dataset and a remote sensing battlefield haze removal dataset by using a fog synthesis strategy, obtaining an underwater image enhancement dataset, and performing data augmentation on the three datasets to generate a training set and a test set; constructing a battlefield image haze removal network, in which the fog concentration estimation network combines the dark channel prior and the Swin Transformer model to generate fog concentration parameters, the medium transmission estimation network estimates the medium transmission parameters based on an encoder-decoder model, and the ambient light estimation network collaborates with an encoder and local minimum filtering to estimate the ambient light parameters. By substituting the estimated parameters into the imaging model and inversely solving the imaging model, a battlefield haze removal result is generated. A gradient guidance module is used to further enhance the image details, obtaining clear and detail-rich battlefield images. The present invention integrates the complementary advantages of physical model methods and deep learning methods, effectively improves the visual effect of battlefield fog images, and can realize multi-scene haze removal in a single framework, which is convenient for popularization and use.

[0054] The beneficial effects of the present invention are as follows: This method combines the dark channel prior theory and the Swin Transformer model to reasonably estimate the specific parameters of the remote sensing fog image imaging model, namely the fog concentration parameters; collaborates with an encoder and local minimum filtering to reduce the interference caused by high-brightness pixels such as the fire generated by an explosion, etc., and reasonably estimates the ambient light parameters; uses an encoder-decoder model to estimate the medium transmission parameters, and uses guided filtering to refine the edge details of the medium transmission image; when using the imaging model to realize multi-scene battlefield image haze removal, it also considers the problem of image detail blur, constructs a gradient guidance module, and completes image detail enhancement, providing high-quality data for subsequent battlefield vision tasks. Description of the Drawings

[0055] Figure 1 is a flowchart of the method for removing haze from land, sea and air battlefield images based on parametric modeling and gradient guidance of the present invention.

[0056] Figure 2 is a schematic diagram of the network for removing haze from land, sea and air battlefield images with parametric modeling and gradient guidance designed by the present invention.

[0057] Figure 3 is a model diagram of the fog concentration estimation network designed by the present invention.

[0058] Figure 4 is a model diagram of the medium transmission estimation network designed by the present invention.

[0059] Figure 5 is a model diagram of the ambient light estimation network designed by the present invention.

[0060] Figure 6 is a model diagram of the gradient guidance module designed by the present invention.

[0061] Figure 7 It is the defogging result map of the land-sea-air battlefield image generated by the method of the present invention. Specific implementation manners

[0062] The present invention will be further described below in conjunction with embodiments and drawings:

[0063] Reference Figure 1-7 The method for defogging land-sea-air battlefield images based on parametric modeling and gradient guidance of the present invention specifically includes the following steps:

[0064] Figure 1 It is the flowchart of the method of the present invention, including the following processes: obtaining remote sensing battlefield and natural battlefield samples, collecting underwater image defogging benchmarks, generating remote sensing battlefield and natural battlefield defogging benchmarks by using a fog synthesis strategy, performing data augmentation on the three data benchmarks to generate a training set and a test set; constructing a land-sea-air battlefield image defogging network, and training the land-sea-air battlefield image defogging network by using the training set; inputting the test data into the trained land-sea-air battlefield image defogging network for testing to obtain multi-scene defogging results. The method specifically includes the following steps:

[0065] Step 1: Collect natural battlefield image and remote sensing battlefield image samples, calculate the depth of the natural battlefield image by using a depth estimation algorithm, perform random sampling within a fixed range on the medium transmission and ambient light of the remote sensing battlefield image through prior statistics, and construct a natural battlefield defogging data benchmark and a remote sensing battlefield defogging data benchmark by using a fog synthesis strategy based on an imaging model, collect existing underwater image defogging data benchmarks, and divide the above data benchmarks into a training sample set and a test sample set at a ratio of 10:1;

[0066] Step 2: Perform data augmentation on the training sample set, such as cropping by scale and mirror flipping, expand the number of samples to 8 times the original, and generate training samples with a dimension of 256×256×3;

[0067] Step 3: Reference Figure 2 The method for defogging land-sea-air battlefield images based on parametric modeling and gradient guidance includes 3 sub-networks. Input the remote sensing battlefield defogging training samples into the fog concentration estimation network (S-Net), that is Figure 3 , and use the dark channel prior theory to obtain the dark channel of the training samples:

[0068]

[0069] where J dark is the dark channel, c is any one of the RGB color channels, and Ω(x) is the local area centered on pixel x;

[0070] Step 4: Input the dark channel in Step 3 into the Swin Transformer model, which is based on the U-Net architecture and captures both global dependencies and local contexts to generate the fog concentration parameter s r ;

[0071] Step 5: Refer to Figure 4 , and input the remote sensing battlefield defogging training samples, natural battlefield defogging training samples, and underwater image defogging training samples into the medium transmission estimation network (T-Net). Use the encoder-decoder model to generate the medium transmission parameters t r 、t h 、t u with ReLU and Sigmoid as the activation functions;

[0072] Step 6: Use guided filtering to refine the medium transmission parameters in Step 5 and retain the edge information;

[0073] Step 7: Input the remote sensing battlefield defogging training samples, natural battlefield defogging training samples, and underwater image defogging training samples into the ambient light estimation network (A-Net) to generate the ambient light parameters A r 、A h 、A u ;

[0074] Refer to Figure 5 , and the ambient light estimation network specifically includes the following steps:

[0075] Step 7-1: Use local minimum filtering on the input image to reduce the interference of high-brightness pixels;

[0076] Step 7-2: Use 3×3 convolution on the image in Step 7-1 to generate shallow features;

[0077] Step 7-3: Use the channel attention mechanism and pixel attention mechanism on the shallow features in Step 7-2 to generate attention features, treating the foggy areas and non-foggy areas in the image unequally;

[0078] Step 7-4: After performing a combination of 4 times of max pooling, deformable convolution, batch normalization, and ReLU activation on the attention features in Step 7-3, use global max pooling and a fully connected layer to generate the ambient light value.

[0079] Step 8: Input the fog concentration parameter s r 、the medium transmission parameter t r 、and the ambient light parameter A r in Steps 4, 6, and 7 into the remote sensing foggy image imaging model:

[0080] I r (x) = Jr (x)t r (x) + A r s r (x) (2)

[0081] Among them, I r is the remote sensing fog image, J r is the clear remote sensing image, and r represents the remote sensing de - fogging task;

[0082] Step 9: Inversely solve the remote sensing fog image imaging model in Step 8 to obtain a clear remote sensing battlefield image:

[0083]

[0084] Among them, t0 is the critical value, set to 0.1, to avoid the brightness imbalance of the clear remote sensing image;

[0085] Step 10: Input the medium transmission parameter t h and the ambient light parameter A h in Steps 6 and 7 into the atmospheric scattering imaging model:

[0086] I h (x) = J h (x)t h (x) + A h (1 - t h (x)) (4)

[0087] Among them, h represents the natural image de - fogging task;

[0088] Step 11: Inversely solve the atmospheric scattering imaging model in Step 10 to obtain a clear natural battlefield image:

[0089]

[0090] Step 12: Input the medium transmission parameter t u and the ambient light parameter A u in Steps 6 and 7 into the atmospheric scattering imaging model, and inversely solve to obtain a clear underwater image;

[0091] Step 13: Use the white - balance algorithm to compensate for the color attenuation of the underwater image;

[0092] Step 14: Input the clear remote sensing battlefield image, the clear natural battlefield image, and the clear underwater image into the gradient - guided module to enhance the image texture details;

[0093] Reference Figure 6 , the gradient - guided module specifically includes the following steps:

[0094] Step 14 - 1: Use the sobel operator on the input image to generate a gradient map G I;

[0095] Step 14-2: Multiply the gradient map described in Step 14-1 with the network output dehazed image pixel by pixel to generate the detail feature f d :

[0096] f d = J' × G I (6)

[0097] Step 14-3: Add the detail feature described in Step 14-2 to the network output dehazed image element by element to generate the detail-clear dehazed image J c :

[0098] J c = J' + f d (7)

[0099] Step 15: Train the land-sea-air battlefield image dehazing network described in Steps 3 - 14. The optimizer is Adam, and the loss function is a hybrid loss function composed of the L1 loss function L l1 , the gradient loss function L G , and the MS-SSIM loss function L M :

[0100] L = λ1L l1 + λ2L G + λ3L M (8)

[0101] where λ1, λ2, and λ3 are trade-off parameters, set to 1, 1, and 2. The calculation formula of the L1 loss function is as follows:

[0102]

[0103] where J is the real haze-free image and J' is the dehazed image output by the network. The calculation formula of the gradient loss function is as follows:

[0104] L G = E G'～Q(r),G～Q(g) ||G' - G||1 (10)

[0105] where E(·) is the mathematical expectation, G' is the gradient map of J', G is the gradient map of J, Q(r) is the distribution of G', and Q(g) is the distribution of G. The calculation formula of the MS-SSIM loss function is as follows:

[0106] L M = 1 - MS-SSIM(J', J) (11)

[0107] Step 16: Process the remote sensing battlefield fog image, natural battlefield fog image, and underwater fog image using the land-sea-air battlefield image dehazing network to obtain the dehazed images.

[0108] Reference Figure 7 , the method of the present invention has the advantages of good defogging effect and clear details, and can remove the fogging phenomenon in land, sea and air scenes with a unified framework, providing high-quality data for subsequent battlefield vision tasks. The effect of the method of the present invention is further illustrated by the following simulation experiments.

[0109] 1. Simulation conditions.

[0110] The method of the present invention is simulated using Anaconda software on a computer with an Intel Core i7-9750H CPU, 32G of memory, an Nvidia RTX3090 graphics card, and a WINDOWS10 operating system.

[0111] 2. Simulation conditions.

[0112] The data used in the simulation are self-built remote sensing battlefield defogging datasets, self-built natural battlefield defogging datasets, and publicly available underwater EUVP datasets.

[0113] To prove the effectiveness of the proposed method, SDCP, IDeRs, HTM, EVPM, and FCTF-Net are selected as comparison algorithms on the remote sensing battlefield dehazing dataset. Among them, SDCP was proposed in the literature "J. Li, Q. Hu, and M. Ai. Haze and Thin Cloud Removal via Sphere Model Improved Dark Channel Prior. IEEE Geoscience and Remote Sensing Letters, vol. 16, no. 3, pp. 472-476, Mar. 2019."; IDeRs was proposed in the literature "L. Xu, D. Zhao, Y. Yan, S. Kwong, J. Chen, and L.-Y. Duan. IDeRs: Iterative Dehazing Method for Single Remote Sensing Image. Information Sciences, vol. 489, pp. 50-62, Jul. 2019."; HTM was proposed in the literature "Q. Liu, X. Gao, L. He, and W. Lu. Haze Removal for a Single Visible Remote Sensing Image. Signal Processing, vol. 137, pp. 33-43, Aug. 2017."; EVPM was proposed in the literature "J. Han, S. Zhang, N. Fan, and Z. Ye. Local Patchwise Minimal and Maximal Values Prior for Single Optical Remote Sensing Image Dehazing. Information Sciences, vol. 606, pp. 173-193, Aug. 2022."; FCTF-Net was proposed in the literature "Y. Li and X. Chen. A Coarse-to-Fine Two-Stage Attentive Network for Haze Removal of Remote Sensing Images. IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 10, pp. 1751-1755, Oct. 2021." On the natural battlefield dehazing dataset, DCP, GDCP, Haze-lines, ZID, and TCN are selected as comparison algorithms.Among them, DCP was proposed in the literature "K. He, J. Sun, and X. Tang. Single Image Haze Removal Using Dark Channel Prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2341-2353, Dec. 2011."; GDCP was proposed in the literature "Y.-T. Peng, K. Cao, and P. C. Cosman. Generalization of the Dark Channel Prior for Single Image Restoration. IEEE Transactions on Image Processing, vol. 27, no. 6, pp. 2856-2868, Jun. 2018."; Haze-lines was proposed in the literature "D. Berman, T. Treibitz, and S. Avidan. Single Image Dehazing Using Haze-Lines. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 3, pp. 720-734, Mar. 2020."; ZID was proposed in the literature "B. Li, Y. Gou, J. Z. Liu, H. Zhu, J. T. Zhou, and X. Peng. Zero-Shot Image Dehazing. IEEE Transactions on Image Processing, vol. 29, pp. 8457-8466, Aug. 2020."; TCN was proposed in the literature "J. Shin, H. Park, and J. Paik. Region-Based Dehazing via Dual-Supervised Triple-Convolutional Network. IEEE Transactions on Multimedia, vol. 24, pp. 245-260, Jan. 2021." MMLE, IBLA, TACL, Water-Net, and LCNet were selected as comparison algorithms on the EUVP dataset.Among them, MMLE was proposed in the literature "W.Zhang, P.Zhuang, H.-H.Sun, G.Li, S.Kwong, and C.Li. Underwater Image Enhancement via Minimal Color Loss and Locally Adaptive Contrast Enhancement. IEEE Transactions on Image Processing, vol.31, pp.3997-4010, Jun. 2022."; IBLA was proposed in the literature "Y.-T.Peng and P.C.Cosman. Underwater Image Restoration Based on Image Blurriness and Light Absorption. IEEE Transactions on Image Processing, vol.26, no.4, pp.1579-1594, Apr. 2017."; TACL was proposed in the literature "R.Liu, Z.Jiang, S.Yang, and X.Fan. Twin Adversarial Contrastive Learning for Underwater Image Enhancement and Beyond. IEEE Transactions on Image Processing, vol.31, pp.4922-4936, Jul. 2022."; Water-Net was proposed in the literature "C.Li, C.Guo, W.Ren, R.Cong, J.Hou, S.Kwong, and D.Tao. An Underwater Image Enhancement Benchmark Dataset and Beyond. IEEE Transactions on Image Processing, vol.29, pp.4376-4389, Feb. 2020."; LCNet was proposed in the literature "N.Jiang, W.Chen, Y.Lin, T.Zhao, and C.-W.Lin. Underwater Image Enhancement with Lightweight Cascaded Network. IEEE Transactions on Multimedia, vol.24, pp.4301-4313, Sep. 2021."TIO is the result obtained by the method of the present invention, and PSNR and SSIM are image quality evaluation metrics. The comparison results are shown in Table 1.

[0114] As can be seen from Table 1, on the three datasets, the defogging performance of the present invention is significantly better than other comparison algorithms. The higher PSNR and SSIM evaluation results indicate that the images processed by TIO are closer to fog-free images.

[0115] Table 1

[0116]

Claims

1. A method for removing haze from land, sea and air battlefield images based on parametric modeling and gradient guidance, characterized in that The steps are as follows: Step 1: Collect natural battlefield images and remote sensing battlefield image samples, construct a natural battlefield defogging data benchmark and a remote sensing battlefield defogging data benchmark using a fog synthesis strategy based on an imaging model, collect existing underwater image defogging data benchmarks, and divide the above data benchmarks into a remote sensing battlefield defogging, natural battlefield defogging, and underwater image defogging training sample set and a test sample set at a ratio of 10:1; Step 2: Perform data augmentation on the training sample set, crop and mirror flip by scale to generate training samples with dimensions of H×W×3; Step 3: Input the remote sensing battlefield defogging training samples into the fog concentration estimation network S-Net, and use the dark channel prior theory to obtain the dark channels of the training samples; Among them, J dark is the dark channel, c is any one of the RGB color channels, and Ω(x) is the local area centered on pixel x; Step 4: Input the dark channel in Step 3 into the Swin Transformer model to generate the fog concentration parameter s r ; Step 5: Input the remote sensing battlefield dehazing training samples, natural battlefield dehazing training samples, and underwater image dehazing training samples into the medium transmission estimation network T-Net, and use the encoder-decoder model to generate medium transmission parameters t of dimension H×W×1 respectively r , t h , t u , and the activation functions are ReLU and Sigmoid; Step 6: Use guided filtering to refine the medium transmission parameters in Step 5, retaining edge information; Step 7: Input the remote sensing battlefield defogging training samples, natural battlefield defogging training samples, and underwater image defogging training samples into the ambient light estimation network A-Net to generate ambient light parameters A with dimensions of H×W×3 respectively r , A h , A u ; Input the fog concentration parameter s in Step 4, Step 6, and Step 7 r , the medium transmission parameter t r , and the ambient light parameter A r into the remote sensing fog image formation model: I r J(x) = r J(x)t r J(x)+A r s r J(x) Among them, I r is a remote sensing fog image, J r is a clear remote sensing image, and r represents the remote sensing defogging task; Step 9: Inversely solve the remote sensing fog image imaging model in Step 8 to obtain a clear remote sensing battlefield image; Among them, t0 is the critical value, set to 0.1, to avoid brightness imbalance of the clear remote sensing image; Step 10: Input the medium transmission parameter t h and the ambient light parameter A h into the atmospheric scattering imaging model: I h J(x) = h J(x)t h + A h (1 - t h (x)) Among them, h represents the natural image defogging task; Step 11: Inversely solve the atmospheric scattering imaging model in Step 10 to obtain a clear natural battlefield image; Step 12: Input the medium transmission parameter t u and the ambient light parameter A u into the atmospheric scattering imaging model, and inversely solve to obtain a clear underwater image; Step 13: Use a white balance algorithm to compensate for color attenuation of underwater images; Step 14: Input the clear remote sensing battlefield image, clear natural battlefield image, and clear underwater image into the gradient guidance module to enhance image texture details; Step 15: Train the dehazing network for land, sea and air battlefield images to obtain a trained dehazing network for land, sea and air battlefield images. The optimizer is selected as Adam, and the loss function is a hybrid loss function composed of the L1 loss function L l1 , the gradient loss function L G , and the MS-SSIM loss function L M : L = λ1L l1 + λ2L G + λ3L M Among them, λ1, λ2, and λ3 are trade-off parameters, set to 1, 1, and 2; the calculation formula of the L1 loss function is as follows: Among them, J is the true fog-free image, and J' is the defogged image output by the network; the calculation formula of the gradient loss function is as follows: L G = E G'～Q(r),G～Q(g) ||G'-G||1 Among them, E(·) is the mathematical expectation, G' is the gradient map of J', G is the gradient map of J, Q(r) is the distribution of G', and Q(g) is the distribution of G; the calculation formula of the MS-SSIM loss function is as follows: L M = 1 - MS - SSIM(J', J) Use the obtained land-sea-air battlefield image defogging network to process remote sensing battlefield fog images, natural battlefield fog images, and underwater fog images to obtain defogged images.

2. The method for dehazing land, sea and air battlefield images based on parametric modeling and gradient guidance according to claim 1, characterized in that: The specific steps of the environmental light estimation network in Step 7 are as follows: Step 7-1: Use local minimum filtering on the input image to reduce the interference of high-brightness pixels; Step 7-2: Use 3×3 convolution on the image in Step 7-1 to generate shallow features; Step 7-3: Use a channel attention mechanism and a pixel attention mechanism on the shallow features in Step 7-2 to generate attention features, treating the fog area and the fog-free area in the image unequally; Step 7-4: After performing a combination of 4 maximum pooling, deformable convolution, batch normalization, and ReLU activation on the attention features in Step 7-3, use global maximum pooling and a fully connected layer to generate the environmental light value; Step 7-5: Use a dimension expansion operation on the environmental light value in Step 7-4 to generate environmental light parameters with dimensions of H×W×3.

3. The method for dehazing land, sea and air battlefield images based on parametric modeling and gradient guidance according to claim 1, characterized in that: The trade-off parameters λ1, λ2, and λ3 are set to 1, 1, and 2.

4. The method for dehazing land-sea-air battlefield images based on parametric modeling and gradient guidance according to claim 1, wherein: The specific steps of the gradient guidance module in Step 14 are as follows: Step 14-1: Use the sobel operator on the input image to generate the gradient map G I ; Step 14-2: Multiply the gradient map described in Step 14-1 with the network output dehazed image pixel by pixel to generate the detail feature f d : f d = J' × G I Step 14-3: Add the detailed features described in Step 14-2 to the network output dehazed image element by element to generate a dehazed image J with clear details c : J c = J'+ f d .

Images