Image defogging method based on hybrid large-scale convolution and attention fusion

Abstract

The application discloses a method for image defogging based on a mixed large-scale convolution attention mechanism, and the implementation steps are as follows: a large-scale convolution module for extracting multi-scale information of a feature image is constructed, a parallel attention module for extracting shared global information and position-related local information of an original feature is constructed, a mixed large-scale convolution network is built, the mixed large-scale convolution network is trained by using a generated training set, an image to be defogged is input into the trained mixed large-scale convolution network, and a haze-free image is output. The application overcomes the defects of the prior art, such as incomplete defogging of non-uniform fog images, insufficient learning of scene characteristics, and loss of image details and color distortion caused by neglecting multi-scale information of the image. The mixed large-scale convolution network can still achieve a high defogging effect in complex scene defogging.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and more specifically relates to an image dehazing method based on a hybrid parallel large-scale convolutional attention mechanism within the field of digital image processing technology. This invention can be used to sharpen haze noise present in natural images. Background Technology

[0002] Fog is a common weather phenomenon. Compared to clear weather, foggy or hazy weather contains impurities such as water vapor and dust particles in the air. Different impurities have different optical properties, causing light to be absorbed and scattered to varying degrees when it encounters them during transmission. This results in different light received by the camera compared to normal, leading to images with reduced clarity, blurriness, and low scene feature discrimination. Therefore, it is necessary to remove fog noise from images to obtain high-quality images. Currently, deep learning-based methods for removing fog noise from images dominate. In these methods, the neural network automatically learns a mapping model from foggy images to clear images from the training dataset. Using the input foggy image, it directly outputs the defogging image. However, existing image defogging techniques still suffer from color distortion, reduced saturation, or oversaturation when restoring images in low-resolution blurred images, large fog areas, and scenes with varying atmospheric light intensities.

[0003] Shaanxi University of Technology disclosed a single-image dehazing method based on an attention mechanism in its patent application "A Single Image Dehazing Method Based on Attention Mechanism" (application number CN 202310037938.X, publication number CN 115908203 A). The implementation steps of this method are: preprocessing the foggy image; then building a DRC-DehazeNet dehazing network; designing a loss function; training the network model; and finally testing the effectiveness of the DRC-DehazeNet network. While this method overcomes the shortcomings of manual feature extraction by adopting an end-to-end algorithm structure, enabling the network to directly output the dehazed image; and by designing a dense residual attention module, it ensures that the dehazing method can learn deeper feature information and mitigates the gradient vanishing phenomenon, thus improving the quality of the dehazed image. However, this method still has shortcomings: because dense fog caused by weather is unevenly distributed, various chaotic colors appear in the foggy image, forming color banding in the image. In this scenario, color cast may occur after dehazing.

[0004] Nanjing University of Information Science and Technology disclosed a method for image dehazing using a deep learning-based image dehazing network in its patent application "An Image Dehazing Network Based on Deep Learning" (application number CN 202310339386.8, publication number CN 116385294 A). The method involves extracting features from sample images to obtain their corresponding feature information, further obtaining an atmospheric scattering difference model, combining this model to obtain a haze-free parameter image, training an image dehazing network model using this model, and applying this model to dehaze target foggy images to obtain a haze-free parameter image. While this method achieves end-to-end image dehazing, avoiding the loss of image detail information and improving the realism of the dehazing effect, it still has shortcomings. Because the method uses insufficient effective information from small-scale convolutional kernels during feature extraction, it does not fully extract global and multi-scale information of the image. Therefore, when dehazing complex outdoor images, fog residue may still remain, and even color distortion and texture loss may occur. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the existing technology by proposing an image dehazing method based on hybrid large-scale convolution and attention fusion. This method aims to solve the problems of existing technologies failing to fully learn scene features at different fog concentrations and ignoring multi-scale image information, resulting in color cast and texture loss after dehazing.

[0006] The approach to achieving the objective of this invention is as follows: This invention constructs a parallel attention module that integrates channel attention, pixel attention, and cross-kernel attention. Channel attention modifies the original features by extracting global information, pixel attention extracts location-related information, and cross-kernel attention uses standard depthwise convolution and small receptive field kernels to capture local dependencies and compensate for grid problems. It also uses expanded depthwise convolution and large receptive field kernels to simulate long-distance dependencies. This module simultaneously extracts shared global information and location-related local information from the original features, thereby focusing on the most important dense fog features in the uneven fog noise of foggy images. This refines the image features, improves the model's expressiveness, and solves the color distortion problem caused by the inability of existing technologies to fully learn scene features when processing uneven fog images. Because the large-scale convolution module constructed in this invention introduces the SKFusion module and the CGAFusion module, it sequentially implements multi-scale parallel large kernel dilation convolution, dynamic feature fusion, and skip connection operations on features at the feature extraction level. The large-scale convolution module enables the feature image to obtain a larger receptive field. The SKFusion module uses channel attention to cascade and fuse feature information from low-level and high-level branches during fusion. The CGAFusion module enables the network to focus on features in different regions of each channel during fusion. The hybrid channel attention weights and spatial attention weights ensure information interaction and fusion, so that the dehazed image obtains a larger receptive field and multi-scale characteristics. While capturing large areas of blurred regions, it restores texture details and solves the problem of texture loss due to insufficient effective information at small and medium scales during feature extraction in the prior art.

[0007] The steps of this invention include the following:

[0008] Step 1: Construct a large-scale convolutional module for extracting multi-scale information from feature images;

[0009] Step 2: Construct a parallel attention module that shares global information and location-related local information for extracting the original features;

[0010] Step 3: Construct a hybrid large-scale convolutional network;

[0011] Step 4: Generate training and test sets;

[0012] Step 5: Train a hybrid large-scale convolutional network;

[0013] Step 6: Input the test set images into the trained hybrid large-scale convolutional network to output a fog-free image.

[0014] Compared with existing technologies, the present invention has the following advantages:

[0015] First, because the present invention constructs a parallel attention module, it focuses on the most important dense fog features in the non-uniform fog noise of foggy images, refines the image features, improves the model's expressiveness, and overcomes the color distortion problem caused by the inability of existing technologies to fully learn scene features when processing non-uniform fog images. This allows the present invention to avoid color distortion when processing hazy fog images, thereby improving the quality of dehazed images.

[0016] Secondly, the large-scale convolution module constructed in this invention introduces the SKFusion module and the CGAFusion module, which sequentially implements multi-scale parallel large kernel dilation convolution, dynamic feature fusion and skip connection operations on features at the feature extraction level. This allows the dehazed image to obtain a larger receptive field and multi-scale characteristics, and restores texture details while capturing large areas of blurred regions. This overcomes the problem of texture loss due to insufficient effective information at small and medium scales during feature extraction in the prior art. As a result, this invention can better consider local texture features in the dehazing stage, thereby improving the dehazed image to obtain a clearer effect. Attached Figure Description

[0017] Figure 1 This is a flowchart of the present invention;

[0018] Figure 2 A schematic diagram of the large-scale convolutional module structure constructed in this invention;

[0019] Figure 3 This is a schematic diagram of the parallel attention module structure built according to the present invention;

[0020] Figure 4 This is a schematic diagram illustrating the large-scale convolutional network structure constructed for this invention.

[0021] Figure 5 A schematic diagram of the SKFusion module structure in this invention;

[0022] Figure 6 A schematic diagram of the CGAFusion module in this invention;

[0023] Figure 7 The large-scale convolutional network model used in this invention is compared with other image dehazing methods on indoor and outdoor test sets. Detailed Implementation

[0024] The present invention will now be further described with reference to the accompanying drawings and embodiments.

[0025] Reference Figure 1 The implementation steps of the embodiments of the present invention will be further described below.

[0026] Step 1: Construct a large-scale convolutional module for extracting multi-scale information from feature images.

[0027] Reference Figure 2 The large-scale convolution module constructed in this invention will be further described below.

[0028] The large-scale convolution module consists of a Batch Norm layer, a first convolutional layer, a second convolutional layer, a multi-scale convolutional layer group, a large kernel selection layer group, and a multilayer perceptron layer group connected in series. The kernel sizes of the first and second convolutional layers are set to 1×1 and 5×5, respectively. The padding of the second convolutional layer is set to 2. The large-scale convolution module enables the feature image to acquire a large receptive field and multi-scale characteristics, effectively restoring texture details while capturing large areas of blurred regions.

[0029] The multi-scale convolutional layer group consists of a first convolutional layer, a second convolutional layer, and a third convolutional layer connected in parallel. The kernel sizes of the first to third convolutional layers are set to 7×7, 5×5, and 3×3, respectively, and the padding is set to 9, 6, and 3, respectively. The expansion rate of the kernels of the first to third convolutional layers is set to 3. Large kernel expansion convolution allows the feature image to obtain a larger receptive field, and parallel connection convolution can extract multi-scale features of the feature image.

[0030] The large kernel selection layer group is composed of a first branch and a second branch connected in parallel and then connected in series with an adder. The first branch is composed of a first convolutional layer, a second convolutional layer, an average pooling layer, a third convolutional layer, an activation layer, and a multiplier connected in series, wherein the output of the second convolutional layer is connected to the output of the activation layer. The convolutional kernels of the first to third convolutional layers are set to 5×5, 7×7, and 7×7, respectively; the padding of the second and third convolutional layers is set to 9 and 3, respectively; the dilation rate of the second convolutional layer is set to 3; and the activation layer uses the Sigmoid activation function.

[0031] The second branch consists of a first convolutional layer, an average pooling layer, a second convolutional layer, an activation layer, and a multiplier connected in series, with the output of the first convolutional layer connected to the output of the activation layer. The convolutional kernels of the first and second convolutional layers are set to 5×5 and 7×7, respectively. The padding of the second convolutional layer is set to 3, and the dilation rate is set to 3. The activation layer is implemented using the Sigmoid activation function. By using a large kernel selection layer group designed with convolution and the Sigmoid function, the two branches enable the network to dynamically select convolutions with different receptive field sizes for different fog concentration scenes, thereby obtaining rich contextual information, i.e., background information, and thus achieving the best defogging effect.

[0032] The multilayer perceptron consists of a first convolutional layer, an activation layer, a second convolutional layer, and an adder connected in series. The input of the large-scale convolutional module is connected to the output of the second convolutional layer. The kernel size of the first and second convolutional layers is set to 1×1. The activation layer is implemented using the GELU activation function. The multilayer perceptron can not only combine three different types of features, but also play a certain fitting role for denoised features.

[0033] Step 2: Construct a parallel attention module for extracting shared global information and location-related local information from the original features.

[0034] Reference Figure 3 The parallel attention module constructed in this invention will be further described below.

[0035] The parallel attention module is composed of pixel attention layer group, channel attention layer group, cross-large kernel attention layer group connected in parallel and then connected in series with multilayer perceptron layer. It extracts shared global information and location-related local information of the original features, and then focuses on the most important dense fog features in the non-uniform fog noise in the fog image, refines the image features, and improves the model's performance.

[0036] The pixel attention layer group consists of a parallel PF branch and a PA branch, which are then connected in series with a multiplier. The PF branch is composed of a first convolutional layer and a second convolutional layer connected in series to extract position-related information. The kernel sizes of the first and second convolutional layers are set to 1×1 and 3×3, respectively. The PA branch is composed of a convolutional layer and an activation layer connected in series to obtain position weights. The kernel size of the convolutional layer is set to 1×1. The activation layer is implemented using the Sigmoid activation function, which is used to extract global pixel selection features.

[0037] The channel attention layer group consists of an average pooling layer, a first convolutional layer, a first activation layer, a second convolutional layer, and a second activation layer, which are then concatenated with the input of the channel attention module via a multiplier. This modifies the original features by extracting global information. The kernel size of both the first and second convolutional layers is set to 1×1. The first activation layer uses the GELU activation function; the second activation layer uses the Sigmoid activation function.

[0038] The cross-large kernel attention layer group consists of a first convolutional layer, a first activation layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a second activation layer, and a multiplier connected in series. The input of the cross-large kernel attention is connected to the output of the second activation layer. The kernel sizes of the first to fourth convolutional layers are set to 1×1, 5×5, 7×7, and 1×1, respectively. The padding of the second and third convolutional layers is set to 2 and 9, respectively, and the dilation rate of the third convolutional layer is set to 3. The first activation layer is implemented using the GELU activation function. The second activation layer is implemented using the Sigmoid activation function. Standard depthwise convolution and small receptive field kernels are used to capture local dependencies and compensate for grid problems, while dilated depthwise convolution and large receptive field kernels are used to simulate long-distance dependencies.

[0039] The multilayer perceptron consists of a first convolutional layer, an activation layer, a second convolutional layer, and an adder connected in series. The input of the parallel attention module is connected to the output of the second convolutional layer. The kernel size of the first and second convolutional layers is set to 1×1. The activation layer is implemented using the GELU activation function. The multilayer perceptron can not only combine three different types of features, but also play a certain fitting role for denoised features.

[0040] Step 3: Build a hybrid large-scale convolutional network.

[0041] Reference Figure 4 The hybrid large-scale convolutional network constructed in this invention will be further described below.

[0042] The hybrid large-scale convolutional network is composed of a first convolutional layer, a first large-scale convolutional module, a first parallel attention module, a first downsampling layer, a second large-scale convolutional module, a second parallel attention module, a second downsampling layer, a third large-scale convolutional module, a third parallel attention module, a first upsampling layer, an SKFusion module, a fourth large-scale convolutional module, a fourth parallel attention module, a second upsampling layer, a CGAFusion module, a fifth large-scale convolutional module, a fifth parallel attention module, and a second convolutional layer, connected in series. The input of the CGAFusion module is connected to the input of the first parallel attention module... The output of the block and the output of the second upsampling layer are connected. The input of the SKFusion module is connected to the output of the second parallel attention module and the output of the first upsampling layer, respectively. The kernel size of the first and second convolutional layers is set to 3×3. The kernel size of the first and second downsampling layers is set to 3×3, and the stride is set to 2. The first and second upsampling layers are both composed of a pointwise convolutional layer and a PixelShuffle layer connected in series. The kernel size of the pointwise convolutional layer is set to 1×1, and the magnification factor of the PixelShuffle layer is set to 4 to improve the resolution and clarity of the image. The large-scale convolutional module, the parallel attention module, and the downsampling layer are responsible for extracting image feature information at different resolutions. The upsampling layer, the SKFusion module, and the CGAFusion module are responsible for concatenating and fusing the features before outputting them. Skip connections are used so that the final output feature map can fuse deep and shallow features.

[0043] Reference Figure 5 The SKFusion module introduced in this invention will be further described below.

[0044] The SKFusion module consists of an adder, a global average pooling layer, a multilayer perceptron layer group, an activation layer, a segmentation layer, and a weighting layer connected in series. The structure and parameters of the multilayer perceptron layer group are the same as those in the large-scale module. The activation layer is implemented using the Softmax activation function. The segmentation layer is implemented using the Split function, and channel attention is used to cascade and fuse the feature information of low-level and high-level branches.

[0045] The weighted layer is implemented by the following formula:

[0046] y = a1C 1×1 (x1)+a2x2

[0047] Where y represents the output of the weighted layer in the SKFusion module, a1 and a2 represent the outputs of the segmentation layer in the SKFusion module, x1 and x2 represent the output features mapped by the second parallel attention module and the first upsampling layer in the hybrid large-scale convolutional network, respectively, and C 1×1This indicates a convolution operation with a kernel size of 1×1, where x1 and x2 are weighted and fused to combine deep and shallow features.

[0048] Reference Figure 6 The CGAFusion module introduced in this invention will be further described below.

[0049] The CGAFusion module consists of an adder, an attention weight layer group, a channel shuffle layer, a convolutional layer, an activation layer, and a weighted layer connected in series. The kernel size of the convolutional layer is set to 7×7. The activation layer uses the Sigmoid activation function, which enables the network to focus on the features of different regions in each channel. The hybrid channel attention weight and spatial attention weight ensure information interaction and fusion, so that the dehazed image can obtain a larger receptive field and multi-scale characteristics, and retain the details and textures of the scene to the greatest extent.

[0050] The attention weight layer group consists of a spatial attention weight layer and a channel attention weight layer connected in parallel, followed by an adder and a channel concatenation layer connected in series. The output of the adder is connected to the input of the attention weight layer group. The spatial attention weight layer consists of a first pooling layer and a second pooling layer connected in parallel, followed by a convolutional layer connected in series. The first and second pooling layers are implemented using global average pooling and global maximum pooling, respectively. The convolutional layer kernel size is set to 7×7. The channel attention weight layer consists of a pooling layer, a first convolutional layer, an activation layer, and a second convolutional layer connected in series. The pooling layer is implemented using global average pooling. The convolutional layer kernel size is set to 1×1.

[0051] The weighted layer is implemented by the following formula:

[0052] F = l1·W + l2·(1-W) + l1 + l2

[0053] Where F represents the output of the weighted layer in the CGAFusion module, l1 and l2 represent the output features mapped by the first parallel attention module and the second upsampling layer in the hybrid large-scale convolutional network, respectively, and W represents the output of the activation layer in the CGAFusion module.

[0054] Step 4: The generated training and test sets.

[0055] The training set includes an indoor sample set and an outdoor sample set; the indoor sample set includes at least 13,000 image pairs, each pair containing a foggy, blurred image with uneven fog distribution and its corresponding clear image; the outdoor sample set includes at least 290,000 image pairs, each pair containing a foggy, blurred image with uniform fog distribution and its corresponding clear image.

[0056] In the embodiments of this invention, the sample set uses samples from the RESIDE dataset, which is one of the more standard and widely used datasets for image dehazing. It consists of five subsets: Indoor Training Set (ITS), Outdoor Training Set (OTS), Integrated Objective Test Set (SOTS), Real-World Task-Driven Test Set (RTTS), and Hybrid Subjective Test Set (HSTS); ITS and OTS are synthetic datasets, RTTS is a real-world dataset, and HSTS consists of synthetic and real hazy images.

[0057] In this embodiment of the invention, 13,000 image pairs from the RESIDE dataset ITS dataset are selected as the indoor training set, and 500 indoor image pairs from the SOTS test set are selected as the indoor test set; 290,000 image pairs from the RESIDE dataset OTS training set are selected as the outdoor training set, and 500 outdoor image pairs from the SOTS test set are selected as the outdoor test set.

[0058] Step 5: Train a hybrid large-scale convolutional network.

[0059] Step 5.1: The Adam optimizer is used during network training. The initial learning rate is 0.0001, the batch size is 4, and the total training time is 400 epochs.

[0060] Step 5.2: Input the training set into the hybrid large-scale convolutional network, use gradient descent to iteratively update the network parameters, calculate the loss value between the predicted fog-free image and the actual fog-free image output by the network, until the network's loss function converges, and obtain the trained large-scale convolutional network.

[0061] The loss function is as follows:

[0062]

[0063] Where MAE represents the loss value between the predicted image and the real image, n represents the total number of samples in the training set, f(xi) and yi represent the predicted image and the corresponding real image of the i-th sample in the training set, respectively, and |·| represents the absolute value operation.

[0064] In this embodiment of the invention, indoor and outdoor training sets are input into a hybrid large-scale convolutional network. The network parameters are iteratively updated using the gradient descent method. The loss value between the predicted fog-free image and the actual fog-free image output by the network is calculated until the network's loss function converges, thereby obtaining the trained indoor hybrid large-scale convolutional network and the trained outdoor hybrid large-scale convolutional network, respectively.

[0065] Step 6: Input the test set images into the trained hybrid large-scale convolutional network to output a fog-free image.

[0066] In this embodiment of the invention, the indoor test set and the outdoor test set are respectively input into the trained indoor hybrid large-scale convolutional network and the trained outdoor hybrid large-scale convolutional network to obtain indoor fog-free images and outdoor fog-free images, respectively.

[0067] The effects of this invention will be further illustrated below with simulation experiments:

[0068] 1. Simulation experimental conditions:

[0069] The hardware platform for the simulation experiment of this invention is as follows: the processor is an Intel Core i7-9700K CPU with a main frequency of 3.6GHz, the memory is 32GB, the dedicated graphics card is an NVIDIA GeForce RTX 2080SUPER with 8GB of graphics memory.

[0070] The software platform for the simulation experiment of this invention is as follows: Software environment: Operating system is 64-bit Windows 10, programming platform is PyCharm 2021.2.2, programming environment is Python 3.8 + PyTorch 1.10.0.

[0071] The dataset used in the simulation experiment of this invention is selected in the same way as the dataset used in step 4 of the embodiment of this invention.

[0072] 2. Simulation content and result analysis:

[0073] The simulation experiment of this invention uses the method of this invention and six existing technologies (DCP, DehazeNet, AOD-Net, MADN, FFA-Net, DehazeFormer) to dehaze all test samples in the indoor and outdoor sections of the RESIDE dataset, respectively. The results are as follows: Figure 7 As shown.

[0074] In the simulation experiments of this invention, the prior art DCP refers to:

[0075] He Kaiming et al. proposed a method based on dark channel prior in their paper “Single image haze removal using dark channel prior” ([J] / / IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12): 2341-2353.).

[0076] In the simulation experiments of this invention, the prior art DehazeNet refers to:

[0077] Cai B, Xu XM, Jia K, et al. proposed a method based on a trainable end-to-end system in their paper “An end-to-end system for singleimage haze removal” ([J].IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198.).

[0078] In the simulation experiments of this invention, the prior art AOD-Net refers to:

[0079] Li BY, Peng XL, Wang ZY et al. proposed a method based on an integrated dehazing network in their paper "Aod-net: All-in-one dehazing network" ([C] / / Proceedings of the IEEE International Conference on Computer Vision.2017:4770-4778.).

[0080] In the simulation experiments of this invention, the prior art MADN refers to:

[0081] T. Jia, J. Li, L. Zhuo and G. Li et al. proposed an adaptive image dehazing method based on meta-attention in their paper "Effective Meta-Attention Dehazing Networks for Vision-Based Outdoor Industrial Systems" ([J] IEEE Transactions on Industrial Informatics, 2022, 1511-1520.).

[0082] In the simulation experiments of this invention, the prior art FFA-Net refers to:

[0083] Qin X, Wang Z, Bai Y, et al. proposed an end-to-end feature fusion attention network method in their paper “FFA-Net: Feature fusion attention network for single image dehazing” ([C] / / Proceedings of the AAAI Conference on Artificial Intelligence.2020,34(07):11908 11915.).

[0084] In the simulation experiments of this invention, the prior art DehazeFormer refers to:

[0085] Yuda Song, Zhuqing He, Hui Qian, Xin Du, et al. proposed a dehazing method based on the Transformer structure in their paper "VisionTransformers for Single Image Dehazing" ([J]. IEEE Transactions on Image Processing, 2023: 1927–1941.).

[0086] To verify the effectiveness of the simulation experiments of this invention, the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) evaluation metrics were calculated using the reconstruction results and real images to assess the model's performance. A higher PSNR value indicates better image quality after dehazing. Structural similarity measures the similarity between the restored haze-free image and the real haze-free image in terms of brightness, contrast, and structure. A higher SSIM value indicates that the restored dehaze image is closer to the real haze-free image, with less distortion and better dehazing effect. Table 1 shows the PSNR and SSIM of the haze-free and haze-containing images obtained using the seven methods employed in the simulation experiments of this invention.

[0087] Table 1. Test results of different algorithms on synthetic foggy datasets.

[0088]

[0089] Table 1 shows the evaluation of the dehazing effect of the synthesized hazy images on the SOTS dataset. The dehazing effect was evaluated on 500 pairs of indoor images and 500 pairs of outdoor images. PSNR and SSIM were calculated using the dehazed and clear images, and the results were compared with six different dehazing algorithms: DCP, DehazeNet, AOD-Net, MADN, FFA-Net, and DehazeFormer. The best index in Table 1 is shown in bold.

[0090] As can be seen from Table 1, the PSNR and SSIM of the method of the present invention are significantly improved compared with the prior art.

[0091] The following is combined Figure 7 The simulation diagrams further illustrate the effects of the present invention.

[0092] Figure 7 These are the dehazing results of different algorithms on a synthetic dataset, among which, Figure 7 (a) Figure 7(i) represent a foggy image and a true fog-free image, respectively. Figure 7 (b) Figure 7 (c) Figure 7 (d) Figure 7 (e) Figure 7 (f) Figure 7 (g) Figure 7 (h) represents the fog-free images generated by DCP, DehazeNet, AOD-Net, MADN, FFA-Net, DehazeFormer, and the method of this invention, respectively. The top two rows of each sub-figure represent the dehazing results of indoor fog images, and the bottom two rows represent the dehazing results of outdoor fog images.

[0093] Depend on Figure 7 (b) Analysis shows that both indoor and outdoor images processed by the DCP algorithm exhibit severe color distortion, with significant problems in both the indoor wall areas and the outdoor sky areas. Observation Figure 7 (c) and Figure 7 (d) It can be observed that the DehazeNet and AOD-Net algorithms alleviate the color distortion problem to some extent compared to the DCP algorithm. However, in processing indoor images, dehazing is not thorough, leaving residual fog, and the colors of outdoor images are generally darker than those of real, fog-free images. Figure 7 (e) It can be observed that MADN achieves good dehazing results indoors, but significant fog residue still exists in outdoor images. The FFA-Net algorithm, by adding an attention mechanism to the network, produces clearer and more natural dehazed images. Figure 7 (f) It can be seen that the FFA-Net algorithm achieved good dehazing results on both the first image and the outdoor image, and the colors of the dehazed image were close to those of the real fog-free image. However, for the second foggy image, fog residue still remained in the glass area, resulting in a blurred image. Figure 7 (g) and Figure 7 As shown in (h), both the DehazeFormer algorithm and the algorithm proposed in this paper overcome the problems of color distortion and incomplete dehazing. Compared with other algorithms, the image details and textures are clearer, achieving a good dehazing effect. However, compared with the DehazeFormer algorithm, the algorithm in this paper is closer to the real haze-free image in terms of color and preserves details more effectively.

[0094] The simulation experiments above show that the hybrid large-scale convolutional network constructed by the method of this invention can extract multi-scale features of images, obtain a large receptive field, and focus on the most important dense fog features in the uneven fog noise of fog images. While capturing large areas of blurred regions, it restores texture details and avoids color distortion. It solves the problem of incomplete dehazing of uneven fog images in existing technical solutions, which leads to loss of image texture, color distortion and fog residue. It is a very practical image dehazing method.

Claims

1. An image dehazing method based on hybrid large-scale convolution and attention fusion, characterized in that, The dehazing method comprises the following steps: a large-scale convolutional module for extracting multi-scale information from feature images, and a parallel attention module for extracting shared global information and location-related local information from the original features. Step 1: Construct a large-scale convolutional module for extracting multi-scale information from feature images; Step 2: Construct a parallel attention module that shares global information and location-related local information for extracting the original features; Step 3: Construct a hybrid large-scale convolutional network containing large-scale convolutional modules and parallel attention modules; Step 4: Use the generated training set to train a hybrid large-scale convolutional network; Step 5: Input the foggy image into the trained hybrid large-scale convolutional network and output the fog-free image; The parallel attention module is composed of pixel attention layer group, channel attention layer group, cross large kernel attention layer group connected in parallel and then connected in series with multilayer perceptron layer. The pixel attention layer group consists of parallel PF branches and PA branches, followed by a multiplier connected in series; wherein, the PF branch consists of a first convolutional layer and a second convolutional layer connected in series; the kernel size of the first and second convolutional layers is set to 1. 1,3 3; The PA branch is composed of convolutional layers and activation layers connected in series, with the kernel size of the convolutional layers set to 1.

1. The activation layer is implemented using the Sigmoid activation function; The channel attention layer group consists of an average pooling layer, a first convolutional layer, a first activation layer, a second convolutional layer, and a second activation layer, which are then concatenated with the input of the channel attention module via a multiplier; the kernel size of the first and second convolutional layers is set to 1. 1; The first activation layer is implemented using the GELU activation function; the second activation layer is implemented using the Sigmoid activation function; The cross-large kernel attention layer group consists of a first convolutional layer, a first activation layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a second activation layer, and a multiplier, connected in series. The input of the cross-large kernel attention layer is connected to the output of the second activation layer. The kernel size of the first to fourth convolutional layers is set to 1. 1,5 5, 7 7,1 1; Set the padding of the second and third convolutional layers to 2 and 9 respectively, and the dilation rate of the third convolutional layer to 3; The first activation layer is implemented using the GELU activation function; The second activation layer is implemented using the Sigmoid activation function; The multilayer perceptron consists of a first convolutional layer, an activation layer, a second convolutional layer, and an adder connected in series. The input of the parallel attention module is connected to the output of the second convolutional layer. The kernel size of the first and second convolutional layers is set to 1×1. The activation layer is implemented using the GELU activation function.

2. The image dehazing method based on hybrid large-scale convolution and attention fusion according to claim 1, characterized in that, The large-scale convolution module described in step 1 is composed of a Batch Norm layer, a first convolutional layer, a second convolutional layer, a multi-scale convolutional layer group, a large kernel selection layer group, and a multilayer perceptron layer group connected in series. The kernel sizes of the first and second convolutional layers are set to 1×1 and 5×5, respectively. The padding of the second convolutional layer is set to 2. The multi-scale convolutional layer group is composed of a first convolutional layer, a second convolutional layer, and a third convolutional layer connected in parallel. The kernel sizes of the first to third convolutional layers are set to 7×7, 5×5, and 3×3, respectively, and the padding numbers are set to 9, 6, and 3, respectively. The expansion rate of the kernels of the first to third convolutional layers is set to 3. The large kernel selection layer group is composed of a first branch and a second branch connected in parallel and then connected in series with an adder; the first branch is composed of a first convolutional layer, a second convolutional layer, an average pooling layer, a third convolutional layer, an activation layer, and a multiplier connected in series, wherein the output of the second convolutional layer is connected to the output of the activation layer; the convolutional kernels of the first to third convolutional layers are set to 5×5, 7×7, and 7×7, respectively; The padding numbers for the second and third convolutional layers are set to 9 and 3, respectively. The dilation rate of the second convolutional layer is set to 3; The activation layer uses the Sigmoid activation function; The second branch consists of a first convolutional layer, an average pooling layer, a second convolutional layer, an activation layer, and a multiplier connected in series, wherein the output of the first convolutional layer is connected to the output of the activation layer; the convolutional kernels of the first and second convolutional layers are set to 5×5 and 7×7, respectively; the padding of the second convolutional layer is set to 3, and the dilation rate is set to 3. The activation layer is implemented using the Sigmoid activation function; The multilayer perceptron consists of a first convolutional layer, an activation layer, a second convolutional layer, and an adder connected in series. The input of the large-scale convolutional module is connected to the output of the second convolutional layer. The kernel size of the first and second convolutional layers is set to 1×1. The activation layer is implemented using the GELU activation function.

3. The image dehazing method based on hybrid large-scale convolution and attention fusion according to claim 2, characterized in that, The hybrid large-scale convolutional network described in step 3 consists of a first convolutional layer, a first large-scale convolutional module, a first parallel attention module, a first downsampling layer, a second large-scale convolutional module, a second parallel attention module, a second downsampling layer, a third large-scale convolutional module, a third parallel attention module, a first upsampling layer, an SKFusion module, a fourth large-scale convolutional module, a fourth parallel attention module, a second upsampling layer, a CGAFusion module, a fifth large-scale convolutional module, a fifth parallel attention module, and a second convolutional layer connected in series. The input of the CGAFusion module is connected to the output of the first parallel attention module and the output of the second upsampling layer, respectively; the input of the SKFusion module is connected to the output of the second parallel attention module and the output of the first upsampling layer, respectively. The kernel size of the first and second convolutional layers is set to 3×3; the kernel size of the first and second downsampling layers is set to 3.

3. The stride is set to 2 for all layers; the first and second upsampling layers are both composed of a pointwise convolutional layer and a PixelShuffle layer connected in series, and the kernel size of the pointwise convolutional layer is set to 1.

1. Set the magnification of the PixelShuffle layer to 4.

4. The image dehazing method based on hybrid large-scale convolution and attention fusion according to claim 3, characterized in that, The SKFusion module consists of an adder, a global average pooling layer, a multilayer perceptron layer group, an activation layer, a segmentation layer, and a weighting layer connected in series. The structure and parameters of the multilayer perceptron layer group are the same as those in the large-scale module. The activation layer is implemented using the Softmax activation function, and the segmentation layer is implemented using the Split function.

5. The image dehazing method based on hybrid large-scale convolution and attention fusion according to claim 4, characterized in that, The weighted layer is implemented by the following formula: ； Where y represents the output of the weighted layer in the SKFusion module. These represent the outputs of the segmentation layer in the SKFusion module. , These represent the output features mapped from the second parallel attention module and the first upsampling layer in a hybrid large-scale convolutional network, respectively. This indicates a convolution operation with a kernel size of 1×1.

6. The image dehazing method based on hybrid large-scale convolution and attention fusion according to claim 5, characterized in that, The CGAFusion module consists of an adder, an attention weight layer group, a channel shuffle layer, a convolutional layer, an activation layer, and a weighted layer connected in series; the kernel size of the convolutional layer is set to 7×7; the activation layer is implemented using the Sigmoid activation function. The attention weight layer group consists of a spatial attention weight layer and a channel attention weight layer connected in parallel, followed by an adder and a channel concatenation layer connected in series. The output of the adder is connected to the input of the attention weight layer group. The spatial attention weight layer consists of a first pooling layer and a second pooling layer connected in parallel, followed by a convolutional layer connected in series. The first and second pooling layers are implemented using global average pooling and global maximum pooling, respectively. The convolutional layer kernel size is set to 7×7. The channel attention weight layer consists of a pooling layer, a first convolutional layer, an activation layer, and a second convolutional layer connected in series. The pooling layer is implemented using global average pooling. The convolutional layer kernel size is set to 1×1. The weighted layer is implemented by the following formula: ； in, This represents the output of the weighted layer in the CGAFusion module. These represent the output features after mapping by the first parallel attention module and the output features after mapping by the second upsampling layer in a hybrid large-scale convolutional network, respectively. This represents the output of the activation layer in the CGAFusion module.

7. The image dehazing method based on hybrid large-scale convolution and attention fusion according to claim 6, characterized in that, The training set mentioned in step 4 includes an indoor sample set and an outdoor sample set; the indoor sample set includes at least 13,000 image pairs, each pair containing a foggy, blurred image with uneven fog distribution and its corresponding clear image; the outdoor sample set includes at least 290,000 image pairs, each pair containing a foggy, blurred image with uniform fog distribution and its corresponding clear image.

8. The image dehazing method based on hybrid large-scale convolution and attention fusion according to claim 7, wherein training the hybrid large-scale convolutional network in step 4 refers to inputting the training set into the hybrid large-scale convolutional network, using gradient descent to iteratively update the network parameters, calculating the loss value between the predicted haze-free image and the actual haze-free image output by the network, until the network's loss function converges, thereby obtaining the trained large-scale convolutional network.

9. The image dehazing method based on hybrid large-scale convolution and attention fusion according to claim 8, wherein the loss function is as follows: ； in, MAE represents the loss between the predicted image and the ground truth image, n represents the total number of samples in the training set, and f(x) i ) and y i Let represent the predicted image and the corresponding real image of the i-th sample in the training set, respectively. This indicates the absolute value operation.

Images