A dehazing network based on dynamic routing and three-dimensional fusion

Abstract

A kind of dynamic routing and three-dimensional fusion-based defogging network, the network includes double branch network Two-branch Network, dynamic routing feature module DRFM and clear image reconstruction module CIRM;The input end of double branch network Two-branch Network is used to input foggy image I, the output of double branch network Two-branch Network is connected with the input of dynamic routing feature module DRFM, the output of dynamic routing feature module DRFM is connected with the input of clear image reconstruction module CIRM, and the output of clear image reconstruction module CIRM is used to output defogging image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a dehazing network based on dynamic routing and 3D fusion. Background Technology

[0002] Remote sensing imagery plays a vital role in fields such as geological exploration, climate change research, and environmental protection. However, due to the influence of tiny particles in weather phenomena such as haze, remote sensing images often suffer from severe image degradation, including blurred details and color distortion, limiting their application in advanced computer vision tasks such as change detection and semantic segmentation. Therefore, how to effectively remove haze from remote sensing images and restore clear images has attracted widespread attention from researchers.

[0003] Early dehazing methods for both natural and remote sensing images relied on introducing prior assumptions and knowledge to recover clear images. For example, Li et al.'s paper, "Aod-net: All-in-one dehazing network," designed AOD-Net to generate restored images by re-adapting the physical scattering model. However, in complex hazy scenes, the estimated priors often proved inaccurate, resulting in far from ideal dehazed images. In recent years, with the development of deep learning, researchers have proposed numerous end-to-end haze removal methods, utilizing Convolutional Neural Networks (CNNs) and Transformers to learn the mapping relationship between hazy and clear images. The frameworks of these models mainly include U-Net and Full Resolution Convolutional Neural Networks (F-Nets). For example, Dong et al.'s paper, "Multi-Scale Boosted Dehazing Network with DenseFeature Fusion," designed a dense feature fusion module to address the problem of spatial information preservation during U-Net dehazing. Zhang et al.'s paper, "Dense haze removal based on dynamic collaborative inference learning for remote sensing images," proposes a dynamic collaborative inference learning (DCIL) framework to enhance low-level texture features. Qin et al.'s paper, "FFA-Net: Feature Fusion Attention Network for Single Image Dehazing," proposes a non-downsampling feature fusion attention network to focus on more important information. Most methods employing these two architectures have achieved significant improvements in dehazing performance, but two main problems remain:

[0004] (1) How to effectively utilize the features extracted from U-Net and F-Net architectures to achieve clear image reconstruction: The U-Net model is widely used for dehazing remote sensing images. However, the downsampling in the encoding stage of the U-Net model leads to the loss of high-frequency information, resulting in poor recovery of small-scale objects under hazy conditions. In contrast, the F-Net haze removal model without downsampling can effectively preserve high-frequency feature information, but these models have limited receptive fields, resulting in residual haze. Recently, several dehazing models based on sequential and parallel dual-network architectures have been proposed, which have significantly improved dehazing performance. The sequential dual-network model uses the output of the first network as the input of the second network, resulting in the loss of information that the second network cannot fully reconstruct. The parallel dual-network model simply concatenates the features extracted by U-Net and F-Net to achieve clear image reconstruction, but the dehazing effect is still insufficient. These dual-network models fail to effectively utilize the features of different network architectures for optimal image reconstruction and introduce a large number of parameters and information redundancy by superimposing existing network structures.

[0005] (2) Existing dehazing models lack spatial and channel information interaction between multiple features: Existing dehazing networks mainly utilize element-wise addition and channel concatenation for feature fusion. In recent years, various feature fusion methods utilizing learnable parameters and attention mechanisms have been proposed to enhance the performance of dehazing models. These methods assign weights to space, channels, and the feature as a whole, effectively constraining the fusion process and promoting more robust and accurate feature fusion. However, these methods have a coarse granularity and lack fine-grained interaction between spatial and channel information, resulting in insufficient representational power of the fused features. This reduces the dehazing model's ability to recover texture details and colors. Therefore, in the feature fusion process, in-depth research into the intrinsic spatial and channel correlations between multiple features, achieving fine-grained weight allocation, and enhancing the representation of fused features to produce excellent texture detail and color recovery remains an unsolved problem. Summary of the Invention

[0006] The purpose of this invention is to solve the problems of high-frequency information loss caused by downsampling of U-shaped networks and limited receptive field of full-resolution convolutional networks when using existing remote sensing image dehazing techniques. In addition, this invention can also solve the technical problem that existing dehazing models lack fine-grained interaction between spatial information and channel information during the fusion process, resulting in insufficient feature representation ability after fusion. The invention proposes a dehazing network based on dynamic routing and three-dimensional fusion.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] A dehazing network based on dynamic routing and 3D fusion includes a two-branch network, a dynamic routing feature module (DRFM), and a clear image reconstruction module (CIRM).

[0009] The input of the Two-branch Network is used to input the foggy image I. The output of the Two-branch Network is connected to the input of the Dynamic Routing Feature Module (DRFM). The output of the DRFM is connected to the input of the Clear Image Reconstruction Module (CIRM). The output of the CIRM is used to output the defoggy image.

[0010] like Figure 1 As shown, the Two-branch Network includes a U-shaped sub-network U-Net, a full-resolution convolutional sub-network F-Net, and a 3D perception feature fusion module;

[0011] The U-shaped sub-network U-Net is as follows: the input of the first encoding layer is used to input the foggy image I; the output of the first encoding layer is connected to the input of the second encoding layer; the output of the second encoding layer is connected to the input of the third encoding layer; the output of the third encoding layer is connected to the input of the residual feature extraction module; the output of the residual feature extraction module and the output of the third encoding layer are connected together to the input of the first 3D perception feature fusion module; the output of the first 3D perception feature fusion module is connected to the input of the first decoding layer; the output of the first decoding layer and the output of the second encoding layer are connected together to the input of the second 3D perception feature fusion module; the output of the second 3D perception feature fusion module is connected to the input of the second decoding layer; the output of the second decoding layer and the output of the first encoding layer are connected together to the input of the third 3D perception feature fusion module; the output feature of the third 3D perception feature fusion module is denoted as feature F1.

[0012] The full-resolution convolutional subnetwork F-Net is specifically as follows:

[0013] The input of the channel changing module is used to input the foggy image I. The output of the channel changing module is connected to the input of the residual feature extraction module. The output feature of the residual feature extraction module is feature F2.

[0014] The specific operation of the channel changing module is as follows:

[0015] The foggy image I is input to the first ReflectionPad2d layer of the channel alteration module. The output of the first ReflectionPad2d layer of the channel alteration module is input to the first 7×7 Conv layer of the channel alteration module. The output of the first 7×7 Conv layer of the channel alteration module is input to the first InstanceNorm layer of the channel alteration module. The output of the first InstanceNorm layer of the channel alteration module is input to the first ReLU layer of the channel alteration module.

[0016] Features F1 and F2 are fused to obtain feature F3; then the Dynamic Routing Feature Module (DRFM) scores features F1, F2, and F3 and selects the most suitable one for reconstructing a clear image to calculate the loss.

[0017] The process of training a network using four loss constraints, which are as follows:

[0018] (1) SmoothL1 loss, the specific formula is:

[0019]

[0020]

[0021] G(χ j χ represents the pixel value at position i in the input image, which is then processed by the dehazing network parameters. P represents the number of pixels in the image. j Z represents the value of pixel j in a foggy image. j This represents the value of pixel j in the ground truth (GT) field.

[0022] (2) Adversarial loss, the specific definition of adversarial loss is as follows:

[0023]

[0024] D represents the generator, and G represents the discriminator. D(G(x)) represents the probability of classifying a dehazed image as a clear image, and m represents the total number of images.

[0025] (3) Perceptual loss, which calculates the L2 norm of the VGG16 pre-trained model at different layers for the ground truth (GT) and fog-free images from the network. The perceptual loss is defined as follows:

[0026]

[0027] Where η i This represents the size of the feature map obtained from the i-th layer of the pre-trained VGG16 model. P represents the number of feature layers used for perceptual loss. X and Y represent the foggy image and the clear, fog-free image, respectively.

[0028] (4) Multi-scale structural similarity loss (MS-SSIM): To make the structural similarity between the dehazed image and the clear image closer, multi-scale structural similarity loss (MS-SSIM) is introduced, which is expressed as follows:

[0029]

[0030] in and η represents the mean of the dehazed image and the ground truth (GT), respectively. ij η is the covariance between the dehazed image and the ground plane. i and η j Let C be the standard deviation of the dehazed image from the ground truth (GT), and C be a constant. The overall network loss function is expressed as:

[0031] L=θ1L smoothL1 +θ2L adv +θ3L perc +θ4L ms-ssim

[0032] Where θ1, θ2, θ3, and θ4 are hyperparameters of each loss function.

[0033] The feature extraction operation of the coding layer is as follows:

[0034] The feature input is fed into the first ReflectionPad2d layer of the coding layer. The output of the first ReflectionPad2d layer of the coding layer is fed into the first 7×7 Conv layer of the coding layer. The output of the first 7×7 Conv layer of the coding layer is fed into the first InstanceNorm layer of the coding layer. The output of the first InstanceNorm layer of the coding layer is fed into the first ReLU layer of the coding layer. The output of the first ReLU layer of the coding layer is used to obtain the first layer of encoded features.

[0035] The first layer of encoded features is input into the second 3×3 Conv layer of the encoding layer. The output of the second 3×3 Conv layer of the encoding layer is input into the second InstanceNorm layer of the encoding layer. The output of the second InstanceNorm layer of the encoding layer is input into the second ReLU layer of the encoding layer. The output of the second ReLU layer of the encoding layer is the second layer of encoded features.

[0036] The second layer of encoded features is input to the third 3×3 Conv layer of the encoding layer. The output of the third 3×3 Conv layer of the encoding layer is input to the third InstanceNorm layer of the encoding layer. The output of the third InstanceNorm layer of the encoding layer is input to the third ReLU layer of the encoding layer. The output of the third ReLU layer of the encoding layer is the third layer of encoded features.

[0037] The residual feature extraction module consists of 6 cascaded residual blocks, and the feature extraction of the residual blocks is as follows:

[0038] The input features are fed into the first 3×3 Conv layer of the residual block. The output of the first 3×3 Conv layer of the residual block is fed into the first ReLU layer of the residual block. The output of the first ReLU layer of the residual block and the input features are fed into the first 3D perception feature fusion module of the residual block. The output features of the first 3D perception feature fusion module of the residual block are fed into the second 3×3 Conv layer of the residual block. The output of the second 3×3 Conv layer of the residual block and the input features are fed into the second 3D perception feature fusion module of the residual block. The output of the second 3D perception feature fusion module of the residual block is the output feature of the residual block.

[0039] The image restoration operation of the decoding layer is as follows:

[0040] The output of the residual feature extraction module and the output of the third layer encoding are input together to the first 3D perception feature fusion module of the decoding layer. The output of the first 3D perception feature fusion module of the decoding layer is input to the first Dconv layer of the decoding layer. The output of the first Dconv layer of the decoding layer is input to the first InstanceNorm layer of the decoding layer. The output of the first InstanceNorm layer of the decoding layer is input to the first ReLU layer of the decoding layer. The output of the first ReLU layer of the decoding layer is the first layer of decoded features.

[0041] The first layer of decoded features and the output of the second encoding layer are input together to the second 3D perception feature fusion module of the decoding layer. The output of the second 3D perception feature fusion module of the decoding layer is input to the second Dconv layer of the decoding layer. The output of the second Dconv layer of the decoding layer is input to the second InstanceNorm layer of the decoding layer. The output of the second InstanceNorm layer of the decoding layer is input to the second ReLU layer of the decoding layer. The output of the second ReLU layer of the decoding layer is the second layer of decoded features.

[0042] The second layer of decoding features and the output of the first encoding layer are fed together into the third 3D perception feature fusion module of the decoding layer. The output of the third 3D perception feature fusion module of the decoding layer is the feature F1 extracted by the U-shaped sub-network.

[0043] The three-dimensional perception feature fusion module, as shown below Figure 2 As shown, the process is divided into two stages: coarse fusion and fine fusion. The specific operation of coarse fusion is as follows:

[0044] Feature map and feature map The first Flatten operation of coarse fusion yields corresponding output feature maps. These two feature maps are then subjected to the first Concat operation followed by the first adaptive global average pooling operation to obtain a single feature vector. This feature vector is then subjected to the first 1×1 Conv operation, the first ReLU operation, and the first Sigmoid operation of coarse fusion to obtain vectors C1 and C2. Vectors C1 and C2 are then compared with the feature maps... and feature map The feature map is obtained through the first multiplication of the coarse fusion. and feature map Feature map and feature map The coarse fusion output feature X is obtained by the first element-wise addition of the coarse fusion. CF .

[0045] The specific steps for fine-tuning are as follows:

[0046] Coarse fusion output feature X CF The channel feature vector CI is obtained by sequentially performing the first adaptive global average pooling operation, the first 1×1 Conv operation, and the first ReLU operation of fine fusion. The channel feature vector CI then undergoes two 1×1 Conv operations in fine fusion to obtain channel feature vectors CI1 and CI2. Finally, channel feature vectors CI1 and CI2 are broadcast through the fine fusion mechanism to obtain feature maps C. r1 and feature map C r2 ;

[0047] Coarse fusion output feature X CF Spatial feature map PI is obtained by sequentially performing a fourth 1×1 Conv operation and a second ReLU operation in fine fusion. Spatial feature map PI is then subjected to two more 1×1 Conv operations in fine fusion to obtain spatial feature maps PI1 and PI2. Spatial feature maps PI1 and PI2 are then subjected to a broadcast mechanism in fine fusion to obtain feature map PI. r1 and feature map PI r2 ;

[0048] Channel feature vector C r1 Spatial feature map PI r1 The first pixel-by-pixel addition after fine fusion yields the pixel-level three-dimensional information matrix V1;

[0049] Channel feature vector C r2 Spatial feature map PI r2 The second pixel-by-pixel addition after fine fusion yields the pixel-level three-dimensional information matrix V2;

[0050] The pixel-level 3D information matrices V1 and V2 undergo a first concat operation in fine fusion to obtain a feature map. This feature map then undergoes a first softmax operation in fine fusion to obtain a pixel-level 3D weight matrix. and pixel-level 3D weight matrix

[0051] Fusion features and pixel-level 3D weight matrix The first pixel-by-pixel multiplication operation after fine fusion yields the fused features. Weighted feature map;

[0052] Fusion features and pixel-level 3D weight matrix The fused features are obtained through a second pixel-wise multiplication operation after fine fusion. Weighted feature map;

[0053] Fusion features Weighted feature map and fused feature The weighted feature maps undergo a third pixel-wise addition process after fine fusion to obtain the fused feature V. out .

[0054] Dynamic routing feature modules such as Figure 3 As shown, the details are as follows:

[0055] Feature maps F1, F2, and F3 are first detached. The results of the detached operation are then input to the corresponding feature scoring units FSU1, FSU2, and FSU3, respectively. The features input to the feature scoring units first pass through the feature extraction module, which consists of a global average pooling layer, a 1×1 Conv layer, and a ReLU layer connected in sequence. The feature extraction module outputs three feature vectors corresponding to feature maps F1, F2, and F3, respectively. These feature vectors are then input to the feature extraction module, which consists of a 1×1 Conv layer and a Sigmoid layer connected in sequence. The feature extraction module outputs feature scores corresponding to feature maps F1, F2, and F3, respectively. The three feature scores are then subjected to a Concat operation and a Gumbel-Softmax function to obtain a discrete gated sequence.

[0056] The clear image reconstruction module specifically includes:

[0057] The feature map is input to the first 3×3 Conv layer of the sharp image reconstruction module. The output of the first 3×3 Conv layer of the sharp image reconstruction module is input to the first InstanceNorm layer of the sharp image reconstruction module. The output of the first InstanceNorm layer of the sharp image reconstruction module is input to the first ReLU layer of the sharp image reconstruction module. The output of the first ReLU layer of the sharp image reconstruction module is input to the first ReflectionPad2d layer of the sharp image reconstruction module. The output of the first ReflectionPad2d layer of the sharp image reconstruction module is input to the second 3×3 Conv layer of the sharp image reconstruction module. The output of the second 3×3 Conv layer of the sharp image reconstruction module is input to the first Tanh layer of the sharp image reconstruction module. The output of the first Tanh layer of the sharp image reconstruction module is the dehazed image.

[0058] Compared with the prior art, the present invention has the following technical effects:

[0059] 1) This invention, by designing a well-designed Dynamic Routing Feature Framework (DRFF) and introducing a dynamic routing feature module, dynamically routes features with representative information extracted from the U-shaped subnet and the full-resolution convolutional subnet, as well as the features fused from the two. This enables the reconstruction of more ideal remote sensing images, thereby significantly improving the dehazing effect, especially for small-scale objects under haze in remote sensing images.

[0060] 2) This invention proposes a 3D perceptual feature fusion module, exploring how to combine spatial and channel dimensions to effectively preserve important global context for coarse fusion, and utilizing 3D interaction to generate pixel-level granular weight matrices through self-replication for final feature fusion. Therefore, the 3D perceptual feature fusion module can enhance the fused feature representation and improve the recovery of texture details and colors;

[0061] 3) This invention proposes a novel method called Dynamic Routing and 3D Fusion Remote Sensing Image Dehazing Network, which seamlessly integrates a carefully designed dynamic routing feature framework and a 3D perception feature fusion module. Through more efficient dynamic feature selection and fusion feature representation, experiments on challenging datasets such as SateHaze1k, RICE, and RSID demonstrate that the remote sensing haze removal method of this invention has superior performance compared to several state-of-the-art methods. Attached Figure Description

[0062] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0063] Figure 1This is a structural diagram of the overall dynamic routing feature framework of an embodiment of the present invention;

[0064] Figure 2 for Figure 1 A schematic diagram of the structure of the 3D perception feature fusion module in the image;

[0065] Figure 3 for Figure 1 A schematic diagram of the dynamic routing feature module in the diagram. Detailed Implementation

[0066] like Figure 1 As shown, a dehazing network based on dynamic routing and 3D fusion is presented. The network includes a two-branch network, a dynamic routing feature module (DRFM), and a clear image reconstruction module (CIRM).

[0067] The input of the Two-branch Network is used to input the foggy image I. The output of the Two-branch Network is connected to the input of the Dynamic Routing Feature Module (DRFM). The output of the DRFM is connected to the input of the Clear Image Reconstruction Module (CIRM). The output of the CIRM is used to output the defoggy image.

[0068] like Figure 1 As shown, the Two-branch Network includes a U-shaped sub-network U-Net, a full-resolution convolutional sub-network F-Net, and a 3D perception feature fusion module;

[0069] The constructed U-shaped subnetwork is specifically as follows:

[0070] The foggy image I is input to the encoding layer 201. The output of the encoding layer 201 is input to the encoding layer 202. The output of the encoding layer 202 is input to the encoding layer 203. The output of the encoding layer 203 is input to the residual feature extraction module 204. The output of the residual feature extraction module 204 and the output of the encoding layer 203 are input to the 3D perception feature fusion module 205. The output of the 3D perception feature fusion module 205 is input to the decoding layer 206. The output of the decoding layer 206 and the output of the encoding layer 202 are input to the 3D perception feature fusion module 207. The output of the 3D perception feature fusion module 207 is input to the decoding layer 208. The output of the decoding layer 208 and the output of the encoding layer 201 are input to the 3D perception feature fusion module 209. The output feature of the 3D perception feature fusion module 209 is denoted as F1.

[0071] The feature extraction operation of the coding layer is as follows:

[0072] The foggy image I is input to the first ReflectionPad2d layer of the coding layer. The output of the first ReflectionPad2d layer of the coding layer is input to the first 7×7 Conv layer of the coding layer. The output of the first 7×7 Conv layer of the coding layer is input to the first InstanceNorm layer of the coding layer. The output of the first InstanceNorm layer of the coding layer is input to the first ReLU layer of the coding layer. The output of the first ReLU layer of the coding layer is used to obtain the first layer of coded features.

[0073] The first layer of encoded features is input to the second 3×3 Conv layer of the encoding layer. The output of the second 3×3 Conv layer of the encoding layer is input to the second InstanceNorm layer of the encoding layer. The output of the second InstanceNorm layer of the encoding layer is input to the second ReLU layer of the encoding layer. The output of the second ReLU layer of the encoding layer is the second layer of encoded features.

[0074] The second layer of encoded features is input to the third 3×3 Conv layer of the encoding layer. The output of the third 3×3 Conv layer of the encoding layer is input to the third InstanceNorm layer of the encoding layer. The output of the third InstanceNorm layer of the encoding layer is input to the third ReLU layer of the encoding layer. The output of the third ReLU layer of the encoding layer is the third layer of encoded features.

[0075] The residual feature extraction module consists of 6 cascaded residual blocks, and the residual block features are extracted as follows:

[0076] The input features are fed into the first 3×3 Conv layer of the residual block. The output of the first 3×3 Conv layer of the residual block is fed into the first ReLU layer of the residual block. The output of the first ReLU layer of the residual block and the input features are fed into the first 3D perception feature fusion module of the residual block. The output features of the first 3D perception feature fusion module of the residual block are fed into the second 3×3 Conv layer of the residual block. The output of the second 3×3 Conv layer of the residual block and the input features are fed into the second 3D perception feature fusion module of the residual block. The output of the second 3D perception feature fusion module of the residual block is the output feature of the residual block.

[0077] The image restoration operation in the decoding layer is as follows:

[0078] The output of the residual feature extraction module 204 and the output of the encoding layer 203 are input together to the first 3D perception feature fusion module 205 of the decoding layer. The output of the 3D perception feature fusion module 205 of the decoding layer is input to the first Dconv layer of the decoding layer. The output of the first Dconv layer of the decoding layer is input to the first InstanceNorm layer of the decoding layer. The output of the first InstanceNorm layer of the decoding layer is input to the first ReLU layer of the decoding layer. The output of the first ReLU layer of the decoding layer is the first layer of decoded features.

[0079] The first layer of decoded features and the output of the second encoding layer 202 are input together to the second 3D perception feature fusion module 207 of the decoding layer. The output of the second 3D perception feature fusion module 207 of the decoding layer is input to the second Dconv layer of the decoding layer. The output of the second Dconv layer of the decoding layer is input to the second InstanceNorm layer of the decoding layer. The output of the second InstanceNorm layer of the decoding layer is input to the second ReLU layer of the decoding layer. The output of the second ReLU layer of the decoding layer is the second layer of decoded features.

[0080] The second layer of decoding features and the output of the first encoding layer 201 are input together to the third three-dimensional perception feature fusion module 209 of the decoding layer. The output of the third three-dimensional perception feature fusion module 209 of the decoding layer is the feature F1 extracted by the U-shaped sub-network.

[0081] The proposed 3D perception feature fusion module, such as Figure 2 As shown, the process is divided into two stages: coarse fusion and fine fusion. The specific operation of coarse fusion is as follows:

[0082] Feature map and feature map Each feature map undergoes a first Flatten operation to obtain output feature map 702 and output feature map 703. Output feature map 702 and output feature map 703 then undergo a first Concat operation in coarse fusion to obtain feature map 705. Feature map 705 then undergoes a first adaptive global average pooling operation in coarse fusion to obtain feature vector 706. Feature vector 706 then undergoes a first 1×1 Conv operation, a first ReLU operation, and a first Sigmoid operation in coarse fusion to obtain vectors C1 and C2. Vectors C1 and C2 are then compared with the feature maps... and feature map The feature map is obtained through the first multiplication of the coarse fusion. and feature map Feature map and feature map The coarse fusion output feature X is obtained by the first element-wise addition of the coarse fusion. CF .

[0083] The specific steps for fine-tuning are as follows:

[0084] Coarse fusion output feature X CF The channel feature vector CI is obtained by sequentially performing the first adaptive global average pooling operation, the first 1×1 Conv operation, and the first ReLU operation in fine fusion. The channel feature vector CI then undergoes two 1×1 Conv operations in fine fusion to obtain channel feature vectors CI1 and CI2. Channel feature vectors CI1 and CI2 are then subjected to a broadcasting mechanism in fine fusion to obtain feature maps C. r1 and feature map C r2 ;

[0085] Coarse fusion output feature X CF The spatial feature map PI is obtained by sequentially performing a fourth 1×1 Conv operation and a second ReLU operation in fine fusion. Spatial feature map PI is then subjected to two more 1×1 Conv operations in fine fusion to obtain spatial feature maps PI1 and PI2. Spatial feature maps PI1 and PI2 are then finely fused and broadcast to obtain feature map PI2. r1 and feature map PI r2 ;

[0086] Channel feature vector C r1 Spatial feature map PI r1 The first pixel-by-pixel addition after fine fusion yields the pixel-level three-dimensional information matrix V1;

[0087] Channel feature vector C r2 Spatial feature map PI r2 The second pixel-by-pixel addition after fine fusion yields the pixel-level three-dimensional information matrix V2;

[0088] The pixel-level 3D information matrices V1 and V2 undergo a first concat operation in fine fusion to obtain a feature map 724. Feature map 724 then undergoes a first softmax operation in fine fusion to obtain a pixel-level 3D weight matrix. and pixel-level 3D weight matrix

[0089] Fusion features and pixel-level 3D weight matrix The first pixel-wise multiplication operation after fine fusion yields the weighted feature map 727;

[0090] Fusion features and pixel-level 3D weight matrix The second pixel-wise multiplication operation after fine fusion yields the weighted feature map 728;

[0091] The weighted feature maps 727 and 728 are subjected to a third pixel-wise addition in fine fusion to obtain the fused feature V. out .

[0092] The constructed full-resolution sub-network F-Net is specifically as follows:

[0093] The foggy image I is input to the channel changing module 301, and the output of the channel changing module 301 is input to the residual feature extraction module 302. The output of the residual feature extraction module 302 is the feature F2 extracted by the full-resolution sub-network.

[0094] The specific operation of the channel changing module 301 is as follows:

[0095] The foggy image I is input to the first ReflectionPad2d layer of the channel changing module 301. The output of the first ReflectionPad2d layer of the channel changing module 301 is input to the first 7×7 Conv layer of the channel changing module 301. The output of the first 7×7 Conv layer of the channel changing module 301 is input to the first InstanceNorm layer of the channel changing module 301. The output of the first InstanceNorm layer of the channel changing module 301 is input to the first ReLU layer of the channel changing module 301.

[0096] The specific operation of residual feature extraction module 302 is the same as that of residual feature extraction module 204.

[0097] The constructed dynamic feature routing module, such as Figure 3 As shown, the details are as follows:

[0098] Feature maps F1, F2, and F3 first undergo a detach operation 501. The results of the detach operation 501 are then input to the corresponding feature scoring units FSU1, FSU2, and FSU3, respectively. The features input to the feature scoring units first pass through the feature extraction module 505, which consists of a global average pooling layer, a 1×1 Conv layer, and a ReLU layer connected in sequence. The feature extraction module 505 outputs three feature vectors 506 corresponding to feature maps F1, F2, and F3, respectively. Feature vectors 506 are input to the feature extraction module 507, which consists of a 1×1 Conv layer and a Sigmoid layer connected in sequence. The feature extraction module 507 outputs feature scores 508 corresponding to feature maps F1, F2, and F3, respectively. The three feature scores 508 are then processed by a Concat operation 509 and a Gumbel-Softmax function 510 to obtain a discrete gated sequence 511.

[0099] The clear image reconstruction module specifically includes:

[0100] The feature map is input to the first 3×3 Conv layer of the sharp image reconstruction module. The output of the first 3×3 Conv layer of the sharp image reconstruction module is input to the first InstanceNorm layer of the sharp image reconstruction module. The output of the first InstanceNorm layer of the sharp image reconstruction module is input to the first ReLU layer of the sharp image reconstruction module. The output of the first ReLU layer of the sharp image reconstruction module is input to the first ReflectionPad2d layer of the sharp image reconstruction module. The output of the first ReflectionPad2d layer of the sharp image reconstruction module is input to the second 3×3 Conv layer of the sharp image reconstruction module. The output of the second 3×3 Conv layer of the sharp image reconstruction module is input to the first Tanh layer of the sharp image reconstruction module. The output of the first Tanh layer of the sharp image reconstruction module is the dehazed image.

[0101] In step S4, the network is trained using four different loss constraints as follows:

[0102] (1) SmoothL1 loss, the specific formula is:

[0103]

[0104]

[0105] G(χ j χ represents the pixel value at position i in the input image, which is then processed by the dehazing network parameters. P represents the number of pixels in the image. j Z represents the value of pixel j in a foggy image. j This represents the value of pixel j in the ground truth (GT) field.

[0106] (2) Adversarial loss, the specific definition of adversarial loss is as follows:

[0107]

[0108] D represents the generator, and G represents the discriminator. D(G(x)) represents the probability of classifying a dehazed image as a clear image, and m represents the total number of images.

[0109] (3) Perceptual loss, which calculates the L2 norm of the VGG16 pre-trained model at different layers for the ground truth (GT) and fog-free images from the network. The perceptual loss is defined as follows:

[0110]

[0111] Where η iThis represents the size of the feature map obtained from the i-th layer of the pre-trained VGG16 model. P represents the number of feature layers used for perceptual loss. X and Y represent the foggy image and the clear, fog-free image, respectively.

[0112] (4) Multi-scale structural similarity loss (MS-SSIM): To make the structural similarity between the dehazed image and the clear image closer, multi-scale structural similarity loss (MS-SSIM) is introduced, which is expressed as follows:

[0113]

[0114] in and η represents the mean of the dehazed image and the ground truth (GT), respectively. ij η is the covariance between the dehazed image and the ground plane. i and η j Let C be the standard deviation of the dehazed image from the ground truth (GT), and C be a constant. The overall network loss function is expressed as:

[0115] L=θ1L smoothL1 +θ2L adv +θ3L perc +θ4L ms-ssim

[0116] Where θ1, θ2, θ3, and θ4 are hyperparameters of each loss function.

[0117] When this model is working, the process is as follows:

[0118] like Figure 1 As shown, the input foggy image I to the dynamic routing feature framework includes a dual-branch network, a dynamic routing feature module, and a sharp image reconstruction module. The dual-branch network consists of an upper branch U-shaped subnetwork and a lower branch full-resolution convolutional subnetwork operating in parallel. The two subnetworks extract and fuse features that enable the dual-branch network to output three different feature information (feature F1 extracted by the U-shaped subnetwork, feature F2 extracted by the full-resolution convolutional subnetwork, and feature F3 obtained by fusing the features extracted by the two subnetworks through 3D perception features). The feature scoring module scores the three features, and the Gumbel-Softmax gate function selects the feature with the highest score. The sharp image reconstruction module then reconstructs a sharp remote sensing image.

[0119] The upper branch U-shaped subnet encoding layer feature extraction module employs three convolutional operations followed by 4x downsampling for feature extraction. The first layer of the encoding stage consists of a ReflectionPad2d layer, a convolutional layer, an activation layer, and an InstanceNorm. The second and third layers of the encoding stage consist of convolutional layers, activation layers, and an InstanceNorm. Next, six consecutive residual blocks are combined to further enhance feature extraction on the extracted low-resolution features. Finally, the features extracted from the consecutive residual blocks, containing a large amount of semantic scene information, are decoded and reconstructed using deconvolution and convolution operations. During this process, the corresponding encoding and decoding layer features are fused using a 3D perception feature fusion module, ultimately obtaining the F1 score of the features extracted by the U-shaped subnet with the original image resolution and 64 channels. The lower branch full-resolution convolutional subnetwork consists of one convolutional block and six consecutive residual blocks. The convolutional block comprises a ReflectionPad2d layer, a 7×7 Conv layer, InstanceNorm, and ReLU, expanding the input image channels to 64. The six consecutive residual blocks further enhance feature extraction, ultimately yielding feature F2 extracted by the upper branch full-resolution subnetwork. Furthermore, the features obtained from the two subnetworks are fused using a 3D perception feature fusion module to obtain feature F3. The formula is expressed as follows:

[0120] F1 = Unet(I)

[0121] F2 = Fnet(I)

[0122] F3 = 3DPFF(F1, F2)

[0123] Among them, F1 is the feature extracted by U-Net, which contains rich contextual information; F2 is the feature extracted by F-Net without downsampling, which can retain more high-frequency detailed information; and F3 is the feature obtained by fusing F1 and F2 through the 3D perception feature fusion module.

[0124] Construct a 3D perception feature fusion module, such as Figure 2 As shown, the 3D perceptual feature fusion module comprises two stages: coarse fusion and fine fusion. Coarse fusion first aggregates spatial and channel information from the two fusion features using adaptive global average pooling, thereby learning the global context information of each input feature. Subsequently, fine-grained fusion achieves 3D interaction across spatial and channel dimensions through self-replication, generating pixel-level granular weight matrices assigned to the corresponding input features for the second fusion. This two-stage fusion enhances the representation of the fusion features and improves the recovery of texture details and colors.

[0125] like Figure 3 As shown, the feature scoring module consists of three independent scoring units, used to specifically evaluate each feature sample in F1, F2, and F3. The evaluation process can be represented as:

[0126]

[0127] In the formula, F j The score represents the score for each sample within the three features, bs represents the number of samples in the batch, and FSU() is the feature scoring unit. Specifically, the evaluation process includes two steps. First, spatial information is aggregated using global average pooling and abstracted into a contribution vector using learnable convolution, enhancing robustness to spatial variations. Then, the contribution vector is compressed using a 1*1 convolution and mapped to contribution scores using a Sigmoid activation function. Next, the Gumbel-Softmax function is used to select the features that contribute most to the reconstructed haze-free image. Feature scores S1, S2, and S3 are passed to the GumbelSoftmax function, which maps them to a binary vector with values ​​from 0 to 1 (where only one element is "1", representing the selected feature). Then, the vector is multiplied by the corresponding feature to complete feature routing. The formula is expressed as follows:

[0128]

[0129] Where F represents the feature map selected and passed to the subsequent network, and g() represents the Gumbel-Softmax function, which allows the network to optimize the discrete decision through backpropagation. This function is used after the feature scoring module so that it can adaptively update the scoring strategy through backpropagation.

[0130] Example

[0131] To facilitate a better understanding of the present invention by those skilled in the art, further explanation is provided below:

[0132] 1. Parameter settings

[0133] All experiments in this study were performed on an NVIDIA RTX 3090 GPU, and the code was implemented using PyTorch. The proposed network was optimized using the Adam optimizer, with momentum decay metrics set to β1 = 0.9 and β2 = 0.999. The learning rate and batch size were configured to 0.0001 and 4, respectively. The initial learning rate was set to 0.001, and MultiStepLR was used to dynamically adjust between the two. Throughout the training process, the parameters of the loss function in the network were set to θ1 = 1, θ2 = 0.0005, θ3 = 0.01, and θ4 = 0.5. Furthermore, this invention evaluated various dehazing algorithms on the synthetic datasets SateHaze1k and RSID, as well as the real-world dataset RICE. The SateHaze1k dataset consists of three subsets: thin, moderate, and thick. Each subset contains 400 pairs of synthetic remote sensing images, of which 320 pairs are the training set, 35 pairs are the validation set, and 45 pairs are the test set. Each image has a resolution of 512*512 pixels. The RSID dataset contains 1000 pairs of synthetic remote sensing images with a resolution of 256*256 pixels. 900 pairs were randomly selected for training, and the remainder were used for testing. The RICE dataset contains two subsets: RICE1 and RICE2, which are real-world remote sensing datasets. The RICE1 dataset consists of 500 pairs of 512*512 resolution images from Google Earth. The RICE2 dataset consists of 450 pairs of 512*512 resolution Landsat 8OLI / TIRS satellite images. The entire RICE1 dataset was randomly divided into training and testing sets at a 9:1 ratio, and a similar division was performed on the RICE2 dataset. To verify the correctness and effectiveness of the method of this invention, it was compared with the method of this invention using currently excellent dehazing algorithms, including: DCP, FFA, FCTF, TBN, SGID, DCI-Net, MFINEA, Trinity-Net, and PSMB-Net.

[0134] 2. Experimental Results

[0135] Table 1 compares the image dehazing results with various excellent image dehazing methods on the SateHaze1k dataset.

[0136]

[0137] Table 2 compares the results with various excellent image dehazing methods on the RICE and RSID datasets.

[0138]

[0139] This invention achieved first place in PSNR and SSIM on the SateHaze1k, RSID, and RICE datasets. As shown in Tables 1, 2, and 3, the end-to-end haze removal method exhibits superior performance compared to prior-based methods. Compared to U-Net-based dehazing networks such as MFINEA and PSMB-Net, this invention achieves significant improvements on the SateHaze1k dataset. Specifically, the average PSNR is improved by 1.41 dB and 0.84 dB, and the SSIM is improved by 0.028 dB and 0.005 dB, respectively. On the RSID dataset, this invention shows even better results, with PSNR improvements of 3.8 dB and 2.7 dB, and SSIM improvements of 0.003 and 0.005, respectively. Compared to F-Net-based dehazing networks such as FFA, this invention improves PSNR and SSIM by 1.41 dB and 0.032 on SateHaze1k, and by 3.6 dB and 0.004 on RSID, respectively. This invention achieves state-of-the-art performance on both the RICE1 and RICE2 datasets. Compared to methods such as FFA, DCI-Net, and PSMB-Net, this invention shows significant improvements on the RICE1 dataset. PSNR is improved by 2.32 dB, 2.30 dB, and 2.65 dB, respectively, and SSIM is improved by 0.014, 0.005, and 0.002, respectively. On the RICE2 dataset, this invention continues to outperform DCI-Net and PSMB-Net, with PSNR improvements of 3.28 dB and 1.46 dB, respectively, and SSIM improvements of 0.012 and 0.005, respectively.

[0140] 3. Ablation analysis

[0141] To evaluate the effectiveness of each module of the invention, ablation experiments were designed based on the innovations in the dynamic routing feature framework and attention mechanism. These experiments comprised five experiments: 1) F-Net: F-Net subnet only; 2) U-Net: U-Net subnet only; 3) base: F-Net + U-Net; 4) base + DFRF; 5) base + DFRF + 3D-PFF. Experiments 1-4 employed Element-Wise Addition for feature fusion. Ablation experiments were conducted on the SateHaze1k dataset.

[0142] Table 3 shows the ablation experimental results on the SateHaze1k dataset.

[0143]

[0144] The results of experiments (1) and (2) show that the single haze removal performance of a single U-Net or F-Net is limited due to the loss of high-frequency information in U-Net and the limited receptive field of F-Net. This experiment constructed a baseline model by deploying a dual-branch network composed of U-Net and F-Net, achieving scores of 25.35 dB and 0.910 on the baseline model. Based on this, DFRF was developed, which significantly enhances the PSNR and SSIM metrics. DFRF can dynamically route the extracted and fused features of U-Net and F-Net, effectively utilizing the advantages of both models in terms of large receptive field and detail preservation, thereby achieving effective haze removal and recovery of small-scale objects. In Experiment 5, the introduction of the 3D perception feature fusion module into the network can enhance the expressive ability of the fused features within the network, thereby further improving the texture details and colors of the dehazed images. Significant improvements were achieved on the Thin, Moderate, and Thick subsets, with improvements of 1.22dB and 0.011 on the Thin subset, 2.04dB and 0.017 on the Moderate subset, and 0.92dB and 0.009 on the Thick subset.

[0145] This invention provides a dual-branch remote sensing image dehazing network method based on dynamic routing and 3D fusion, comprising a dynamic routing feature framework and a 3D perception feature fusion module. The dynamic routing feature framework utilizes features extracted by the dynamic routing U-Net and F-Net, containing multiple representative feature information, thereby significantly improving haze removal and the reconstruction of small-scale objects under haze conditions. The 3D perception feature fusion module enhances the interaction between the spatial and channel information of multiple features, assigning pixel-level weights to feature fusion and improving the texture details and color of the dehazed image.

Claims

1. A defogging network based on dynamic routing and 3D fusion, characterized in that, The network includes a two-branch network (1), a dynamic routing feature module (DRFM) (5), and a clear image reconstruction module (CIRM) (6). The input of the Two-branch Network (1) is used to input the foggy image I. The output of the Two-branch Network (1) is connected to the input of the Dynamic Routing Feature Module DRFM (5). The output of the Dynamic Routing Feature Module DRFM (5) is connected to the input of the Clear Image Reconstruction Module CIRM (6). The output of the Clear Image Reconstruction Module CIRM (6) is used to output the defoggy image. The Two-branch Network (1) includes a U-shaped sub-network U-Net (2), a full-resolution convolutional sub-network F-Net (3), and a 3D perception feature fusion module; Among them, the U-shaped sub-network U-Net (2) is as follows: the input of the first coding layer (201) is used to input the foggy image I; the output of the first coding layer (201) is connected to the input of the second coding layer (202); the output of the second coding layer (202) is connected to the input of the third coding layer (203); the output of the third coding layer (203) is connected to the input of the first residual feature extraction module (204); the output of the first residual feature extraction module (204) and the output of the third coding layer (203) are connected together with the input of the first three-dimensional perception feature fusion module (205); the first three-dimensional perception feature The output of the fusion module (205) is connected to the input of the first decoding layer (206). The output of the first decoding layer (206) and the output of the second encoding layer (202) are connected to the input of the second three-dimensional perception feature fusion module (207). The output of the second three-dimensional perception feature fusion module (207) is connected to the input of the second decoding layer (208). The output of the second decoding layer (208) and the output of the first encoding layer are connected to the input of the third three-dimensional perception feature fusion module (209). The output feature of the third three-dimensional perception feature fusion module (209) is denoted as feature F1 (401). The full-resolution convolutional subnetwork F-Net(3) is specifically as follows: The input of the channel changing module (301) is used to input the foggy image I. The output of the channel changing module (301) is connected to the input of the second residual feature extraction module (302). The output feature of the second residual feature extraction module (302) is feature F2 (402). Feature F1 (401) and feature F2 (402) are fused with three-dimensional perception features to obtain feature F3 (403); then the dynamic routing feature module DRFM (5) scores feature F1 (401), feature F2 (402) and feature F3 (403) and selects the most suitable one for reconstructing a clear image to calculate the loss; The 3D perception feature fusion module is divided into two stages: coarse fusion and fine fusion. The specific operation of coarse fusion is as follows: Feature map (701) and feature map (704) Each feature map undergoes the first Flatten operation of coarse fusion to obtain output feature map (702) and output feature map (703). Output feature map (702) and output feature map (703) undergo the first Concat operation of coarse fusion to obtain feature map (705). Feature map (705) undergoes the first adaptive global average pooling operation of coarse fusion to obtain feature vector (706). Feature vector (706) undergoes the first Conv operation, the first ReLU operation, and the first Sigmoid operation of coarse fusion to obtain vector. (707) and vector (708), Vector (707) and vector (708) respectively with the feature map (701) and feature map (704) The feature map is obtained through the first multiplication of the coarse fusion. (709) and feature map (710), Feature Map (709) and feature map (710) The coarse fusion output features are obtained by the first element-wise addition of the coarse fusion. (711); The specific steps for fine-tuning are as follows: Coarse fusion output features (711) The channel feature vector CI is obtained after the first adaptive global average pooling operation, the first Conv operation, and the first ReLU operation of fine fusion. (712) The channel feature vector CI (712) is then subjected to two Conv operations of fine fusion to obtain the channel feature vector. (713) and channel feature vectors (714), Channel Feature Vector (713) and channel feature vectors (714) Feature maps are obtained through a fine-grained broadcasting mechanism. (715) and feature map (716); Coarse fusion output features (711) The spatial feature map PI(717) is obtained after the fourth Conv operation of fine fusion and the second ReLU operation of fine fusion. The spatial feature map PI(717) is then subjected to two Conv operations of fine fusion to obtain the spatial feature map. (718) and spatial feature map (719), Spatial Feature Map (718) and spatial feature map (719) Feature maps are obtained through a fine-grained broadcasting mechanism. (720) and feature map (721); Channel feature vector (715) and spatial feature map (720) The first pixel-by-pixel addition after fine fusion yields a pixel-level three-dimensional information matrix. (722); Channel feature vector (716) and spatial feature map (721) The pixel-level three-dimensional information matrix is ​​obtained by the second pixel-by-pixel addition after fine fusion. (723); Pixel-level 3D information matrix (722) and pixel-level three-dimensional information matrix (723) The first Concat operation after fine fusion yields a feature map (724), and the feature map (724) undergoes the first Softmax operation after fine fusion to obtain a pixel-level three-dimensional weight matrix. (725) and pixel-level 3D weight matrix (726); Fusion features and pixel-level 3D weight matrix (725) The first pixel-wise multiplication operation after fine fusion yields the weighted feature map (727); Fusion features and pixel-level 3D weight matrix (726) The weighted feature map (728) is obtained by the second pixel-wise multiplication operation after fine fusion; The weighted feature maps (727) and (728) are combined using a third pixel-wise addition process to obtain the fused feature. (729); The dynamic routing feature module is as follows: Feature map (401) Feature Map (402) and feature map (403) First, after the detach operation (501), the results of the three feature maps after the detach operation are input to the corresponding feature scoring units. (502), Feature scoring unit (504) and feature scoring unit (503) The features input to the feature scoring unit first pass through the first feature extraction module (505), which consists of a global average pooling layer, a Conv layer, and a ReLU layer connected in sequence. After passing through the first feature extraction module (505), three features are output, which are related to the feature map. (401) Feature Map (402) and feature map (403) The corresponding feature vectors (506) are input to the second feature extraction module (507), which is composed of a Conv layer and a Sigmoid layer connected in sequence. After passing through the second feature extraction module (507), the output is the feature map. (401) Feature Map (402) and feature map (403) The corresponding feature scores (508) are obtained sequentially. The three feature scores (508) are then subjected to a sequential Concat operation (509) and a Gumbel-Softmax function (510) to obtain a discrete gated sequence (511).

2. The network according to claim 1, characterized in that, The specific operation of the channel changing module (301) is as follows: The foggy image I is input to the first ReflectionPad2d layer of the channel alteration module. The output of the first ReflectionPad2d layer of the channel alteration module is input to the first Conv layer of the channel alteration module. The output of the first Conv layer of the channel alteration module is input to the first InstanceNorm layer of the channel alteration module. The output of the first InstanceNorm layer of the channel alteration module is input to the first ReLU layer of the channel alteration module.

3. The network according to claim 1, characterized in that, The process of training a network using four loss constraints, which are as follows: (1) SmoothL1 loss, the specific formula is: ； ； Representative input image The pixel value at that location is then processed by the dehazing network parameters to obtain the value, where P represents the number of pixels in the image; Represents pixels in a foggy image The value, Represents the ground truth value GT pixels The value; (2) Adversarial loss, the specific definition of adversarial loss is as follows: ； Represents a generator. Indicates the discriminator, This represents the probability of classifying a dehazed image as a clear image. Represents the total number of images; (3) Perceptual loss, which calculates the L2 norm of the VGG16 pre-trained model on the ground truth and fog-free images from the network at different layers. The perceptual loss is defined as follows: ； in denoted by , where represents the size of the feature map obtained from the i-th layer of the pre-trained VGG16 model, P represents the number of feature layers used for perceptual loss, and X and Y represent the foggy image and the clear, fog-free image, respectively. (4) Multi-scale structural similarity loss MS-SSIM: To make the structural similarity between the dehazed image and the clear image closer, multi-scale structural similarity loss MS-SSIM is introduced, which is expressed as follows: ； in and These are the mean values ​​of the dehazed image and the ground truth (GT), respectively. The covariance between the dehazed image and the ground plane. and The standard deviation of the dehazed image from the ground truth (GT). The loss function of the entire network is expressed as: ； in , , , These are the hyperparameters for each loss function.

4. The network according to claim 1, characterized in that, The feature extraction operation of the coding layer is as follows: The feature input is fed into the first ReflectionPad2d layer of the encoding layer, the output of the first ReflectionPad2d layer of the encoding layer is fed into the first Conv layer of the encoding layer, the output of the first Conv layer of the encoding layer is fed into the first InstanceNorm layer of the encoding layer, the output of the first InstanceNorm layer of the encoding layer is fed into the first ReLU layer of the encoding layer, and the output of the first ReLU layer of the encoding layer is used to obtain the first layer of encoded features. The first layer of encoded features is input into the second Conv layer of the encoding layer. The output of the second Conv layer of the encoding layer is input into the second InstanceNorm layer of the encoding layer. The output of the second InstanceNorm layer of the encoding layer is input into the second ReLU layer of the encoding layer. The output of the second ReLU layer of the encoding layer is the second layer of encoded features. The second layer of encoded features is input to the third Conv layer of the encoding layer. The output of the third Conv layer of the encoding layer is input to the third InstanceNorm layer of the encoding layer. The output of the third InstanceNorm layer of the encoding layer is input to the third ReLU layer of the encoding layer. The output of the third ReLU layer of the encoding layer is the third layer of encoded features.

5. The network according to claim 1, characterized in that, The residual feature extraction module consists of 6 cascaded residual blocks, and the feature extraction of the residual blocks is as follows: The input features are fed into the first Conv layer of the residual block. The output of the first Conv layer of the residual block is fed into the first ReLU layer of the residual block. The output of the first ReLU layer of the residual block and the input features are fed into the first 3D perception feature fusion module of the residual block. The output features of the first 3D perception feature fusion module of the residual block are fed into the second Conv layer of the residual block. The output of the second Conv layer of the residual block and the input features are fed into the second 3D perception feature fusion module of the residual block. The output of the second 3D perception feature fusion module of the residual block is the output feature of the residual block.

6. The network according to claim 1, characterized in that, The image restoration operation of the decoding layer is as follows: The output of the first residual feature extraction module (204) and the output of the third encoding layer (203) are input together to the first three-dimensional perception feature fusion module (205). The output of the first three-dimensional perception feature fusion module (205) is input to the first Dconv layer of the decoding layer. The output of the first Dconv layer of the decoding layer is input to the first InstanceNorm layer of the decoding layer. The output of the first InstanceNorm layer of the decoding layer is input to the first ReLU layer of the decoding layer. The output of the first ReLU layer of the decoding layer is used to obtain the first layer of decoded features. The first layer of decoding features and the output of the second encoding layer (202) are input together to the second three-dimensional perception feature fusion module (207). The output of the second three-dimensional perception feature fusion module (207) is input to the second Dconv layer of the decoding layer. The output of the second Dconv layer of the decoding layer is input to the second InstanceNorm layer of the decoding layer. The output of the second InstanceNorm layer of the decoding layer is input to the second ReLU layer of the decoding layer. The output of the second ReLU layer of the decoding layer is used to obtain the second layer of decoding features. The second layer decoding features and the output of the first encoding layer (201) are fed together into the third three-dimensional perception feature fusion module (209), and the output of the third three-dimensional perception feature fusion module (209) is used to obtain the U-shaped sub-network extracted features F1 (401).

7. The network according to claim 1, characterized in that, The Clear Image Reconstruction (CIRM) module specifically includes: The feature map is input to the first Conv layer of the CIRM (Clear Image Reconstruction Module). The output of the first Conv layer is input to the first InstanceNorm layer of the CIRM. The output of the first InstanceNorm layer is input to the first ReLU layer of the CIRM. The output of the first ReLU layer is input to the first ReflectionPad2d layer of the CIRM. The output of the first ReflectionPad2d layer is input to the second Conv layer of the CIRM. The output of the second Conv layer is input to the first Tanh layer of the CIRM. The output of the first Tanh layer is the dehazed image.

Images