Defogging network based on dynamic routing and three-dimensional fusion

Abstract

The invention relates to a defogging network based on dynamic routing and three-dimensional fusion. The network comprises a dual-branch network Twood-branch network, a dynamic routing feature module DRFM and a clear image reconstruction module CIRM, the input end of the double-branch network Twood-branch network is used for inputting a foggy image I, the output of the double-branch network Twood-branch network is connected with the input of the dynamic routing feature module DRFM, the output of the dynamic routing feature module DRFM is connected with the input of the clear image reconstruction module CIRM, and the output of the clear image reconstruction module CIRM is used for outputting a defogged image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a defogging network based on dynamic routing and three-dimensional fusion. Background Art

[0002] Remote sensing images are widely used in geological exploration, climate change research, environmental protection and other fields and play a vital role. However, affected by tiny particles in the weather such as haze, remote sensing images often suffer from severe image degradation such as blurred details and color distortion, which limits their application in advanced computer vision tasks such as change detection and semantic segmentation. Therefore, how to effectively remove haze from remote sensing images and restore clear images has attracted widespread attention from researchers.

[0003] Early dehazing methods for natural images and remote sensing images rely on the introduction of prior assumptions and knowledge to restore clear images. For example, Li et al. published "Aod-net: All-in-one dehazing network" to design AOD-Net to generate restored images by reformulating the physical scattering model. However, in complex haze scenes, the estimated priors are often proven to be inaccurate, resulting in very unsatisfactory dehazed images. In recent years, with the development of deep learning, researchers have proposed a large number of end-to-end haze removal methods, using convolutional neural networks (CNN) and transformers to learn the mapping relationship between hazy and clear images. The frameworks of these models mainly include U-Net and full resolution convolutional networks (F-Net). For example, Dong et al. published "Multi-Scale Boosted Dehazing Network with DenseFeature Fusion" and designed a dense feature fusion module to solve the problem of retaining spatial information in the U-Net dehazing process. Zhang et al. proposed a dynamic collaborative inference learning (DCIL) framework to strengthen the underlying texture features in Dense haze removal based on dynamic collaborative inference learning for remote sensing images. Qin et al. proposed a non-downsampling feature fusion attention network in FFA-Net: Feature Fusion Attention Network for Single Image Dehazing to focus on more important information. Most methods using these two architectures have achieved significant progress in dehazing performance, but there are still two main problems:

[0004] (1) How to effectively utilize the features extracted in U-Net and F-Net architectures to achieve clear image reconstruction: U-Net models are widely used in remote sensing image dehazing. However, the downsampling of U-Net models in the encoding stage leads to the loss of high-frequency information, resulting in poor recovery of small-scale objects under haze conditions. In contrast, F-Net dehazing models without downsampling can effectively retain high-frequency feature information, but these models have limited receptive fields, resulting in residual haze. Recently, several dehazing models based on sequential and parallel dual network architectures have been proposed, which have achieved significant improvements in dehazing. Sequential dual network models use the output of the first network as the input of the second network, resulting in the loss of information that the second network cannot fully reconstruct. Parallel dual network models simply splice the features extracted by U-Net and F-Net together to achieve clear image reconstruction, but the dehazing effect is still insufficient. These dual network models fail to effectively utilize the features of different network architectures for optimal image reconstruction, and introduce a large amount of parameter and information redundancy by superimposing existing network structures.

[0005] (2) Feature fusion of existing dehazing models lacks spatial and channel information interactions between multiple features: Existing dehazing networks mainly use element addition and channel concatenation for feature fusion. In recent years, various feature fusion methods using learnable parameters and attention mechanisms have been proposed to enhance the performance of dehazing models. These methods assign weights to space, channels, and features as a whole to achieve effective constraints on the fusion process and promote more robust and accurate feature fusion. However, the fusion granularity of these methods is coarse and lacks fine-grained interactions between spatial and channel information, resulting in insufficient representation of the fused features. This reduces the dehazing model's ability to restore texture details and colors. Therefore, in the feature fusion process, it is an unresolved problem to deeply study the intrinsic spatial and channel correlations between multiple features, achieve fine-grained weight allocation, and enhance the representation of fused features to produce excellent texture details and color restoration. Summary of the invention

[0006] The purpose of the present invention is to solve the problems of high-frequency information loss caused by U-shaped network downsampling and limited receptive field of full-resolution convolutional network in existing remote sensing image defogging technology when using U-shaped network and full-resolution convolutional network; in addition, the present invention can also solve the technical problem that the existing defogging model lacks fine-grained interaction between spatial information and channel information in the fusion process, resulting in insufficient feature expression ability after fusion, and proposes a defogging network based on dynamic routing and three-dimensional fusion.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A defogging network based on dynamic routing and three-dimensional fusion, the network includes a two-branch network, a dynamic routing feature module DRFM and a clear image reconstruction module CIRM;

[0009] The input end of the two-branch network is used to input the foggy image I, the output of the two-branch network is connected to the input of the dynamic routing feature module DRFM, the output of the dynamic routing feature module DRFM is connected to the input of the clear image reconstruction module CIRM, and the output of the clear image reconstruction module CIRM is used to output the dehazed image.

[0010] like Figure 1 As shown in the figure, the two-branch network includes a U-shaped sub-network U-Net, a full-resolution convolutional sub-network F-Net, and a three-dimensional perception feature fusion module;

[0011] Among them, the U-shaped subnetwork U-Net is as follows: the input of the first coding layer is used to input the foggy image I, the output of the first coding layer is connected to the input of the second coding layer, the output of the second coding layer is connected to the input of the third coding layer, the output of the third coding layer is connected to the input of the residual feature extraction module, the output of the residual feature extraction module and the output of the third coding layer are connected together with the input of the first 3D perception feature fusion module; the output of the first 3D perception feature fusion module is connected to the input of the first decoding layer, the output of the first decoding layer and the output of the second coding layer are connected together with the input of the second 3D perception feature fusion module, the output of the second 3D perception feature fusion module is connected to the input of the second decoding layer, the output of the second decoding layer and the output of the first coding layer are connected together with the input of the third 3D perception feature fusion module, and the output feature of the third 3D perception feature fusion module is recorded as feature F1.

[0012] The full-resolution convolutional subnetwork F-Net is specifically:

[0013] The input of the channel change module is used to input the foggy image I, and the output of the channel change module is connected to the input of the residual feature extraction module, and the output feature of the residual feature extraction module is feature F2;

[0014] Among them, the specific operations of the channel change module are:

[0015] The foggy image I is input to the first ReflectionPad2d layer of the channel change module, the output of the first ReflectionPad2d layer of the channel change module is input to the first 7×7 Conv layer of the channel change module, the output of the first 7×7 Conv layer of the channel change module is input to the first InstanceNorm layer of the channel change module, and the output of the first InstanceNorm layer of the channel change module is input to the first ReLu layer of the channel change module.

[0016] Feature F1 and feature F2 are fused through three-dimensional perception features to obtain feature F3; then the dynamic routing feature module DRFM is used to score features F1, F2, and F3 and select the most suitable one for reconstructing a clear image to calculate the loss.

[0017] The process of network training is constrained by four losses. The four loss constraints are as follows:

[0018] (1) SmoothL1 loss, the specific formula is:

[0019]

[0020]

[0021] G(χ j ) represents the pixel value at position i of the input image and then the value obtained by the defogging network parameter operation, and P represents the number of pixels in the image. j represents the value of pixel j in the foggy image, z j represents the value of the ground truth (GT) pixel j.

[0022] (2) Adversarial loss. The specific definition of adversarial loss is as follows:

[0023]

[0024] D represents the generator and G represents the discriminator. D(G(x)) represents the probability of judging the dehazed image as a clear image, and m represents the total number of images.

[0025] (3) Perceptual loss, which calculates the L2 norm of the VGG16 pre-trained model for GT and haze-free images from the network at different layers. The perceptual loss is defined as follows:

[0026]

[0027] where η i represents the size of the feature map obtained from the i-th layer of the pre-trained VGG16 model. P represents the number of feature layers used for perceptual loss. X and Y represent the hazy image and the clear haze-free image, respectively.

[0028] (4) Multi-scale structural similarity loss (MS-SSIM), in order to make the structural similarity between the dehazed image and the clear image closer, the multi-scale structural similarity loss (MS-SSIM) is introduced, which is specifically expressed as follows:

[0029]

[0030] in and are the mean values ​​of the dehazed image and GT respectively. ij is the covariance between the dehazed image and GT, η i and η j is the standard deviation between the dehazed image and GT, and C is a constant. The loss function of the overall network is expressed as:

[0031] L=θ1L smoothL1 +θ2L adv +θ3L perc +θ4L ms-ssim

[0032] Where θ1, θ2, θ3, θ4 are the hyperparameters of each loss function.

[0033] The feature extraction operation of the encoding layer is as follows:

[0034] The feature is input to the first ReflectionPad2d layer of the encoding layer, the output of the first ReflectionPad2d layer of the encoding layer is input to the first 7×7 Conv layer of the encoding layer, the output of the first 7×7 Conv layer of the encoding layer is input to the first InstanceNorm layer of the encoding layer, the output of the first InstanceNorm layer of the encoding layer is input to the first ReLu layer of the encoding layer, and the output of the first ReLu layer of the encoding layer obtains the first layer of encoding features;

[0035] The first layer of encoded features is input into the second 3×3 Conv layer of the encoding layer, the output of the second 3×3 Conv layer of the encoding layer is input into the second InstanceNorm layer of the encoding layer, the output of the second InstanceNorm layer of the encoding layer is input into the second ReLu layer of the encoding layer, and the output of the second ReLu layer of the encoding layer obtains the second layer of encoded features;

[0036] The second layer of encoded features is input to the third 3×3 Conv layer of the encoding layer, the output of the third 3×3 Conv layer of the encoding layer is input to the third InstanceNorm layer of the encoding layer, the output of the third InstanceNorm layer of the encoding layer is input to the third ReLu layer of the encoding layer, and the output of the third ReLu layer of the encoding layer obtains the third layer of encoded features.

[0037] The residual feature extraction module consists of 6 residual blocks connected in series. The feature extraction of the residual block is as follows:

[0038] The input features are input to the first 3×3 Conv layer of the residual block, the output of the first 3×3 Conv layer of the residual block is input to the first ReLu layer of the residual block, the output of the first ReLu layer of the residual block and the input features are input together to the first 3D perception feature fusion module of the residual block, the output features of the first 3D perception feature fusion module of the residual block are input to the second 3×3 Conv layer of the residual block, the output of the second 3×3 Conv layer of the residual block and the input features are output together to the second 3D perception feature fusion module of the residual block, and the output of the second 3D perception feature fusion module of the residual block obtains the output features of the residual block.

[0039] The image recovery operation of the decoding layer is as follows:

[0040] The output of the residual feature extraction module and the output of the third layer encoding are input to the first 3D perception feature fusion module of the decoding layer. The output of the first 3D perception feature fusion module of the decoding layer is input to the first Dconv layer of the decoding layer. The output of the first Dconv layer of the decoding layer is input to the first InstanceNorm layer of the decoding layer. The output of the first InstanceNorm layer of the decoding layer is input to the first ReLu layer of the decoding layer. The output of the first ReLu layer of the decoding layer obtains the first layer of decoding features.

[0041] The first layer of decoding features and the output of the second encoding layer are input together to the second 3D perception feature fusion module of the decoding layer. The output of the second 3D perception feature fusion module of the decoding layer is input to the second Dconv layer of the decoding layer. The output of the second Dconv layer of the decoding layer is input to the second InstanceNorm layer of the decoding layer. The output of the second InstanceNorm layer of the decoding layer is input to the second ReLu layer of the decoding layer. The output of the second ReLu layer of the decoding layer obtains the second layer of decoding features.

[0042] The second layer decoding features and the output of the first encoding layer are input together to the third 3D perception feature fusion module of the decoding layer. The output of the third 3D perception feature fusion module of the decoding layer obtains the feature F1 extracted by the U-shaped sub-network.

[0043] The three-dimensional perception feature fusion module is as follows: Figure 2 As shown, it is divided into two stages: coarse fusion and fine fusion. The specific operation of coarse fusion is as follows:

[0044] Feature Map and feature map After the first Flatten operation of rough fusion, the corresponding output feature maps are obtained. The two feature maps output by the first Flatten operation of rough fusion are first subjected to the first Concat operation of rough fusion and then to the first adaptive global average pooling operation of rough fusion to obtain a feature vector. The feature vector is subjected to the first 1×1 Conv operation of rough fusion, the first ReLu operation of rough fusion and the first Sigmoid operation of rough fusion to obtain vectors C1 and C1. Vectors C1 and C1 are respectively associated with feature maps and feature map The feature map is obtained by the first multiplication of coarse fusion and feature map Feature Map and feature map The rough fusion output feature X is obtained by the first element-by-element addition of the rough fusion CF .

[0045] The detailed fusion operations are as follows:

[0046] Rough fusion output feature X CF The channel feature vector CI is obtained by sequentially undergoing the first adaptive global average pooling operation of fine fusion, the first 1×1 Conv operation of fine fusion, and the first ReLu operation of fine fusion. The channel feature vector CI is also subjected to two 1×1 Conv operations of fine fusion to obtain the channel feature vector CI1 and the channel feature vector CI2. The channel feature vector CI1 and the channel feature vector CI2 are respectively subjected to the broadcast mechanism of fine fusion to obtain the feature map C. r1 And feature map C r2 ;

[0047] Rough fusion output feature X CF The spatial feature map PI is obtained by the fourth 1×1 Conv operation of fine fusion and the second ReLu operation of fine fusion. The spatial feature map PI is also subjected to two 1×1 Conv operations of fine fusion to obtain the spatial feature maps PI1 and PI2. The spatial feature maps PI1 and PI2 are respectively subjected to the broadcast mechanism of fine fusion to obtain the feature map PI. r1 And feature map PI r2 ;

[0048] Channel feature vector C r1 and spatial feature map PI r1 After the first pixel-by-pixel addition of fine fusion, the pixel-level three-dimensional information matrix V1 is obtained;

[0049] Channel feature vector C r2 and spatial feature map PI r2 After the second pixel-by-pixel addition of fine fusion, the pixel-level three-dimensional information matrix V2 is obtained;

[0050] The pixel-level three-dimensional information matrix V1 and the pixel-level three-dimensional information matrix V2 are subjected to the first Concat operation of fine fusion to obtain a feature map, and the feature map is subjected to the first Softmax operation of fine fusion to obtain a pixel-level three-dimensional weight matrix and the pixel-level 3D weight matrix

[0051] Fusion Features and the pixel-level 3D weight matrix After the first pixel-by-pixel multiplication operation of fine fusion, the fused feature is obtained Weighted feature map;

[0052] Fusion Features and the pixel-level 3D weight matrix After the second pixel-by-pixel multiplication operation of fine fusion, the fusion feature is obtained Weighted feature map;

[0053] Fusion Features Weighted feature maps and fusion features The weighted feature map is subjected to the third pixel-by-pixel addition of fine fusion to obtain the fused feature V out .

[0054] Dynamic routing feature modules such as Figure 3 As shown, the details are as follows:

[0055] The feature maps F1, F2 and F3 are first subjected to a detach operation, and the results of the detach operation on the three feature maps are respectively input into the corresponding feature scoring units FSU1, FSU2 and FSU3. The features input into the feature scoring units are first passed through a feature extraction module, which is composed of a global average pooling layer, a 1×1 Conv layer and a ReLu layer connected in sequence. The feature extraction module outputs three feature vectors corresponding to the feature maps F1, F2 and F3 in sequence. The feature vectors are input into the feature extraction module, which is composed of a 1×1 Conv layer and a Sigmoid layer connected in sequence. The feature extraction module outputs feature scores corresponding to the feature maps F1, F2 and F3 in sequence. The three feature scores are subjected to a Concat operation and a Gumbel-Softmax function in sequence to obtain a discrete gating sequence.

[0056] The clear image reconstruction module is specifically:

[0057] The feature map is input to the first 3×3 Conv layer of the clear image reconstruction module, the output of the first 3×3 Conv layer of the clear image reconstruction module is input to the first InstanceNorm layer of the clear image reconstruction module, the output of the first InstanceNorm layer of the clear image reconstruction module is input to the first ReLu layer of the clear image reconstruction module, the output of the first ReLu layer of the clear image reconstruction module is input to the first ReflectionPad2d layer of the clear image reconstruction module, the output of the first ReflectionPad2d layer of the clear image reconstruction module is input to the second 3×3 Conv layer of the clear image reconstruction module, the output of the second 3×3 Conv layer of the clear image reconstruction module is input to the first Tanh layer of the clear image reconstruction module, and the output of the first Tanh layer of the clear image reconstruction module obtains the dehazed image.

[0058] Compared with the prior art, the present invention has the following technical effects:

[0059] 1) The present invention introduces a dynamic routing feature module by designing a well-designed dynamic routing feature framework (DRFF), and dynamically routes the features extracted from the U-shaped subnet and the full-resolution convolutional subnet with representative information, as well as the features fused from the two, so as to reconstruct a more ideal remote sensing image, thereby significantly improving the defogging effect, especially for small-scale objects under haze in remote sensing images;

[0060] 2) This paper proposes a 3D perception feature fusion module, exploring how to combine spatial and channel dimensions to effectively retain important global context for coarse fusion, and uses 3D interaction to generate pixel-level granular weight matrices through self-replication for final feature fusion. Therefore, the 3D perception feature fusion module can enhance the fused feature representation and improve the restoration of texture details and colors;

[0061] 3) This paper proposes a new method, called Dynamic Routing and 3D Fusion Remote Sensing Image Dehazing Network, which seamlessly integrates a well-designed dynamic routing feature framework and a 3D perception feature fusion module. Through more effective dynamic feature selection and fusion feature representation, experiments on challenging datasets such as SateHaze1k, RICE, and RSID show that the remote sensing haze removal method of this paper has superior performance compared with several state-of-the-art methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0063] Figure 1It is a structural diagram of the overall dynamic routing feature framework of an embodiment of the present invention;

[0064] Figure 2 for Figure 1 A schematic diagram of the structure of the 3D perception feature fusion module in FIG.

[0065] Figure 3 for Figure 1 Schematic diagram of the structure of the dynamic routing feature module in . DETAILED DESCRIPTION

[0066] like Figure 1 As shown, a defogging network based on dynamic routing and three-dimensional fusion includes a two-branch network, a dynamic routing feature module DRFM and a clear image reconstruction module CIRM;

[0067] The input end of the two-branch network is used to input the foggy image I, the output of the two-branch network is connected to the input of the dynamic routing feature module DRFM, the output of the dynamic routing feature module DRFM is connected to the input of the clear image reconstruction module CIRM, and the output of the clear image reconstruction module CIRM is used to output the dehazed image.

[0068] like Figure 1 As shown in the figure, the two-branch network includes a U-shaped sub-network U-Net, a full-resolution convolutional sub-network F-Net, and a three-dimensional perception feature fusion module;

[0069] The constructed U-shaped sub-network is specifically:

[0070] The foggy image I is input to the encoding layer 201, the output of the encoding layer 201 is input to the encoding layer 202, the output of the encoding layer 202 is input to the encoding layer 203, the output of the encoding layer 203 is input to the residual feature extraction module 204, the output of the residual feature extraction module 204 and the output of the encoding layer 203 are input to the three-dimensional perception feature fusion module 205 together, the output of the three-dimensional perception feature fusion module 205 is input to the decoding layer 206, the output of the decoding layer 206 and the output of the encoding layer 202 are input to the three-dimensional perception feature fusion module 207 together, the output of the three-dimensional perception feature fusion module 207 is input to the decoding layer 208, the output of the decoding layer 208 and the output of the encoding layer 201 are input to the three-dimensional perception feature fusion module 209 together, and the output feature of the three-dimensional perception feature fusion module 209 is recorded as F1.

[0071] Among them, the encoding layer feature extraction operation is as follows:

[0072] The foggy image I is input to the first ReflectionPad2d layer of the encoding layer, the output of the first ReflectionPad2d layer of the encoding layer is input to the first 7×7 Conv layer of the encoding layer, the output of the first 7×7 Conv layer of the encoding layer is input to the first InstanceNorm layer of the encoding layer, the output of the first InstanceNorm layer of the encoding layer is input to the first ReLu layer of the encoding layer, and the output of the first ReLu layer of the encoding layer obtains the first layer of encoding features;

[0073] The first layer of encoded features is input to the second 3×3 Conv layer of the encoding layer, the output of the second 3×3 Conv layer of the encoding layer is input to the second InstanceNorm layer of the encoding layer, the output of the second InstanceNorm layer of the encoding layer is input to the second ReLu layer of the encoding layer, and the output of the second ReLu layer of the encoding layer obtains the second layer of encoded features;

[0074] The second layer of encoded features is input to the third 3×3 Conv layer of the encoding layer, the output of the third 3×3 Conv layer of the encoding layer is input to the third InstanceNorm layer of the encoding layer, the output of the third InstanceNorm layer of the encoding layer is input to the third ReLu layer of the encoding layer, and the output of the third ReLu layer of the encoding layer obtains the third layer of encoded features.

[0075] The residual feature extraction module consists of 6 residual blocks connected in series. The residual block feature extraction is as follows:

[0076] The input features are input to the first 3×3 Conv layer of the residual block, the output of the first 3×3 Conv layer of the residual block is input to the first ReLu layer of the residual block, the output of the first ReLu layer of the residual block and the input features are input to the first 3D perception feature fusion module of the residual block together, the output features of the first 3D perception feature fusion module of the residual block are input to the second 3×3 Conv layer of the residual block, the output of the second 3×3 Conv layer of the residual block and the input features are output to the second 3D perception feature fusion module of the residual block together, and the output of the second 3D perception feature fusion module of the residual block obtains the output features of the residual block.

[0077] The decoding layer image recovery operation is as follows:

[0078] The output of the residual feature extraction module 204 and the output of the encoding layer 203 are input to the first 3D perception feature fusion module 205 of the decoding layer together, the output of the 3D perception feature fusion module 205 of the decoding layer is input to the first Dconv layer of the decoding layer, the output of the first Dconv layer of the decoding layer is input to the first InstanceNorm layer of the decoding layer, the output of the first InstanceNorm layer of the decoding layer is input to the first ReLu layer of the decoding layer, and the output of the first ReLu layer of the decoding layer obtains the first layer of decoding features;

[0079] The first layer of decoding features and the output of the second encoding layer 202 are input together to the second 3D perception feature fusion module 207 of the decoding layer, the output of the second 3D perception feature fusion module 207 of the decoding layer is input to the second Dconv layer of the decoding layer, the output of the second Dconv layer of the decoding layer is input to the second InstanceNorm layer of the decoding layer, the output of the second InstanceNorm layer of the decoding layer is input to the second ReLu layer of the decoding layer, and the output of the second ReLu layer of the decoding layer obtains the second layer of decoding features;

[0080] The second layer decoding features and the output of the first encoding layer 201 are input together to the third 3D perception feature fusion module 209 of the decoding layer, and the output of the third 3D perception feature fusion module 209 of the decoding layer obtains the feature F1 extracted by the U-shaped sub-network.

[0081] The proposed 3D perception feature fusion module is as follows Figure 2 As shown, it is divided into two stages: coarse fusion and fine fusion. The specific operation of coarse fusion is as follows:

[0082] Feature Map and feature map After the first Flatten operation, the output feature map 702 and the output feature map 703 are obtained. After the first Concat operation of the coarse fusion, the output feature map 702 and the output feature map 703 are obtained. After the first adaptive global average pooling operation of the coarse fusion, the feature map 705 is obtained to obtain the feature vector 706. The feature vector 706 is successively subjected to the first 1×1 Conv operation, the first ReLu operation and the first Sigmoid operation of the coarse fusion to obtain the vector C1 and the vector C2. The vector C1 and the vector C2 are respectively connected to the feature map and feature map The feature map is obtained by the first multiplication of coarse fusion and feature map Feature Map and feature map The rough fusion output feature X is obtained by the first element-by-element addition of the rough fusion CF .

[0083] The detailed fusion operations are as follows:

[0084] Rough fusion output feature X CF The channel feature vector CI is obtained by sequentially undergoing the first adaptive global average pooling operation, the first 1×1 Conv operation, and the first ReLu operation of fine fusion. The channel feature vector CI is also subjected to two 1×1 Conv operations of fine fusion to obtain the channel feature vector CI1 and the channel feature vector CI2. The channel feature vector CI1 and the channel feature vector CI2 are respectively subjected to the broadcast mechanism of fine fusion to obtain the feature map C. r1 And feature map C r2 ;

[0085] Rough fusion output feature X CF The spatial feature map PI is obtained by the fourth 1×1 Conv operation and the second ReLu operation of fine fusion. The spatial feature map PI is also subjected to two 1×1 Conv operations of fine fusion to obtain the spatial feature maps PI1 and PI2. The spatial feature maps PI1 and PI2 are respectively finely fused and broadcasted to obtain the feature map PI. r1 And feature map PI r2 ;

[0086] Channel feature vector C r1 and spatial feature map PI r1 After the first pixel-by-pixel addition of fine fusion, the pixel-level three-dimensional information matrix V1 is obtained;

[0087] Channel feature vector C r2 and spatial feature map PI r2 After the second pixel-by-pixel addition of fine fusion, the pixel-level three-dimensional information matrix V2 is obtained;

[0088] The pixel-level three-dimensional information matrix V1 and the pixel-level three-dimensional information matrix V2 are subjected to the first Concat operation of fine fusion to obtain a feature map 724, and the feature map 724 is subjected to the first Softmax operation of fine fusion to obtain a pixel-level three-dimensional weight matrix and the pixel-level 3D weight matrix

[0089] Fusion Features and the pixel-level 3D weight matrix After the first pixel-by-pixel multiplication operation of fine fusion, a weighted feature map 727 is obtained;

[0090] Fusion Features and the pixel-level 3D weight matrix After the second pixel-by-pixel multiplication operation of fine fusion, a weighted feature map 728 is obtained;

[0091] The weighted feature map 727 and the weighted feature map 728 are subjected to the third pixel-by-pixel addition of fine fusion to obtain the fused feature V out .

[0092] The constructed full-resolution sub-network F-Net is specifically:

[0093] The foggy image I is input to the channel change module 301, the output of the channel change module 301 is input to the residual feature extraction module 302, and the output of the residual feature extraction module 302 obtains the feature F2 extracted by the full-resolution sub-network.

[0094] The specific operations of the channel changing module 301 are:

[0095] The foggy image I is input to the first ReflectionPad2d layer of the channel change module 301, the output of the first ReflectionPad2d layer of the channel change module 301 is input to the first 7×7 Conv layer of the channel change module 301, the output of the first 7×7 Conv layer of the channel change module 301 is input to the first InstanceNorm layer of the channel change module 301, and the output of the first InstanceNorm layer of the channel change module 301 is input to the first ReLu layer of the channel change module 301;

[0096] The specific operation of the residual feature extraction module 302 is the same as the operation of the residual feature extraction module 204 .

[0097] The dynamic feature routing module constructed is as follows Figure 3 As shown, the details are as follows:

[0098] The feature maps F1, F2 and F3 are first subjected to a detach operation 501. The results of the three feature maps after the detach operation 501 are respectively input into the corresponding feature scoring units FSU1, FSU2 and FSU3. The features input into the feature scoring units are first subjected to a feature extraction module 505, which is composed of a global average pooling layer, a 1×1 Conv layer and a ReLu layer connected in sequence. The feature extraction module 505 outputs three feature vectors 506 corresponding to the feature maps F1, F2 and F3 in sequence. The feature vectors 506 are input into a feature extraction module 507, which is composed of a 1×1 Conv layer and a Sigmoid layer connected in sequence. The feature extraction module 507 outputs feature scores 508 corresponding to the feature maps F1, F2 and F3 in sequence. The three feature scores 508 are subjected to a Concat operation 509 and a Gumbel-Softmax function 510 in sequence to obtain a discrete gating sequence 511.

[0099] The clear image reconstruction module is specifically:

[0100] The feature map is input to the first 3×3 Conv layer of the clear image reconstruction module, the output of the first 3×3 Conv layer of the clear image reconstruction module is input to the first InstanceNorm layer of the clear image reconstruction module, the output of the first InstanceNorm layer of the clear image reconstruction module is input to the first ReLu layer of the clear image reconstruction module, the output of the first ReLu layer of the clear image reconstruction module is input to the first ReflectionPad2d layer of the clear image reconstruction module, the output of the first ReflectionPad2d layer of the clear image reconstruction module is input to the second 3×3 Conv layer of the clear image reconstruction module, the output of the second 3×3 Conv layer of the clear image reconstruction module is input to the first Tanh layer of the clear image reconstruction module, and the output of the first Tanh layer of the clear image reconstruction module obtains the dehazed image.

[0101] In step S4, four types of loss constraints are used during training, as follows:

[0102] (1) SmoothL1 loss, the specific formula is:

[0103]

[0104]

[0105] G(χ j ) represents the pixel value at position i of the input image and then the value obtained by the defogging network parameter operation, and P represents the number of pixels in the image. j represents the value of pixel j in the foggy image, z j represents the value of the ground truth (GT) pixel j.

[0106] (2) Adversarial loss. The specific definition of adversarial loss is as follows:

[0107]

[0108] D represents the generator and G represents the discriminator. D(G(x)) represents the probability of judging the dehazed image as a clear image, and m represents the total number of images.

[0109] (3) Perceptual loss, which calculates the L2 norm of the VGG16 pre-trained model for GT and haze-free images from the network at different layers. The perceptual loss is defined as follows:

[0110]

[0111] where η irepresents the size of the feature map obtained from the i-th layer of the pre-trained VGG16 model. P represents the number of feature layers used for perceptual loss. X and Y represent the hazy image and the clear haze-free image, respectively.

[0112] (4) Multi-scale structural similarity loss (MS-SSIM), in order to make the structural similarity between the dehazed image and the clear image closer, the multi-scale structural similarity loss (MS-SSIM) is introduced, which is specifically expressed as follows:

[0113]

[0114] in and are the mean values ​​of the dehazed image and GT respectively. ij is the covariance between the dehazed image and GT, η i and η j is the standard deviation between the dehazed image and GT, and C is a constant. The loss function of the overall network is expressed as:

[0115] L=θ1L smoothL1 +θ2L adv +θ3L perc +θ4L ms-ssim

[0116] Where θ1, θ2, θ3, θ4 are the hyperparameters of each loss function.

[0117] When the model is working, the process is:

[0118] like Figure 1 As shown, the foggy image I is input to the dynamic routing feature framework, which includes a dual-branch network, a dynamic routing feature module, and a clear image reconstruction module. The dual-branch network is composed of an upper branch U-shaped subnet and a lower branch full-resolution convolutional subnet in parallel. The two subnets extract and fuse three features with different feature information (the U-shaped subnet extracts feature F1, the full-resolution convolutional subnet extracts feature F2, and the features extracted by the two subnets are fused through three-dimensional perception features to obtain feature F3). The three features are scored by the feature scoring module, and the feature with the highest score is selected by the Gumbel-Softmax gating function. A clear remote sensing image is reconstructed through the clear image reconstruction module.

[0119] The upper branch U-shaped subnet encoding layer feature extraction module uses 3 convolution operations to perform 4 times downsampling for feature extraction. The first layer of the encoding stage consists of ReflectionPad2d layer, convolution layer, activation layer, and InstanceNorm, and the second and third layers of the encoding stage consist of convolution layer, activation layer, and InstanceNorm. Secondly, 6 consecutive residual blocks are used to further enhance the feature extraction on the extracted low-resolution features. Finally, the features extracted by the continuous residual blocks containing a large amount of semantic scene information are decoded and reconstructed using deconvolution and convolution operations. During this period, the corresponding encoding and decoding layer features are fused using the three-dimensional perception feature fusion module, and finally the feature F1 extracted by the U-shaped subnet with the original image resolution and 64 channels is obtained. The lower branch full-resolution convolution subnetwork consists of a convolution block and 6 consecutive residual blocks. The convolution block consists of a ReflectionPad2d layer, a 7×7 Conv layer, InstanceNorm, and ReLu, which expands the channels of the input image to 64 channels. The combination of 6 consecutive residual blocks further enhances the feature extraction, and finally obtains the feature F2 extracted by the upper branch full-resolution subnetwork. In addition, the features obtained by the two subnetworks are fused through the 3D perception feature fusion module to obtain the feature F3. The formula is expressed as follows:

[0120] F1=Unet(I)

[0121] F2=Fnet(I)

[0122] F3=3DPFF(F1,F2)

[0123] Among them, F1 is the feature extracted by U-Net, which contains rich contextual information, F2 is the feature extracted by F-Net without downsampling, which can retain more high-frequency detailed information, and F3 is the feature obtained by fusing F1 and F2 through the 3D perception feature fusion module.

[0124] Construct a 3D perception feature fusion module, such as Figure 2 As shown in Figure 1, the 3D-aware feature fusion module includes two stages: coarse fusion and fine fusion. Coarse fusion first aggregates spatial and channel information by adaptive global average pooling of two fused features, thereby learning the global context information of each input feature. Subsequently, fine-grained fusion realizes 3D interaction across spatial and channel dimensions through self-replication, generating pixel-level granular weight matrices assigned to corresponding input features for the second fusion. This two-stage fusion enhances the representation of the fused features and improves the restoration of texture details and colors.

[0125] like Figure 3 As shown in Figure 1, the feature scoring module consists of three independent scoring units, which are used to specifically evaluate each feature sample in F1, F2 and F3. The evaluation process can be expressed as:

[0126]

[0127] In the formula, F j Represents the score corresponding to each sample in the three features, bs represents the number of samples in the batch, and FSU() is the feature scoring unit. Specifically, the evaluation process consists of two steps. First, the spatial information is aggregated using global average pooling, and is abstracted into a contribution vector using learnable convolution, which enhances the robustness to spatial changes; then, the contribution vector is compressed using 1*1 convolution, and the Sigmoid activation function is used to map it to a contribution score. Next, the Gumbel-Softmax function is used to select the features that contribute most to the reconstruction of the haze-free image. The feature scores S1, S2, and S3 are passed to the GumbelSoftmax function, which maps them to a binary vector with values ​​ranging from 0 to 1 (only one element is "1", representing the selected feature). Then, the vector is multiplied by the corresponding feature to complete the feature routing. The formula is expressed as follows:

[0128]

[0129] Among them, F represents the feature map selected and passed to the subsequent network, g() represents the Gumbel-Softmax function, which allows the network to optimize this discrete decision through back-propagation. This function is used after the feature scoring module, enabling it to adaptively update the scoring strategy through back-propagation.

[0130] Example

[0131] In order to facilitate those skilled in the art to better understand the present invention, further description is as follows:

[0132] 1. Parameter settings

[0133] All experiments in this study were conducted on an NVIDIA RTX3090 GPU, and the code was implemented using PyTorch. The proposed network was optimized using the Adam optimizer, and the momentum decay index was set to β1=0.9 and β2=0.999. The learning rate and batch size were configured to 0.0001 and 4, respectively. The initial learning rate was set to 0.001, and MultiStepLR was used to dynamically adjust between the two. Throughout the training process, the parameters of the loss function in the network were set to θ1=1, θ2=0.0005, θ3=0.01, and θ4=0.5. In addition, the present invention evaluates various dehazing algorithms on the synthetic datasets SateHaze1k and RSID, as well as the real dataset RICE. The SateHaze1k dataset consists of three subsets: thin, moderate, and thick. Each subset contains 400 pairs of synthetic remote sensing images, of which 320 pairs are training sets, 35 pairs are validation sets, and 45 pairs are test sets. The resolution of each image is 512*512 pixels. The RSID dataset contains 1000 pairs of synthetic remote sensing images with a resolution of 256*256 pixels. 900 pairs are randomly selected for training and the rest are tested. The RICE dataset contains two subsets: RICE1 and RICE2, which are real-world remote sensing datasets. The RICE1 dataset consists of 500 pairs of 512*512 resolution images from Google Earth. The RICE2 dataset consists of 450 pairs of 512*512 resolution Landsat 8OLI / TIRS satellite images. The entire RICE1 dataset is randomly divided into training and test sets at a ratio of 9:1, and the RICE2 dataset is similarly divided. In order to verify the correctness and effectiveness of the method of the present invention, the current excellent defogging algorithms are added for comparison with the method of the present invention. These methods are as follows: DCP, FFA, FCTF, TBN, SGID, DCI-Net, MFINEA, Trinity-Net, PSMB-Net.

[0134] 2. Experimental results

[0135] Table 1 Comparison with various excellent image dehazing methods on the SateHaze1k dataset

[0136]

[0137] Table 2 Comparison with various excellent image dehazing methods on RICE and RSID datasets

[0138]

[0139] The present invention achieved the first place in PSNR and SSIM on the Satehaze1k, RSID and RICE datasets. As shown in Tables 1, 2 and 3, the end-to-end haze removal method shows superior performance compared with the prior-based method. Compared with U-Net-based dehazing networks such as MFINEA and PSMB-Net, the present invention has achieved significant improvements on the Satehaze1k dataset. Among them, the average PSNR is improved by 1.41dB and 0.84dB, and the SSIM is improved by 0.028dB and 0.005dB. On the RSID dataset, the present invention showed better results, with PSNR improved by 3.8dB and 2.7dB, and SSIM improved by 0.003 and 0.005, respectively. Compared with F-Net-based dehazing networks such as FFA, the PSNR and SSIM of the present invention on Satehaze1k are improved by 1.41dB and 0.032, respectively, and RSID is improved by 3.6dB and 0.004, respectively. The present invention achieves state-of-the-art performance on both RICE1 and RICE2 datasets. Compared with FFA, DCI-Net and PSMB-Net, the present invention has significant improvements on the RICE1 dataset. PSNR is improved by 2.32dB, 2.30dB and 2.65dB, and SSIM is improved by 0.014, 0.005 and 0.002, respectively. On the RICE2 dataset, the present invention continues to outperform DCI-Net and PSMB-Net, with PSNR improved by 3.28dB and 1.46dB, and SSIM improved by 0.012 and 0.005, respectively.

[0140] 3. Ablation Analysis

[0141] In order to evaluate the effectiveness of each module of the invention, the present invention designed an ablation experiment according to the innovation of the dynamic routing feature framework and the innovation of attention. There are 5 experiments in total: 1) F-Net: only F-Net subnet; 2) U-Net: only U-Net subnet; 3) base: F-Net+U-Net; 4) base+DFRF; 5) base+DFRF+3D-PFF. Experiments 1-4 use Element-Wise Addition for feature fusion. Ablation experiments were performed on the SateHaze1k dataset.

[0142] Table 3 Ablation experiment results on the SateHaze1k dataset

[0143]

[0144] The results of experiments (1) and (2) show that the haze removal performance of a single U-Net or F-Net is limited due to the loss of high-frequency information in U-Net and the limited receptive field of F-Net. This experiment builds a baseline model by deploying a dual-branch network consisting of U-Net and F-Net, and achieves scores of 25.35dB and 0.910 on the baseline model. On this basis, DFRF is developed, which significantly enhances the PSNR and SSIM indicators. DFRF can dynamically route the extracted and fused features of U-Net and F-Net, effectively utilizing the advantages of the two models in large receptive field and detail preservation, thereby achieving effective haze removal and restoration of small-scale objects. In Experiment 5, the 3D perception feature fusion module is introduced into the network to enhance the expression ability of the fused features within the network, thereby further improving the texture details and color of the dehazed image. The indicators on the Thin, Moderate and Thick subsets have been significantly improved, with an improvement of 1.22dB and 0.011 on the Thin sub-dataset, 2.04dB and 0.017 on the Moderate sub-dataset, and 0.92dB and 0.009 on the Thick sub-dataset.

[0145] The present invention provides a dual-branch remote sensing image defogging network method based on dynamic routing and three-dimensional fusion, which includes a dynamic routing feature framework and a three-dimensional perception feature fusion module. The dynamic routing feature framework dynamically routes features containing multiple representative feature information extracted by U-Net and F-Net, thereby significantly improving the haze removal and reconstruction of small-scale objects under haze. The three-dimensional perception feature fusion module enhances the interaction between the spatial information and channel information of multiple features, assigns pixel-level weights to feature fusion, and improves the texture details and color of the defogging image.

Claims

1. A defogging network based on dynamic routing and three-dimensional fusion, characterized in that: The network includes a two-branch network (1), a dynamic routing feature module DRFM (5) and a clear image reconstruction module CIRM (6); The input end of the two-branch network (1) is used to input a foggy image I, the output of the two-branch network (1) is connected to the input of the dynamic routing feature module DRFM (5), the output of the dynamic routing feature module DRFM (5) is connected to the input of the clear image reconstruction module CIRM (6), and the output of the clear image reconstruction module CIRM (6) is used to output a dehazed image.

2. The network according to claim 1, characterized in that The two-branch network (1) includes a U-shaped sub-network (U-Net) (2), a full-resolution convolutional sub-network (F-Net) (3), and a three-dimensional perception feature fusion module; The U-shaped subnetwork U-Net (2) is as follows: the input of the first coding layer (201) is used to input the foggy image I, the output of the first coding layer (201) is connected to the input of the second coding layer (202), the output of the second coding layer (202) is connected to the input of the third coding layer (203), the output of the third coding layer (203) is connected to the input of the residual feature extraction module (204), the output of the residual feature extraction module (204) and the output of the third coding layer (203) are connected together with the input of the first three-dimensional perception feature fusion module (205); the first three-dimensional perception feature fusion module The output of the module (205) is connected to the input of the first decoding layer (206), the output of the first decoding layer (206) and the output of the second encoding layer (202) are connected to the input of the second three-dimensional perception feature fusion module (207), the output of the second three-dimensional perception feature fusion module (207) is connected to the input of the second decoding layer (208), the output of the second decoding layer (208) and the output of the first encoding layer are connected to the input of the third three-dimensional perception feature fusion module (209), and the output feature of the third three-dimensional perception feature fusion module (209) is recorded as feature F1 (401).

3. The network according to claim 2, characterized in that The full-resolution convolutional subnetwork F-Net(3) is specifically: The input of the channel changing module (301) is used to input the foggy image I, the output of the channel changing module (301) is connected to the input of the residual feature extraction module (302), and the output feature of the residual feature extraction module (302) is feature F2 (402); The specific operations of the channel changing module (301) are as follows: The foggy image I is input to the first ReflectionPad2d layer of the channel change module, the output of the first ReflectionPad2d layer of the channel change module is input to the first Conv layer of the channel change module, the output of the first Conv layer of the channel change module is input to the first InstanceNorm layer of the channel change module, and the output of the first InstanceNorm layer of the channel change module is input to the first ReLu layer of the channel change module.

4. The network according to claim 3, characterized in that Feature F1 (401) and feature F2 (402) are fused with the three-dimensional perception feature (210) to obtain feature F3 (403); then, the dynamic routing feature module DRFM (5) scores the features F1 (401), F2 (402), and F3 (403) and selects the most suitable one for reconstructing a clear image to calculate the loss.

5. The network according to claim 4, characterized in that The process of network training is constrained by four losses. The four loss constraints are as follows: (1) SmoothL1 loss, the specific formula is: G(χ j ) represents the pixel value at position i of the input image and then the value obtained by the defogging network parameter operation, P represents the number of pixels in the image; χ j represents the value of pixel j in the foggy image, z j represents the value of ground truth (GT) pixel j; (2) Adversarial loss. The specific definition of adversarial loss is as follows: D represents the generator, G represents the discriminator, D(G(x)) represents the probability of judging the dehazed image as a clear image, and m represents the total number of images; (3) Perceptual loss, which calculates the L2 norm of the VGG16 pre-trained model for GT and haze-free images from the network at different layers. The perceptual loss is defined as follows: where η i represents the size of the feature map obtained from the i-th layer of the pre-trained VGG16 model, P represents the number of feature layers used for perceptual loss, X and Y represent foggy images and clear fog-free images, respectively; (4) Multi-scale structural similarity loss (MS-SSIM). In order to make the structural similarity between the dehazed image and the clear image closer, the multi-scale structural similarity loss (MS-SSIM) is introduced. The specific expression is as follows: in and are the mean values ​​of the dehazed image and GT, respectively, ij is the covariance between the dehazed image and GT, η i and η j is the standard deviation of the dehazed image and GT, C is a constant, and the loss function of the overall network is expressed as: L=θ1L smoothL1 +θ2L adv +θ3L perc +θ4L ms-ssim Where θ1, θ2, θ3, θ4 are the hyperparameters of each loss function.

6. The network according to claim 2, characterized in that The feature extraction operation of the encoding layer is as follows: The feature is input to the first ReflectionPad2d layer of the encoding layer, the output of the first ReflectionPad2d layer of the encoding layer is input to the first Conv layer of the encoding layer, the output of the first Conv layer of the encoding layer is input to the first InstanceNorm layer of the encoding layer, the output of the first InstanceNorm layer of the encoding layer is input to the first ReLu layer of the encoding layer, and the output of the first ReLu layer of the encoding layer obtains the first layer of encoding features; The first layer of encoded features is input into the second Conv layer of the encoding layer, the output of the second Conv layer of the encoding layer is input into the second InstanceNorm layer of the encoding layer, the output of the second InstanceNorm layer of the encoding layer is input into the second ReLu layer of the encoding layer, and the output of the second ReLu layer of the encoding layer obtains the second layer of encoded features; The second layer of encoding features is input to the third Conv layer of the encoding layer, the output of the third Conv layer of the encoding layer is input to the third InstanceNorm layer of the encoding layer, the output of the third InstanceNorm layer of the encoding layer is input to the third ReLu layer of the encoding layer, and the output of the third ReLu layer of the encoding layer obtains the third layer of encoding features.

7. The network according to claim 2, characterized in that The residual feature extraction module consists of 6 residual blocks connected in series. The feature extraction of the residual block is as follows: The input features are input to the first Conv layer of the residual block, the output of the first Conv layer of the residual block is input to the first ReLu layer of the residual block, the output of the first ReLu layer of the residual block and the input features are input to the first 3D perception feature fusion module of the residual block together, the output features of the first 3D perception feature fusion module of the residual block are input to the second Conv layer of the residual block, the output of the second Conv layer of the residual block and the input features are output to the second 3D perception feature fusion module of the residual block together, and the output of the second 3D perception feature fusion module of the residual block obtains the output features of the residual block.

8. The network according to claim 2, characterized in that The image recovery operation of the decoding layer is as follows: The output of the residual feature extraction module (204) and the output of the third layer encoding (203) are input to the first three-dimensional perception feature fusion module (205), the output of the first three-dimensional perception feature fusion module (205) is input to the first Dconv layer of the decoding layer, the output of the first Dconv layer of the decoding layer is input to the first InstanceNorm layer of the decoding layer, the output of the first InstanceNorm layer of the decoding layer is input to the first ReLu layer of the decoding layer, and the output of the first ReLu layer of the decoding layer obtains the first layer of decoding features; The first layer of decoding features and the output of the second encoding layer (202) are input together to a second three-dimensional perception feature fusion module (207), the output of the second three-dimensional perception feature fusion module (207) is input to a second Dconv layer of the decoding layer, the output of the second Dconv layer of the decoding layer is input to a second InstanceNorm layer of the decoding layer, the output of the second InstanceNorm layer of the decoding layer is input to a second ReLu layer of the decoding layer, and the output of the second ReLu layer of the decoding layer obtains a second layer of decoding features; The second layer decoding features and the output of the first encoding layer (201) are input together to the third three-dimensional perception feature fusion module (209), and the output of the third three-dimensional perception feature fusion module (209) obtains the U-shaped sub-network extraction feature F1 (401).

9. The network according to claim 2, characterized in that The 3D perception feature fusion module is divided into two stages: coarse fusion and fine fusion. The specific operation of coarse fusion is as follows: Feature Map (701) and feature map (701) are subjected to the first Flatten operation of the rough fusion to obtain the output feature map (702) and the output feature map (703), the output feature map (702) and the output feature map (703) are subjected to the first Concat operation of the rough fusion to obtain the feature map (705), the feature map (705) is subjected to the first adaptive global average pooling operation of the rough fusion to obtain the feature vector (706), the feature vector (706) is subjected to the first Conv operation of the rough fusion, the first ReLu operation of the rough fusion and the first Sigmoid operation of the rough fusion to obtain the vector C1 (707) and the vector C1 (708), the vector C1 (707) and the vector C1 (708) are respectively associated with the feature map (701) and feature map (701) The feature map is obtained by the first multiplication of coarse fusion (709) and feature maps (710),Feature Map (709) and feature maps (710) The rough fusion output feature X is obtained by the first element-by-element addition of the rough fusion CF (711); The detailed fusion operations are as follows: Rough fusion output feature X CF (711) After the first adaptive global average pooling operation of fine fusion, the first Conv operation of fine fusion and the first ReLu operation of fine fusion, the channel feature vector CI (712) is obtained. The channel feature vector CI (712) is also subjected to two Conv operations of fine fusion to obtain the channel feature vector CI1 (713) and the channel feature vector CI2 (714). The channel feature vector CI1 (713) and the channel feature vector CI2 (714) are respectively subjected to the broadcast mechanism of fine fusion to obtain the feature map C r1 (715) and feature map C r2 (716); Rough fusion output feature X CF (711) After the fourth Conv operation of fine fusion and the second ReLu operation of fine fusion, the spatial feature map PI (717) is obtained. The spatial feature map PI (717) is also subjected to two Conv operations of fine fusion to obtain the spatial feature map PI1 (718) and the spatial feature map PI2 (719). The spatial feature map PI1 (718) and the spatial feature map PI2 (719) are respectively subjected to the broadcast mechanism of fine fusion to obtain the feature map PI r1 (720) and feature map PI r2 (721); Channel feature vector C r1 (715) and spatial feature map PI r1 (720) After the first pixel-by-pixel addition of fine fusion, a pixel-level three-dimensional information matrix V1 (722) is obtained; Channel feature vector C r2 (716) and spatial feature map PI r2 (721) After the second pixel-by-pixel addition of fine fusion, the pixel-level three-dimensional information matrix V2 (723) is obtained; The pixel-level three-dimensional information matrix V1 (722) and the pixel-level three-dimensional information matrix V2 (723) are subjected to the first Concat operation of fine fusion to obtain a feature map (724), and the feature map (724) is subjected to the first Softmax operation of fine fusion to obtain a pixel-level three-dimensional weight matrix (725) and the pixel-level 3D weight matrix (726); Fusion Features (701) and the pixel-level three-dimensional weight matrix (725) After the first pixel-by-pixel multiplication operation of fine fusion, a weighted feature map (727) is obtained; Fusion Features (704) and the pixel-level three-dimensional weight matrix (726) After the second pixel-by-pixel multiplication operation of fine fusion, a weighted feature map (728) is obtained; The weighted feature map (727) and the weighted feature map (728) are subjected to the third pixel-by-pixel addition of fine fusion to obtain the fused feature V out (729).

10. The network according to claim 2, characterized in that The dynamic routing feature modules are as follows: The feature map F1 (401), the feature map F2 (402) and the feature map F3 (403) are first subjected to a detach operation (501). The results of the three feature maps after the detach operation are respectively input into the corresponding feature scoring unit FSU1 (502), the feature scoring unit FSU2 (504) and the feature scoring unit FSU3 (503). The features input into the feature scoring unit are first subjected to a feature extraction module (505), which is composed of a global average pooling layer, a Conv layer and a ReLu layer connected in sequence. The feature extraction module (505) outputs three feature maps F1 (401), F2 (402) and F3 (403). The feature vectors (506) corresponding to the feature graphs F2 (402) and F3 (403) are input to a feature extraction module (507), which is formed by sequentially connecting a Conv layer and a Sigmoid layer. The feature extraction module (507) outputs feature scores (508) corresponding to the feature graphs F1 (401), F2 (402) and F3 (403) respectively. The three feature scores (508) are sequentially subjected to a Concat operation (509) and a Gumbel-Softmax function (510) to obtain a discrete gating sequence (511). The clear image reconstruction module is specifically: The feature map is input to the first Conv layer of the clear image reconstruction module, the output of the first Conv layer of the clear image reconstruction module is input to the first InstanceNorm layer of the clear image reconstruction module, the output of the first InstanceNorm layer of the clear image reconstruction module is input to the first ReLu layer of the clear image reconstruction module, the output of the first ReLu layer of the clear image reconstruction module is input to the first ReflectionPad2d layer of the clear image reconstruction module, the output of the first ReflectionPad2d layer of the clear image reconstruction module is input to the second Conv layer of the clear image reconstruction module, the output of the second Conv layer of the clear image reconstruction module is input to the first Tanh layer of the clear image reconstruction module, and the output of the first Tanh layer of the clear image reconstruction module obtains the dehazed image.

Images