A construction method of an embedded U-shaped remote sensing defogging network

Abstract

The application provides a construction method of an embedded U-shaped remote sensing defogging network, comprising the following steps: S1, constructing a U-shaped image defogging network; S2, constructing a plurality of parallel embedded U-shaped network frameworks added between a U-shaped network coding layer and a decoding layer to extract multi-scale feature information; S3, constructing an adaptive fusion mechanism capable of adaptively fusing multi-scale features extracted from different hierarchical embedded U-shaped network modules; S4, sending a remote sensing image with fog into the U-shaped image defogging network, finally outputting a clear fog-free image through the plurality of parallel embedded U-shaped network frameworks and the cross-level feature adaptive fusion module, and finally calculating a loss by using the output clear image to constrain the training of the network, and constructing the embedded U-shaped remote sensing defogging network based on the cross-level feature adaptive fusion of the U-shaped network through the above steps, so that the defogging effect of the defogging network can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a method for constructing an embedded U-shaped remote sensing dehazing network. Background Technology

[0002] With the advancement of remote sensing technology, remote sensing images are widely used in environmental, meteorological, ground-based observation, marine monitoring, disaster monitoring, and aerospace fields. However, during image acquisition, remote sensing satellites and drones are affected by tiny particles such as smog and dust in the atmosphere, leading to problems such as blurred images, reduced contrast, color distortion, and other quality degradation. Therefore, how to effectively remove haze information from remote sensing images and restore image details has attracted considerable attention from researchers.

[0003] Currently, image dehazing methods are mainly divided into two categories: prior-based dehazing methods and deep learning-based dehazing methods. Most early dehazing work was developed based on classical atmospheric scattering models. Physical models based on atmospheric scattering models have been used for image dehazing, such as Dark Channel Prior (DCP) and Color Attenuation (CAP) priors. In recent years, deep learning-based methods have been widely used in image dehazing, image classification, and face recognition. In the field of image dehazing, Convolutional Neural Networks (CNNs) have been widely used for natural image dehazing and have achieved significant success. Among them, many researchers have made improvements and innovations based on the traditional U-shaped network, striving to improve the performance and effectiveness of deep learning algorithms. Initially, researchers improved the architecture of the U-shaped network by adding skip connections and improving the model's expressive power. Although these methods have achieved certain dehazing effects on remote sensing images, they still have the following problems when effectively fusing U-shaped networks with features from different levels: Most existing dehazing network models fail to fully utilize the feature information extracted by the encoding layer in the decoding layer, resulting in feature dilution in the network model and unsatisfactory effects in the dehazed remote sensing images.

[0004] Most dehazing algorithms ignore the interaction of cross-level feature information, resulting in overly redundant fused features, which in turn affects scene restoration and the constraints of contextual information in the image.

[0005] Therefore, there is an urgent need to provide methods for dehazing networks that can achieve better restoration results on real fog, non-uniform fog images, and remote sensing images with large scene changes. Summary of the Invention The purpose of this invention is to solve the technical problems mentioned above, and to propose a method for constructing an embedded U-shaped remote sensing defogging network, comprising the following steps: S1. Construct a U-shaped image dehazing network; S2. Construct several parallel embedded U-shaped network frameworks and add them between the encoding and decoding layers of the U-shaped network to extract multi-scale feature information; S3. Construct an adaptive fusion mechanism that can adaptively fuse multi-scale features extracted from embedded U-shaped network modules at different levels. S4. The foggy remote sensing image is fed into the U-shaped image dehazing network. Through several parallel embedded U-shaped network frameworks and cross-level feature adaptive fusion modules, a clear fog-free image is finally output. Finally, the loss is calculated using the output clear image to constrain the training of the network. The above steps are used to construct an embedded U-shaped remote sensing dehazing network based on cross-level feature adaptive fusion.

[0006] In the preferred embodiment, the U-shaped image dehazing network in step S1 includes a coding layer feature extraction module, a feature repair module, a decoding layer image restoration module, a parallel embedded U-shaped framework, and a cross-level feature adaptive fusion module CHAF.

[0007] In the preferred embodiment, in step S1, the U-shaped image dehazing network structure is constructed as follows: First layer of the coding layer feature extraction module →Feature extraction of the second layer of the coding layer →Feature extraction of the coding layer, third layer →Feature extraction of the fourth layer of the coding layer ; First layer of the coding layer feature extraction module →The embedded U-shaped network framework EU1→outputs three features E11, E12, and E13; Second layer of the coding layer feature extraction module →The embedded U-shaped network framework EU2→outputs three features E21, E22, and E23; The third layer of the coding layer feature extraction module →The embedded U-shaped network framework EU3→outputs three features E31, E32, and E33; Features E33, E23, E13 → Cross-level feature adaptive fusion module CHAF, decoding and recovery layer →First deconvolutional layer→fusion operation→channel attention module CA→pixel attention module PA→decoding and recovery layer ; Features E32, E22, E12 → Cross-level feature adaptive fusion module CHAF, decoding and recovery layer →Second deconvolutional layer→fusion operation→channel attention module CA→pixel attention module PA→features obtained ; Features E31, E21, E11 → Cross-level feature adaptive fusion module CHAF, features → Fusion operation → Third deconvolution layer → Fog-free remote sensing image.

[0008] In the preferred embodiment, the feature extraction process using the embedded U-shaped network framework in step S2 is expressed by the following formula:

[0009]

[0010]

[0011] in, Represents a parallel embedded U-shaped network framework. Indicates the process The different scales of hierarchical features output after the module.

[0012] In the preferred embodiment, the parallel embedded U-shaped network framework structures in step S2 include: The specific operation of EU1 is as follows: For the first layer of the coding layer feature extraction module... → 1*1 Convolution without changing size → 3*3 Convolution → 3*3 Convolution → obtain features E15, E14, E13; E14, E13 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E12; E12, E15 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E11; The specific operation of EU2 is as follows: The second layer of the coding layer feature extraction module... → 1*1 Convolution without changing size → 3*3 Convolution → 3*3 Convolution → obtain features E25, E24, E23; E24, E23 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E22; E22, E25 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E21; The specific operation of EU3 is as follows: The third layer of the coding layer feature extraction module... → 1*1 Convolution without changing size → 3*3 Convolution → 3*3 Convolution → obtain features E35, E34, E33; E34, E33 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E32; E32, E35 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E31.

[0013] In the preferred embodiment, step S3 includes: constructing a cross-level feature adaptive fusion module CHAF, which has a size of and The hierarchical features M and N are used to extract effective features through pixel attention and channel attention, respectively, specifically including the following steps: First, adjust the feature map's width, height, and number of channels using 3x3 convolution and 3x3 deconvolution, respectively. The formula is:

[0014]

[0015] in and All The feature map, TConv represents a standard transposed convolution with a kernel size of 3*3, and Conv2D represents a convolution with a kernel size of 3 that does not change the size of the feature map; Will Compression is performed along the channel dimension, with each channel representing a type of semantic information. The formula after global average pooling is:

[0016] in Let represent the value of the nth single-channel feature map at position (i,j). Represents the global average pooling function, and the shape of the feature map F is from squeezed into ; channel vector After two 1x1 convolutions and activation functions, the entire process can be represented as follows:

[0017] in These represent the Sigmoid and ReLU activation functions, respectively. and This represents two 1x1 convolutions; Secondly, We directly use two convolutions, a Sigmoid function, and a ReLU activation function, as shown in the formula: ; Then, Features of adjacent levels Multiply, we get The extracted semantic information is embedded into shallow features. Features of adjacent levels Multiply to get This process can be expressed by the following formula: ; Finally, an adaptive weight allocator is used to fuse cross-level features. The formula for the adaptive weight allocator is as follows:

[0018] Where Awa represents the adaptive weight assigner. This represents two different levels of features. This represents the learnable factor in the adaptive weight assigner.

[0019] In the preferred embodiment, the cross-level feature fusion module CHAF structure in step S3 includes: Input hierarchical feature M → 3*3 Convolution → obtain feature M' → 1*1 Convolution → activation function ReLU → 1*1 Convolution → activation function sigmoid → feature A with 1 channel; Input hierarchical features N → 3*3 deconvolution → obtain features N' → global average pooling → 1*1 convolution → ReLU activation function → 1*1 convolution → sigmoid activation function → Feature B; Feature A, Feature N' → pixel-wise multiplication → Feature M"; Feature B, Feature M' → pixel-wise multiplication → Feature N"; Adaptive weight allocator assigns weights M" → multiply pixel by pixel; Adaptive weight allocator assigns weights ,N"→pixel-wise multiplication; The two features after weighted multiplication are then fused and added together, resulting in feature J after cross-level fusion.

[0020] In the preferred embodiment, in step S4, the embedded U-shaped remote sensing dehazing network, the embedded U-shaped network framework, and the cross-level feature adaptive fusion module are used in accordance with the following steps: S41. Input the foggy image into the U-shaped network image defogging network; S42. Input the coding layers of different levels into the embedded U-shaped network structure of the corresponding layer to obtain information features; S43. Input the features of different scales extracted by the embedded U-shaped network into the cross-level feature adaptive fusion module, and further fuse the fused features with the corresponding coding layer features to obtain the final output clear and fog-free image; S44. Apply loss constraints to the network training process.

[0021] In the preferred embodiment, the loss constraints in step S44 include: LI loss constraint, adversarial loss constraint, perceptual loss constraint, and multi-scale structural similarity loss constraint. The specific formula for L1 loss is:

[0022]

[0023] Where G represents the generation of NHU-Net, and Let i represent the values ​​of the blurred and sharp images at pixel i, respectively, and N represent the number of pixels in the image; The adversarial loss constraint uses the adversarial loss of GAN, and the specific definition of adversarial loss is as follows:

[0024] Where G represents the generator, which converts the image with and without fog into a clear image, and D is the discriminator, which distinguishes between the generated clear image and the real fog-free image; The perceptual loss constraint preserves more texture information in the recovered image, and calculates the L2 norm of the sharp and hazy images at different layers of the VGG16 pre-trained model. The perceptual loss is defined as:

[0025] in, represents the size of the feature map obtained from layer i of the VGG16 pre-trained model, N represents the number of feature layers of the VGG16 pre-trained model used in the perceptual loss, where x and y represent the foggy image and the clear image, respectively. The multi-scale structural similarity loss constraint is defined by the following formula:

[0026] in represents the mean values ​​of the dehazed image and the clear image, respectively. These represent the standard deviations of the dehazed image and the clear image, respectively. Used to represent the covariance between the dehazed image and the clear image. It is the importance coefficient of the equation. and It is a constant term.

[0027] In the preferred scheme, the overall network loss function is expressed as:

[0028] in , , and It is a hyperparameter that balances different losses.

[0029] The beneficial effects of this invention are as follows: 1) This invention proposes an embedded U-shaped remote sensing dehazing network with cross-level feature adaptive fusion. The network extracts information from the encoding layer through a parallel embedded U-shaped network structure and passes it to the decoding layer to restore the clear image. The cross-level feature adaptive fusion module adaptively fuses features, achieving high-quality image dehazing results. Furthermore, the proposed algorithm has been shown to achieve the best performance on two public remote sensing image dehazing datasets, StateHaze1k and RICE1.

[0030] 2) The parallel embedded U-shaped network framework proposed in this invention introduces multiple embedded parallel U-shaped structures between the decoding layer and the encoding layer, which enhances the feature extraction of the encoding layer and makes it supplement the information of each decoding layer, thereby reducing feature dilution in the process of the decoding layer reconstructing a clear image and increasing the receptive field of the network.

[0031] 3) The cross-level feature adaptive fusion module proposed in this invention explores how to effectively fuse features from adjacent levels. By using learnable factors to adaptively fuse features from adjacent levels extracted by channel attention and pixel attention, the expressive power of the network can be enhanced, thereby enabling better recovery of scene, detail and color information. Attached Figure Description

[0032] Figure 1 This is a diagram of the overall network structure and multiple parallel embedded U-shaped network frameworks in an embodiment of the present invention. Figure 2 The diagram shows the structure of the CHAF (Chain-Agent Fusion) module, which is a cross-level feature adaptive fusion module. Detailed Implementation

[0033] Example 1 like Figure 1 and Figure 2 As shown, a method for constructing an embedded U-shaped remote sensing dehazing network with cross-level feature adaptive fusion includes the following steps: S1. Construct a U-shaped image dehazing network, which includes: a coding layer feature extraction module, a feature repair module, a decoding layer image restoration module, a parallel embedded U-shaped framework, and a cross-level feature adaptive fusion module CHAF.

[0034] S2. Construct a parallel embedded U-shaped framework. Utilize the features extracted by the feature extraction module of each coding layer in S1 to perform feature extraction on each layer through the embedded U-shaped network.

[0035] S3. Construct a cross-level feature adaptive fusion module to effectively and adaptively fuse multiple adjacent level features extracted by channel attention and pixel attention through learnable factors.

[0036] S4. The foggy image is fed into the U-shaped image dehazing network. Through parallel embedded U-shaped network and cross-level feature adaptive fusion module, a clear fog-free image is output. Finally, the clear image output is used to calculate the loss and constrain the training of the network.

[0037] Specifically, S1 includes: like Figure 1 As shown, the feature extraction module of the coding layer uses four convolution operations to perform 8x downsampling to obtain coding layers of different scales. , , , Next, in the extracted low-resolution features The feature extraction is further enhanced by combining six consecutive residual blocks. Finally, the features extracted by the consecutive residual blocks, which contain a large amount of semantic scene information, are decoded and reconstructed using deconvolution to restore the original image resolution.

[0038] Step S2 specifically includes: constructing the embedded U-shaped network framework EU-Net Construct a parallel embedded U-shaped network framework and utilize the features extracted by the feature extraction module of the coding layer in S1. , , , Further feature extraction is then performed. For example... Figure 1 The section with the background color is shown. Taking the first layer of the encoding layer as an example, the features of the encoding layer are input into the embedded U-shaped network to obtain three cross-scale features. The feature extraction process of the embedded U-shaped network framework proposed in this paper is expressed by the following formula:

[0039]

[0040]

[0041] in, Represents a parallel embedded U-shaped network framework. Indicates the process The module outputs hierarchical features at different scales. This invention not only further extracts the information from the encoding layer, but also adds features extracted from the encoding layer at different scales as supplementary information to each decoding layer, thus alleviating feature dilution during decoding and increasing the network's receptive field.

[0042] Step S3 specifically includes: constructing the cross-level feature adaptive fusion module CHAF.

[0043] like Figure 2 As shown, the size is and The feature maps have hierarchical features M and N. Shallow features contain more edge and texture information, while deeper features contain more semantic information. Effective features are extracted from the two cross-level features using pixel attention columns and channel attention, respectively. First, M and N are adjusted to their respective dimensions and number of channels using 3x3 convolutions and 3x3 deconvolutions. As shown in the formula below:

[0044]

[0045] in and All The feature map is defined as follows: TConv represents a standard transposed convolution with a kernel size of 3*3, and Conv2D represents a convolution with a kernel size of 3 that does not change the size of the feature map.

[0046] Due to deep features Including more semantic information, we will Compression is performed according to the channel dimension, with each channel representing a type of semantic information. The formula for global average pooling is as follows:

[0047] in Let represent the value of the nth single-channel feature map at position (i,j). Represents the global average pooling function, and the shape of the feature map F is from squeezed into To better obtain information for each channel, we will use channel vectors. After two 1x1 convolutions and a simple activation function, the entire process can be represented as:

[0048] in These represent the Sigmoid and ReLU activation functions, respectively. and This represents two 1x1 convolutions.

[0049] Secondly, in order to extract the spatial details and texture information of shallow features, we will We directly use two convolutions, a Sigmoid function, and a ReLU activation function. The formula is shown below:

[0050] Then, in order to embed detailed texture information into features of adjacent layers... This article will Features of adjacent levels Multiply, we get The extracted semantic information is embedded into shallow features. Features of adjacent levels Multiply to get The process is illustrated by the following formula:

[0051] Finally, in order to adaptively fuse two cross-level features Having obtained feature J, we consider an adaptive weight allocator to fuse features across layers. The adaptive weight allocator can be formulated as:

[0052] Where Awa represents the adaptive weight assigner. This represents two different levels of features. This represents the learnable factor in the adaptive weight assigner.

[0053] Step S4 specifically includes: A hazy image is input into a U-shaped image dehazing network. Through multiple parallel embedded U-shaped network modules and a cross-layer feature adaptive fusion module, a clear, haze-free image is finally output. The loss is calculated using the output clear image. The four constraints of the network training process are as follows: The first type is L1 loss, and the specific formula is as follows:

[0054]

[0055] Where G represents the generation of NHU-Net, and These represent the values ​​of the blurred and sharp images at pixel i, respectively. N represents the number of pixels in the image.

[0056] The second type is adversarial loss. We use the adversarial loss of GANs, where G represents the generator, which mainly transforms the hazy image into a clear image, and D is the discriminator, which mainly distinguishes between the generated clear image and the real haze-free image. The specific definition of adversarial loss is as follows:

[0057] The third type is perceptual loss. Perceptual loss can effectively preserve more texture information in the recovered image and can also effectively calculate the L2 norm of clear and haze-free images at different layers of the VGG16 pre-trained model. The perceptual loss is defined as follows:

[0058] in, This represents the size of the feature map obtained from layer i of the VGG16 pre-trained model. N represents the number of feature layers of the VGG16 pre-trained model used in the perceptual loss. Here, x and y represent the foggy image and the clear image, respectively.

[0059] The fourth method uses multi-scale structural similarity loss to constrain the network, and the specific formula is as follows:

[0060] in represents the mean values ​​of the dehazed image and the clear image, respectively. These represent the standard deviations of the dehazed image and the clear image, respectively. Used to represent the covariance between the dehazed image and the clear image. These are two important terms in the equation. and It is a constant term.

[0061] The overall network loss function is expressed as:

[0062] in , , and It is a hyperparameter that balances different losses. , , , .

[0063] Example 2 Parameter settings This invention evaluates various dehazing algorithms on two public datasets, SateHaze1k and RICE. The SateHaze1k dataset contains three subsets: SateHaze 1k Thin, SateHaze 1k Mederate, and SateHaze 1k Thick, each containing 400 pairs of synthesized remote sensing images (320 pairs for training, 35 pairs for validation, and 45 pairs for testing). The RICE dataset, collected from Google Earth, contains 500 pairs of 512x512 images, each pair consisting of one clear image and one hazy image. To validate the effectiveness of each method, the RICE1 dataset is divided into training and testing sets in a 9:1 ratio. Furthermore, the proposed network is optimized using the Adam optimizer with momentum decay exponents β1=0.9 and β2=0.999, and the learning rate and batch size are set to 0.0001 and 4, respectively. The initial learning rate is set to 0.001, and MultiStepLR is used to dynamically adjust the learning rates. During training, we set the parameters of the loss function in the network model to... , , , All experiments in this invention were conducted on an NVIDIA RTX 3090 GPU, and the code was implemented using PyTorch. To better evaluate the proposed method, Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) were used as evaluation metrics. Our dehazing network was compared with seven other recent excellent algorithms. The algorithms compared are DCP, FFA, AECR, SGID, FCTF, DCRD, and DCI-Net.

[0064] 2. Experimental Results Table 1. Comparison results of the method in this invention with each other on the StateHaze1k and RICE1 datasets.

[0065] This invention was quantitatively evaluated against seven other algorithms on the SateHaze1k and RICE1 datasets. Table I shows the average PSNR and SSIM values ​​of the tested methods. As shown in the fifth column of Table 1, DCP achieved a PSNR of only 11.37 dB on the SateHaze1k dataset, while the other algorithms all achieved PSNRs higher than 20 dB, demonstrating that the end-to-end dehazing algorithm outperforms traditional parameter estimation algorithms. We noted that our faming on the SateHaze1k dataset significantly outperformed other methods in light fog, medium fog, and thick fog. Specifically, this invention showed a significant improvement over the second-place method on the SateHaze 1k Thin, SateHaze 1k Mederate, and SateHaze1k Thick datasets, improving PSNR by 3.03 dB, 1.24 dB, and 2.71 dB, respectively, and improving SSIM by 0.028, 0.11, and 0.04, respectively. Furthermore, the proposed method achieves better results on the RICE1 dataset compared to other algorithms, especially with PSNR exceeding 4 dB and SSIM exceeding 0.01 compared to algorithms in the remote sensing field.

[0066] In summary, the dehazing network of this invention exhibits superior dehazing performance in light fog, medium fog, dense fog, and realistic fog. Through the network structure of this invention, the impact of haze in images can be effectively reduced, improving image clarity and detail, and restoring a more realistic scene.

[0067] 3. Ablation test To evaluate the effectiveness of each module, this invention mainly conducted ablation experiments on the innovative design of the proposed embedded U-shaped network framework and cross-level feature adaptive fusion module. These experiments included five experiments: (1) Base: The basic U-shaped framework mainly consists of four downsampling layers, six residual blocks, channel attention, pixel attention, and three upsampling layers, with skip connections used between the encoding and decoding layers. (2) Base+EU1: Based on the basic framework, the first layer uses an embedded U-shaped network for feature extraction. (3) Base+EU2: The first and second layers of the basic framework + downsampling use an embedded U-shaped network for feature extraction and skip connections. (4) Base+EU3: The basic framework + the first three layers use an embedded U-shaped network for feature extraction and skip connections. (5) Base+3EU+CHAF: The basic framework + the first three downsampling layers use an embedded U-shaped network for feature extraction + cross-level feature adaptive fusion module and feature fusion of the decoding layer.

[0068] Table 2 shows the PSNR and SSIM results on the SateHaze 1k Thin dataset.

[0069] To ensure experimental fairness, we trained all five experiments on the SateHaze 1k Thin dataset in the same manner, with PSNR and SSIM results shown in Table II. First, the base framework achieved PSNR and SSIM scores of 23.56 and 0.897, respectively. Applying the proposed EU-Net to the first downsampling layer on the base framework improved PSNR and SSIM by 1.24 dB and 0.016, respectively. Experimental results demonstrate the effectiveness of our proposed EU-Net. Next, we extracted features from the first three downsampling layers using EU-Net, achieving significant improvements in both PSNR and SSIM. Finally, we used the CHAF module for cross-layer feature adaptive fusion and then fused it with the upsampling, resulting in a 0.28 dB improvement in PSNR and a 0.001 improvement in SSIM compared to skip connections. The experimental results demonstrate that this invention has a certain effect on dehazing remote sensing images and verify the effectiveness of the CHAF module.

[0070] This invention proposes an embedded U-shaped remote sensing dehazing network algorithm with cross-level feature adaptive fusion. The network mainly consists of multiple parallel embedded U-shaped network frameworks and a cross-level feature adaptive fusion module. The embedded U-shaped network frameworks fully utilize the features extracted from the encoding layer and add them as supplementary information to each decoding layer, which can alleviate feature dilution during the encoding process and increase the receptive field of the network. The cross-level feature adaptive fusion module can effectively utilize learnable factors to adaptively fuse cross-level features, thereby further improving scene restoration and contextual information constraints. This invention verifies the effectiveness of the proposed framework and module, and demonstrates that it can restore higher-quality, clearer remote sensing images while improving network performance.

[0071] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing an embedded U-shaped remote sensing dehazing network, characterized by: Includes the following steps: S1. Construct a U-shaped image dehazing network; The U-shaped image dehazing network includes a coding layer feature extraction module, a feature inpainting module, a decoding layer image restoration module, a parallel embedded U-shaped framework, and a cross-level feature adaptive fusion module CHAF. The U-shaped image dehazing network structure is as follows: First layer of the coding layer feature extraction module →Feature extraction of the second layer of the coding layer →Feature extraction of the coding layer, third layer →Feature extraction of the fourth layer of the coding layer ; First layer of the coding layer feature extraction module →The embedded U-shaped network framework EU1→outputs three features E11, E12, and E13; Second layer of the coding layer feature extraction module →The embedded U-shaped network framework EU2→outputs three features E21, E22, and E23; The third layer of the coding layer feature extraction module →The embedded U-shaped network framework EU3→outputs three features E31, E32, and E33; Features E33, E23, E13 → Cross-level feature adaptive fusion module CHAF, decoding and recovery layer →First deconvolutional layer→fusion operation→channel attention module CA→pixel attention module PA→decoding and recovery layer ; Features E32, E22, E12 → Cross-level feature adaptive fusion module CHAF, decoding and recovery layer →Second deconvolutional layer→fusion operation→channel attention module CA→pixel attention module PA→features obtained ; Features E31, E21, E11 → Cross-level feature adaptive fusion module CHAF, features → Fusion operation → Third deconvolution layer → Fog-free remote sensing image; S2. Construct several parallel embedded U-shaped network frameworks and add them between the encoding and decoding layers of the U-shaped network to extract multi-scale feature information; Several parallel embedded U-shaped network framework structures include: The specific operation of EU1 is as follows: For the first layer of the coding layer feature extraction module... → 1*1 Convolution without changing size → 3*3 Convolution → 3*3 Convolution → obtain features E15, E14, E13; E14, E13 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E12; E12, E15 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E11; The specific operation of EU2 is as follows: The second layer of the coding layer feature extraction module... → 1*1 Convolution without changing size → 3*3 Convolution → 3*3 Convolution → obtain features E25, E24, E23; E24, E23 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E22; E22, E25 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E21; The specific operation of EU3 is as follows: The third layer of the coding layer feature extraction module... → 1*1 Convolution without changing size → 3*3 Convolution → 3*3 Convolution → obtain features E35, E34, E33; E34, E33 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E32; E32, E35 → cross-level feature adaptive fusion operation → 1*1 Convolution → obtain feature E31; S3. Construct an adaptive fusion mechanism that can adaptively fuse multi-scale features extracted from embedded U-shaped network modules at different levels. S4. The foggy remote sensing image is fed into the U-shaped image dehazing network. Through several parallel embedded U-shaped network frameworks and cross-level feature adaptive fusion modules, a clear fog-free image is finally output. Finally, the loss is calculated using the output clear image to constrain the training of the network. The above steps are used to construct an embedded U-shaped remote sensing dehazing network based on cross-level feature adaptive fusion of U-shaped meshes.

2. The method for constructing an embedded U-shaped remote sensing defogging network according to claim 1, characterized in that: The feature extraction process using the embedded U-shaped network framework in step S2 is expressed by the following formula: ； ； ； in, Represents a parallel embedded U-shaped network framework. Indicates the process The different scales of hierarchical features output after the module.

3. The method for constructing an embedded U-shaped remote sensing defogging network according to claim 1, characterized in that: Step S3 includes: constructing a cross-level feature adaptive fusion module CHAF, which has a size of and The hierarchical features M and N are used to extract effective features through pixel attention and channel attention, respectively, specifically including the following steps: First, adjust the feature map's width, height, and number of channels using 3x3 convolution and 3x3 deconvolution, respectively. The formula is: ； ； in and All The feature map, TConv represents a standard transposed convolution with a kernel size of 3*3, and Conv2D represents a convolution with a kernel size of 3 that does not change the size of the feature map; Will Compression is performed along the channel dimension, with each channel representing a type of semantic information. The formula after global average pooling is: ； in Let represent the value of the nth single-channel feature map at position (i,j). Represents the global average pooling function, and the shape of the feature map F is from squeezed to ; channel vector After two 1x1 convolutions and activation functions, the entire process can be represented as follows: ； in These represent the Sigmoid and ReLU activation functions, respectively. and This represents two 1x1 convolutions; Secondly, We directly use two convolutions, a Sigmoid function, and a ReLU activation function, as shown in the formula: ； Then, Features of adjacent levels Multiply, we get The extracted semantic information is embedded into shallow features. Features of adjacent levels Multiply to get This process can be expressed by the following formula: ； Finally, an adaptive weight allocator is used to fuse cross-level features. The formula for the adaptive weight allocator is as follows: ； Where Awa represents the adaptive weight assigner. This represents two different levels of features. This represents the learnable factor in the adaptive weight assigner.

4. The method for constructing an embedded U-shaped remote sensing defogging network according to claim 1, characterized in that: In step S3, the cross-level features are processed in the adaptive fusion module CHAF structure, which includes: Input hierarchical feature M → 3*3 Convolution → obtain feature M' → 1*1 Convolution → activation function ReLU → 1*1 Convolution → activation function sigmoid → feature A with 1 channel; Input hierarchical features N → 3*3 deconvolution → obtain features N' → global average pooling → 1*1 convolution → ReLU activation function → 1*1 convolution → sigmoid activation function → Feature B; Feature A, Feature N' → pixel-wise multiplication → Feature M"; Feature B, Feature M' → pixel-wise multiplication → Feature N"; Adaptive weight allocator assigns weights M" → pixel-wise multiplication; Adaptive weight allocator assigns weights ,N"→pixel-wise multiplication; The two features after weighted multiplication are then fused and added together, resulting in feature J after cross-level fusion.

5. The method for constructing an embedded U-shaped remote sensing defogging network according to claim 1, characterized in that: in In step S4, the embedded U-shaped remote sensing dehazing network, the embedded U-shaped network framework, and the cross-level feature adaptive fusion module are used in accordance with the following steps: S41. Input the foggy image into the U-shaped network image defogging network; S42. Input the coding layers of different levels into the embedded U-shaped network structure of the corresponding layer to obtain information features; S43. Input the features of different scales extracted by the embedded U-shaped network into the cross-level feature adaptive fusion module, and further fuse the fused features with the corresponding coding layer features to obtain the final output clear and fog-free image; S44. Apply loss constraints to the network training process.

6. The method for constructing an embedded U-shaped remote sensing defogging network according to claim 1, characterized in that: The loss constraints in step S44 include: LI loss constraint, adversarial loss constraint, perceptual loss constraint, and multi-scale structural similarity loss constraint. The specific formula for L1 loss is: ； ； Where G represents the generation of NHU-Net, and Let i represent the values ​​of the blurred and sharp images at pixel i, respectively, and N represent the number of pixels in the image; The adversarial loss constraint uses the adversarial loss of GAN, and the specific definition of adversarial loss is as follows: ； Where G represents the generator, which converts the image with and without fog into a clear image, and D is the discriminator, which distinguishes between the generated clear image and the real fog-free image; The perceptual loss constraint preserves more texture information in the recovered image, and calculates the L2 norm of the sharp and hazy images at different layers of the VGG16 pre-trained model. The perceptual loss is defined as: ； in, represents the size of the feature map obtained from layer i of the VGG16 pre-trained model, N represents the number of feature layers of the VGG16 pre-trained model used in the perceptual loss, where x and y represent the foggy image and the clear image, respectively. The multi-scale structural similarity loss constraint is defined by the following formula: ； in represents the mean values ​​of the dehazed image and the clear image, respectively. These represent the standard deviations of the dehazed image and the clear image, respectively. Used to represent the covariance between the dehazed image and the clear image. It is the importance coefficient of the equation. and It is a constant term.

7. The method for constructing an embedded U-shaped remote sensing defogging network according to claim 1, characterized in that: The overall network loss function is expressed as: ； in , , and It is a hyperparameter that balances different losses.

Images