Channel-enhanced progressive water body extraction network based on near-infrared information

Abstract

According to the invention, optimization is carried out on the basis of U-Net, a channel enhancement progressive water body extraction network based on near-infrared information is provided, and a remote sensing water body extraction network model with higher precision is realized. According to the network, a channel-enhanced multi-scale module is designed to optimize multi-scale information. Meanwhile, a wave band enhanced water area focusing attention module is designed to enhance the sensitivity of the model to the water body. And performing up-sampling in a decoder stage in a progressive prediction mode. The model can be used for solving the problems of insufficient utilization of multispectral information, insufficient relevance between channels of different scales, up-sampling loss and the like in the current remote sensing water body extraction.

Description

Technical Field

[0001] The invention belongs to the field of computer image processing and is a channel-enhanced progressive water body extraction network based on near-infrared information. Background Art

[0002] Water body extraction is a highly valuable research area in remote sensing image segmentation. Remote sensing imagery, with its multispectral characteristics and combined with deep learning methods, offers the potential for precise water body identification. However, unlike ordinary images, remote sensing images possess richer multispectral information. Near-infrared band information offers significant advantages and is highly valuable in water body identification and extraction, effectively distinguishing water bodies from other land features. However, because existing networks are mostly influenced by mainstream computer vision, multispectral information is often neglected in network design. The ASPP module adds multiscale information to the network. However, the ASPP module typically directly concatenates features at different dilation rates across the channel dimension and then simply fuses them through a convolutional layer. This direct concatenation approach fails to fully exploit the synergy between multiscale information. Furthermore, mainstream network architectures typically employ an encoder-decoder structure, with the decoder gradually restoring spatial resolution. However, multiple upsampling steps are required in the decoder, which can blur information such as spatial details. These factors can affect the accuracy of water body extraction. In this context, this paper proposes a channel-enhanced progressive water extraction network based on near-infrared information to improve the above problems and enhance the water extraction effect. Summary of the Invention

[0003] In order to overcome the shortcomings of the above-mentioned prior art, the present invention discloses a channel-enhanced progressive water body extraction network based on near-infrared information. The present invention optimizes U-Net to achieve high-precision water body extraction from remote sensing images. A band-enhanced water area focus attention module based on near-infrared information is designed, and near-infrared information is introduced to enhance the model's sensitivity to water areas and better extract water bodies. Finally, the encoder introduces a channel-enhanced multi-scale module to optimize multi-scale information and achieve more refined segmentation of water areas in remote sensing images. In the decoder stage, a progressive prediction method is used to transfer features.

[0004] The technical steps adopted in the present invention are as follows:

[0005] Step 1: Take an image in NIR, R, and G three-channel format as input.

[0006] Step 2: Use ResNet-34 to extract image features.

[0007] Step 3: Optimize the obtained multi-scale information through a channel-enhanced multi-scale module.

[0008] Step 3.1: Through ASPP, obtain the feature maps of five different scales of the dilated convolution layer.

[0009] Step 3.2: Concatenate the original input feature map with the five feature maps obtained in step 3.1 along the channel.

[0010] Step 3.3: Pass the spliced image through SE to obtain the weighted feature map.

[0011] Step 3.4: Use 1×1 convolution to perform channel compression on the feature map.

[0012] Step 4: The output of the first stage of the backbone network and the original remote sensing image are combined through the band-enhanced water focus attention module to optimize the feature map.

[0013] Step 4.1: The first-stage feature map undergoes global average pooling and global max pooling, generating two channel feature vectors. These two channel feature vectors are nonlinearly mapped using a parameter-sharing fully connected layer. They are fused element-wise and then normalized using a sigmoid function to generate an attention weight matrix that represents the importance of different channels. This weight matrix is then element-wise multiplied with the original input features.

[0014] Step 4.2: The feature map obtained in step 4.1 is subjected to global average pooling and global maximum pooling in the spatial dimension, and two single-channel feature maps are generated and spliced along the channel dimension.

[0015] Step 4.3: The original remote sensing image input to the model needs to be downsampled by 4 times.

[0016] Step 4.4: Calculate the normalized water index for the characteristic graph obtained in step 4.3 and adjust the value range to obtain the characteristic graph.

[0017] Step 4.5: Concatenate the feature maps obtained in steps 4.3 and 4.4 along the channel dimension. Perform a 7×7 convolution operation to optimize the information, and then activate it with a sigmoid function to generate an attention weight map. Multiply the weight map with the original feature map pixel by pixel.

[0018] Step 4.6: Separate the large feature map from step 4.3 into the near-infrared band. Perform global average pooling and global max pooling in the spatial dimension, and concatenate the two generated single-channel feature maps along the channel dimension. Perform a 7×7 convolution operation, and then activate it with a sigmoid function to generate an attention weight map. Multiply the weight map with the original feature map pixel by pixel.

[0019] Step 4.7: Concatenate the output of the first stage of the backbone network, the feature maps obtained in steps 4.5 and 4.6 along the channels. Perform a 1×1 convolution to adjust the number of channels.

[0020] Step 5: Perform feature fusion and aggregation in a progressive manner.

[0021] Step 5.1: The output of the third stage of the backbone network is passed through the SE attention mechanism, concatenated with the feature map obtained in step 3 along the channel, and then a 3×3 double convolution is performed.

[0022] Step 5.2: Adjust the number of channels of the feature map obtained in step 5.1 through 1×1 convolution, and then segment it to obtain the segmentation result.

[0023] Step 5.3: Concatenate the feature maps obtained in step 5.1 and step 5.2 along the channel, fuse them through 1×1 convolution, and upsample them to 2 times.

[0024] Step 5.4: Concatenate the output of the second stage of the backbone network and the feature map obtained in step 5.3 along the channel, then perform a 3×3 double convolution, and repeat steps 5.2 and 5.3.

[0025] Step 5.5: Concatenate the feature map obtained in step 4 and the feature map obtained in step 5.4 along the channel, then perform 3×3 double convolution, and repeat steps 5.2 and 5.3.

[0026] Step 6: Perform final segmentation on the feature map obtained in step 5 to obtain the prediction result.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] (1) It is superior to similar remote sensing water body extraction models in terms of accuracy and visual segmentation effect.

[0029] (2) The method proposed in the present invention effectively solves the problems of insufficient utilization of multispectral information, insufficient correlation between channels of different scales, and upsampling loss in existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is the network structure diagram of the present invention

[0031] Figure 2 Multi-scale module diagram for channel enhancement of the present invention

[0032] Figure 3 This is the module diagram for the band-enhanced water area focusing attention of the present invention

[0033] Figure 4 The progressive prediction module diagram of the present invention is

[0034] Specific implementation steps

[0035] The present invention will be further described below with reference to the accompanying drawings.

[0036] This paper designs a channel-enhanced progressive water extraction network based on near-infrared information. Based on the U-Net network, a channel-enhanced multi-scale module optimizes multi-scale information. Furthermore, a band-enhanced water-focusing attention module enhances the model's sensitivity to water bodies. Upsampling is performed using a progressive prediction approach at the decoder stage.

[0037] like Figure 1 As shown in the figure, the input remote sensing image enters the ResNet-34 encoder for feature extraction. The feature map in the final stage is optimized at multiple scales using the channel-enhanced multi-scale module. The feature maps in the remaining stages are optimized by the SE attention and band-enhanced water-focused attention modules. The optimized feature maps are upsampled in the decoder stage using progressive prediction.

[0038] like Figure 2 As shown in the figure, in the channel-enhanced multi-scale module, in addition to the feature map obtained by ASPP, the input feature map also uses the residual connection to the original feature map and is optimized together through the SE attention mechanism.

[0039] like Figure 3 As shown in the figure, the band-enhanced water-focused attention module adopts a three-branch collaborative architecture. Its input receives both the first-stage feature output and the original remote sensing image. The first-stage input feature map is concatenated along the channel with the normalized water index feature map after channel attention and spatial attention. It then undergoes a 7×7 convolution, weights are calculated using Sigmoid, and the weights are element-wise multiplied with the first-stage feature output. The near-infrared channel of the downsampled original remote sensing image is separated, spatially attended to, and then concatenated with the 7×7 convolution, weights are calculated using Sigmoid, and element-wise multiplied with the near-infrared channel. Finally, the two weighted feature maps are concatenated along the channel with the first-stage feature output and a 1×1 convolution is performed to adjust the number of channels.

Claims

1. A channel-enhanced progressive water extraction network based on near-infrared information. Characterized by: The following steps are involved: The technical steps adopted in the present invention are as follows: Step 1: Take an image in NIR, R, and G three-channel format as input. Step 2: Use ResNet-34 to extract image features. Step 3: Optimize the obtained multi-scale information through a channel-enhanced multi-scale module. Step 3.1: Through ASPP, obtain the feature maps of five different scales of the dilated convolution layer. Step 3.2: Concatenate the original input feature map with the five feature maps obtained in step 3.1 along the channel. Step 3.3: Pass the spliced image through SE to obtain the weighted feature map. Step 3.4: Use 1×1 convolution to perform channel compression on the feature map. Step 4: The output of the first stage of the backbone network and the original remote sensing image are combined through the band-enhanced water focus attention module to optimize the feature map. Step 4.1: The first-stage feature map undergoes global average pooling and global max pooling, generating two channel feature vectors. These two channel feature vectors are nonlinearly mapped using a parameter-sharing fully connected layer. They are fused element-wise and then normalized using a sigmoid function to generate an attention weight matrix that represents the importance of different channels. This weight matrix is then element-wise multiplied with the original input features. Step 4.2: The feature map obtained in step 4.1 is subjected to global average pooling and global maximum pooling in the spatial dimension, and two single-channel feature maps are generated and spliced along the channel dimension. Step 4.3: The original remote sensing image input to the model needs to be downsampled by 4 times. Step 4.4: Calculate the normalized water index for the characteristic graph obtained in step 4.3 and adjust the value range to obtain the characteristic graph. Step 4.5: Concatenate the feature maps obtained in steps 4.3 and 4.4 along the channel dimension. Perform a 7×7 convolution operation to optimize the information, and then activate it with a sigmoid function to generate an attention weight map. Multiply the weight map with the original feature map pixel by pixel. Step 4.6: Separate the large feature map from step 4.3 into the near-infrared band. Perform global average pooling and global max pooling in the spatial dimension, and concatenate the two generated single-channel feature maps along the channel dimension. Perform a 7×7 convolution operation, and then activate it with a sigmoid function to generate an attention weight map. Multiply the weight map with the original feature map pixel by pixel. Step 4.7: Concatenate the output of the first stage of the backbone network, the feature maps obtained in steps 4.5 and 4.6 along the channels. Perform a 1×1 convolution to adjust the number of channels. Step 5: Perform feature fusion and aggregation in a progressive manner. Step 5.1: The output of the third stage of the backbone network is passed through the SE attention mechanism, concatenated with the feature map obtained in step 3 along the channel, and then a 3×3 double convolution is performed. Step 5.2: Adjust the number of channels of the feature map obtained in step 5.1 through 1×1 convolution, and then segment it to obtain the segmentation result. Step 5.3: Concatenate the feature maps obtained in step 5.1 and step 5.2 along the channel, fuse them through 1×1 convolution, and upsample them to 2 times. Step 5.4: Concatenate the output of the second stage of the backbone network and the feature map obtained in step 5.3 along the channel, then perform a 3×3 double convolution, and repeat steps 5.2 and 5.

3. Step 5.5: Concatenate the feature map obtained in step 4 and the feature map obtained in step 5.4 along the channel, then perform 3×3 double convolution, and repeat steps 5.2 and 5.

3. Step 6: Perform final segmentation on the feature map obtained in step 5 to obtain the prediction result.

2. The method according to claim 1, wherein Step 3 optimizes the obtained multi-scale information through the channel-enhanced multi-scale module designed by the present invention.

3. The method according to claim 1, wherein: Step 2 uses the image features of different scales obtained in step 1 to construct a lightweight cost body. The mean measurement method based on group similarity uses similar groups to divide elements into different groups, and then calculates the mean of each group as the representative value of the group. The cost body established based on this method can be used to evaluate problems of different scales, and has a smaller volume and higher efficiency.

4. The method according to claim 1, wherein: Step 4: The feature map of the first stage of the encoder is optimized by the water area focusing attention module designed by the present invention.

5. The method according to claim 1, wherein: In step 5, all encoder output feature maps except the first stage are subjected to SE attention and then upsampled by the progressive prediction module designed in this paper.

Images