Spatial enhancement water body extraction network based on multi-scale and long-distance dependence
By introducing a space-enhanced water body extraction network with long-distance dependence and multi-scale information, the problem of insufficient information utilization and semantic gap in remote sensing water body extraction is solved, and a higher precision water body extraction effect is achieved.
Patent Information
- Application Number
- CN202510600274.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2025-08-15
AI Technical Summary
The existing remote sensing water extraction method lacks a long-distance-dependent spatial relationship in the feature extraction process, resulting in insufficient utilization of multi-scale information, and the semantic differences between low-level and high-level features cannot be effectively reconciled, affecting feature coordination and information utilization.
The spatially enhanced water body extraction network based on multi-scale and long-distance dependence is designed. Through the local-global spatial feature enhancement module and dual spatial multi-scale aggregation module with axial non-local attention, long-distance dependence information and multi-scale information are introduced to optimize the feature fusion process.
The accuracy and visual segmentation effect of remote sensing water body extraction are improved, the problems of difficulty in obtaining global information and insufficient utilization of multi-scale information are solved, and the effective fusion of different levels of features is achieved.
Smart Images
Figure CN120495890A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer image processing and is a spatially enhanced water body extraction network based on multi-scale and long-distance dependence. Background Art
[0002] Remote sensing water extraction is extremely important in fields such as environmental protection, resource management, disaster prevention and control, and scientific research. Using deep learning to accurately extract water bodies from remote sensing imagery can provide data support for decision-making in related fields. However, existing water extraction methods face several challenges. In remote sensing imagery, water bodies are not distributed in isolation. Comprehensively considering long-range dependent spatial relationships can better assist in water extraction. However, existing convolutional neural networks typically rely on convolution kernels for feature extraction, but their receptive fields are often limited by the size of the kernels. In remote sensing imagery, water bodies exhibit significant differences in spatial scale compared to other remotely sensed features. This characteristic makes the effective integration of multi-scale spatial information highly valuable for water extraction. Feature fusion can effectively enable information interaction at different levels, but Deeplab v3+ directly fuses low-level and final high-level semantic features. While this design can quickly integrate key information and reduce computational complexity, the lack of intermediate-level transitions prevents effective reconciliation of semantic differences between low-level and high-level features, hindering feature synergy. Feature maps from other intermediate stages are also underutilized, resulting in a significant amount of valuable spatial information being wasted. To address these issues, a spatially enhanced water extraction network based on multi-scale and long-range dependencies is proposed to improve water extraction. Summary of the Invention
[0003] In order to overcome the shortcomings of the above-mentioned prior art, the present invention discloses a spatially enhanced water body extraction network based on multi-scale and long-range dependency. The present invention optimizes DeepLab v3+ to achieve a high-precision remote sensing water body extraction. A local-global spatial feature enhancement module based on axial non-local attention is designed to introduce long-range dependency information for better water body extraction. A dual spatial multi-scale aggregation module is designed to introduce multi-scale information while improving information utilization. At the decoder stage, a cross-fusion module of spatial feature refinement is used to optimize the feature fusion process.
[0004] The technical steps adopted in the present invention are as follows:
[0005] Step 1: Take an RGB three-channel image as input.
[0006] Step 2: Use ResNet-34 to extract image features.
[0007] Step 3: Obtain multi-scale information through a dual spatial multi-scale aggregation module.
[0008] Step 3.1: Use the channel-enhanced multi-scale module (ASPP) to obtain five feature maps of different scales in the dilated convolution layer. Concatenate the original input feature map with the five feature maps along the channel. Apply SE attention to the concatenated image to obtain a weighted feature map, which is then compressed using a 1×1 convolution.
[0009] Step 3.2: Through the multi-stage multi-scale module. The feature maps of the four stages in the decoder are first processed through 1×1 convolutions to adjust the number of channels. Then, starting from the last feature map, the feature map is first subjected to a 3×3 convolution with stride = 2, reducing its size to half. It is then fused with the output feature map of the next stage from the backbone network through element-wise addition. It is then further processed through a 3×3 convolution, and the resulting feature map is repeated.
[0010] Step 3.3: The results obtained in step 3.1 and step 3.2 are spliced along the channel.
[0011] Step 4: The local spatial feature enhancement component of the local-global spatial feature enhancement module based on axial non-local attention is used. Specifically, the first-stage feature map undergoes global average pooling and global max pooling in the spatial dimension, and the two generated single-channel feature maps are concatenated along the channel dimension. A 7×7 convolution operation is performed, followed by sigmoid activation to generate an attention weight map. This weight map is then multiplied pixel-by-pixel with the original feature map.
[0012] Step 5: Enhance the global spatial features in the local-global spatial feature enhancement module based on axial non-local attention. Combined with the local enhancement part, perform local-global spatial feature enhancement based on axial non-local attention.
[0013] Step 5.1: The second stage feature map is mean pooled in the horizontal and vertical directions respectively.
[0014] Step 5.2: Perform three 1×1 convolutions on the two feature maps obtained in step 5.1 to obtain three feature maps.
[0015] Step 5.3: For the three feature maps obtained in step 5.2, first transpose the third feature map and perform matrix multiplication with the second feature map. The calculated result will be passed through the Softmax function, normalized, and then matrix multiplied with the first feature map.
[0016] Step 5.4: Perform matrix multiplication on the two feature maps obtained in step 5.3. At this time, each pixel in the feature map obtains the global context information of its row and column.
[0017] Step 5.5: Add the features obtained in step 5.4 to the second stage feature map element by element and perform residual connection.
[0018] Step 5.6: The feature maps obtained in step 4 and step 5.5 are concatenated along the channel and then 1×1 convolution is performed to complete feature fusion.
[0019] Step 6: Step 5 and step 3 are combined through the cross fusion module of spatial feature refinement to perform feature fusion.
[0020] Step 6.1: The feature map obtained in step 3 is upsampled to 2 times.
[0021] Step 6.2: Perform global average pooling and global max pooling on the feature maps obtained in Steps 5 and 6.1, respectively, in the spatial dimension. The two single-channel feature maps are concatenated along the channel dimension. A 7×7 convolution is performed, followed by a sigmoid activation function to generate an attention weight map. The weight map is multiplied pixel-by-pixel with the original feature map. A 1×1 convolution is then performed to adjust the number of channels to the same.
[0022] Step 6.3: The feature map obtained in step 6.2 is channel-adjusted and then element-wise multiplied with the corresponding feature maps obtained in step 6.1 and step 5.
[0023] Step 6.4: The two feature maps obtained in step 6.3 are concatenated along the channel and two 3×3 convolutions are performed.
[0024] Step 7: Upsample the feature map obtained in step 6 to the original image size and perform semantic segmentation to obtain the water body extraction result.
[0025] Compared with the prior art, the present invention has the following beneficial effects:
[0026] (1) It is superior to similar and classic remote sensing image water body extraction models in terms of accuracy and visual segmentation effect.
[0027] (2) The method proposed in this paper effectively solves common problems in remote sensing water body extraction, namely, the difficulty in obtaining global information, insufficient utilization of multi-scale information, and the semantic gap caused by the fusion of features at different levels. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 This is the network structure diagram of the present invention
[0029] Figure 2 This is the dual space multi-scale aggregation module diagram of the present invention
[0030] Figure 3 Multi-scale module diagram for channel enhancement of the present invention
[0031] Figure 4 The multi-stage and multi-scale module diagram of the present invention
[0032] Figure 5 This is the local-global spatial feature enhancement module diagram based on axial non-local attention of the present invention
[0033] Figure 6 This is the axial non-local attention module diagram of the present invention
[0034] Figure 7 This is the cross-fusion module diagram of the spatial feature refinement of the present invention
[0035] Specific implementation steps
[0036] The present invention will be further described below with reference to the accompanying drawings.
[0037] This paper designs a spatially enhanced water extraction network based on multi-scale and long-range dependencies. Based on the DeepLab v3+ network, a local-global spatial feature enhancement module based on axial non-local attention is designed to incorporate long-range dependency information. A dual spatial multi-scale aggregation module is also designed to incorporate multi-scale information. At the decoder stage, a cross-fusion module for spatial feature refinement is used to optimize feature fusion.
[0038] As attached Figure 1 As shown in the figure, ResNet-34 is first used as the backbone network to encode semantic information and establish a feature extraction module. The feature map of the last stage, obtained through the channel-enhanced multi-scale module, is concatenated with the feature maps of all stages through the multi-stage multi-scale module along the channel to obtain multi-scale information. Simultaneously, the results of the first and second stage feature maps are input into the local-global spatial feature enhancement module based on axial non-local attention and subjected to convolution to extract features. After the spatial feature enhancement, the multi-scale information feature map is fused through the cross-fusion module of spatial feature refinement and semantic segmentation after upsampling to generate the water body extraction result.
[0039] like Figure 2 As shown in Figure 1, the feature maps of multiple stages after 1×1 convolution and the feature map of the last stage pass through the channel-enhanced multi-scale module and the multi-stage multi-scale module respectively.
[0040] like Figure 3 As shown in Figure 2, the feature map after the ASPP module is concatenated with the original feature map along the channel and then enhanced by SE attention.
[0041] like Figure 4 As shown in the figure, features are transferred from bottom to top. Starting from the feature map of the last stage, a 3×3 convolution with stride = 2 is performed to reduce the size of the feature map to half of its original size. It is then fused with the output feature map of the next stage from the backbone network through element-by-element addition, and then subjected to another 3×3 convolution. The result is used as the initial feature map for the next round.
[0042] like Figure 5 As shown, the first-stage and second-stage feature maps of the encoder are processed through spatial attention and axial non-local attention, respectively.
[0043] like Figure 6 As shown, the input feature map undergoes mean pooling both horizontally and vertically. Three 1×1 convolutions are then performed on each, yielding three feature maps. The third feature map is transposed and matrix-multiplied with the second. The result is normalized using the Softmax function and then matrix-multiplied with the first feature map. The resulting matrix multiplication is then performed on the two feature maps, element-wise added to the input feature map, and a residual connection is performed.
[0044] like Figure 7 As shown in the figure, the high-level feature map is upsampled to the same size as the low-level feature map. Both feature maps are then subjected to global average pooling and global max pooling in the spatial dimension. The resulting two single-channel feature maps are concatenated along the channel dimension, subjected to a 7×7 convolution, and activated with a sigmoid function to generate an attention weight map. The weight map is multiplied pixel-by-pixel with the original feature map to generate an enhanced feature map. Each feature map is then subjected to a 1×1 convolution to adjust the number of channels to match the other. Finally, they are cross-multiplied, the multiplication results are concatenated along the channel dimension, and the features are fused through double convolution.
Claims
1. A spatially enhanced water extraction network based on multi-scale and long-range dependencies. Characterized by: The following steps are involved: The technical steps adopted in the present invention are as follows: Step 1: Take an RGB three-channel image as input. Step 2: Use ResNet-34 to extract image features. Step 3: Obtain multi-scale information through a dual spatial multi-scale aggregation module. Step 3.1: Use the channel-enhanced multi-scale module (ASPP) to obtain five feature maps of different scales in the dilated convolution layer. Concatenate the original input feature map with the five feature maps along the channel. Apply SE attention to the concatenated image to obtain a weighted feature map, which is then compressed using a 1×1 convolution. Step 3.2: Through the multi-stage multi-scale module. The feature maps of the four stages in the decoder are first processed through 1×1 convolutions to adjust the number of channels. Then, starting from the last feature map, the feature map is first subjected to a 3×3 convolution with stride = 2, reducing its size to half. It is then fused with the output feature map of the next stage from the backbone network through element-wise addition. It is then further processed through a 3×3 convolution, and the resulting feature map is repeated. Step 3.3: The results obtained in step 3.1 and step 3.2 are spliced along the channel. Step 4: The local spatial feature enhancement component of the local-global spatial feature enhancement module based on axial non-local attention is used. Specifically, the first-stage feature map undergoes global average pooling and global max pooling in the spatial dimension, and the two generated single-channel feature maps are concatenated along the channel dimension. A 7×7 convolution operation is performed, followed by sigmoid activation to generate an attention weight map. This weight map is then multiplied pixel-by-pixel with the original feature map. Step 5: Enhance the global spatial features in the local-global spatial feature enhancement module based on axial non-local attention. Combined with the local enhancement part, perform local-global spatial feature enhancement based on axial non-local attention. Step 5.1: The second stage feature map is mean pooled in the horizontal and vertical directions respectively. Step 5.2: Perform three 1×1 convolutions on the two feature maps obtained in step 5.1 to obtain three feature maps. Step 5.3: For the three feature maps obtained in step 5.2, first transpose the third feature map and perform matrix multiplication with the second feature map. The calculated result will be passed through the Softmax function, normalized, and then matrix multiplied with the first feature map. Step 5.4: Perform matrix multiplication on the two feature maps obtained in step 5.
3. At this time, each pixel in the feature map obtains the global context information of its row and column. Step 5.5: Add the features obtained in step 5.4 to the second stage feature map element by element and perform residual connection. Step 5.6: The feature maps obtained in step 4 and step 5.5 are concatenated along the channel and then 1×1 convolution is performed to complete feature fusion. Step 6: Step 5 and step 3 are combined through the cross fusion module of spatial feature refinement to perform feature fusion. Step 6.1: The feature map obtained in step 3 is upsampled to 2 times. Step 6.2: Perform global average pooling and global max pooling on the feature maps obtained in Steps 5 and 6.1, respectively, in the spatial dimension. The two single-channel feature maps are concatenated along the channel dimension. A 7×7 convolution is performed, followed by a sigmoid activation function to generate an attention weight map. The weight map is multiplied pixel-by-pixel with the original feature map. A 1×1 convolution is then performed to adjust the number of channels to the same. Step 6.3: The feature map obtained in step 6.2 is channel-adjusted and then element-wise multiplied with the corresponding feature maps obtained in step 6.1 and step 5. Step 6.4: The two feature maps obtained in step 6.3 are concatenated along the channel and two 3×3 convolutions are performed. Step 7: Upsample the feature map obtained in step 6 to the original image size and perform semantic segmentation to obtain the water body extraction result.
2. The method according to claim 1, wherein: The multi-scale feature extraction of the image is performed through the dual space multi-scale aggregation module in step 3.
3. The method according to claim 1, wherein: The spatial features of the encoder output are enhanced through the local-global spatial feature enhancement module based on axial non-local attention proposed in steps 4 and 5.
4. The method according to claim 1, wherein: In step 6, feature fusion is performed through the proposed cross fusion module to achieve interaction of features at different levels.