Multi-scale feature remote sensing image semantic segmentation method and system

Abstract

The invention belongs to the technical field of earth space information. The invention discloses a multi-scale feature remote sensing image semantic segmentation method. The method comprises the following steps: constructing a detail semantic module, a global semantic module and a multilateral feature refining module; a spatial relation sub-block is introduced into the detail semantic module, pooling operation is moved out, a detail sensing network is obtained, and the detail sensing network represents a structure capable of capturing and retaining detail features in the image; introducing a channel relation sub-block into the global semantic module to obtain a global context network; fusing the features of the detail sensing network and the global context network to obtain enhanced feature representation; and performing integration processing on the enhanced feature representation by using a multilateral feature refining module to obtain a refined feature layer. A global-detail multi-scale semantic segmentation network model structure is creatively proposed, a parallel detail semantic module and a global semantic model are established, space and channel dual relationships are integrated, and efficient aggregation of multi-scale and multi-relationship features is realized.

Claims

1. A multi-scale feature remote sensing image semantic segmentation method, characterized in that the method include: Construct detail semantic module, global semantic module and multilateral feature refinement module; Introducing a spatial relationship sub-block into the detail semantic module and removing the pooling operation to obtain a detail-aware network, wherein the detail-aware network represents a structure that can capture and preserve detail features in an image; Introducing a channel relationship sub-block into the global semantic module to obtain a global context network, wherein the global context network represents obtaining high-level semantic information when capturing global context information of an image; Fusing features of the detail-aware network and the global context network to obtain an enhanced feature representation; The enhanced feature representation is integrated using the polygonal feature refinement module to obtain a refined feature layer.

2. The multi-scale feature remote sensing image semantic segmentation method according to claim 1, characterized in that: The steps of constructing the detail semantic module, the global semantic module and the multilateral feature refinement module include: Constructing a first convolutional neural network to obtain the detail semantic module; Constructing a second convolutional neural network to obtain the global semantic module; A feature layer size adjustment convolution layer is constructed to obtain the polygonal feature refinement module, wherein the feature layer size adjustment convolution layer can match and adjust the size of the detail feature map of the detail semantic module and the global feature map of the global semantic module.

3. The multi-scale feature remote sensing image semantic segmentation method according to claim 1, characterized in that: The step of introducing the spatial relationship sub-block into the detail semantic module and removing the pooling operation to obtain the detail perception network includes: Constructing a plurality of non-downsampling modules and the spatial relationship sub-blocks; Receiving the input of the network layer, and processing it through the plurality of non-downsampling modules in sequence to obtain the processing result of the non-downsampling module; Using the spatial relationship sub-block, receiving the processing result of the non-downsampling module; The detail perception network is obtained by combining a plurality of the non-downsampling modules and the spatial relationship sub-blocks.

4. The multi-scale feature remote sensing image semantic segmentation method according to claim 3, characterized in that: The step of receiving the processing result of the non-downsampling module by using the spatial relationship sub-block comprises: According to the processing result of the non-downsampling module, the first feature map X is input to the spatial relationship sub-block, where X∈R C ×H×W , C represents the number of channel layers, H and W represent the height and width of the feature map respectively; In the spatial relationship sub-block: Performing a convolution operation on the first feature map X to obtain a spatial convolution result; Reshape and transpose the spatial convolution result to obtain a feature layer Q; Reshape the spatial convolution result to obtain a feature layer K; The feature size of the feature layer Q and the feature layer K is R C×N , where N = H × W; Perform matrix multiplication on the feature layer Q and the feature layer K, and use the softmax layer to calculate the spatial attention map S, where the calculation formula of the spatial attention map S is: Among them, S∈R N×N , S ij Represents the influence of the pixel at position j on the pixel at position i; The first feature map X is convolved to obtain the target feature map V, where V∈R C×H×W , and reshaped into the same form as the characteristic layer Q and the characteristic layer K; Reshape and transpose the spatial attention map S to obtain a spatial attention map processing result; Perform matrix multiplication calculation on the spatial attention map result and the target feature map V to obtain a feature map O; The scale parameter a is used to multiply the feature map O, and the first feature map X is added to obtain the first feature map Y, where the first feature map Y∈R C×H×W , the calculation formula of the first feature map Y is: Among them, A i represents the learnable bias term, corresponding to the i-th pixel in the first feature map Y.

5. The multi-scale feature remote sensing image semantic segmentation method according to claim 1, characterized in that: The step of introducing the channel relationship sub-block into the global semantic module to obtain a global context network comprises: Construct multiple residual modules and channel relationship sub-blocks; Receiving the input of the network layer, and processing it through the plurality of residual modules in sequence to obtain a residual module processing result; Using the channel relationship sub-block, receiving the residual module processing result; The global context network is obtained by combining a plurality of the residual modules and the channel relationship sub-blocks.

6. The multi-scale feature remote sensing image semantic segmentation method according to claim 1, characterized in that: The step of receiving the processing result of the residual module by using the channel relationship sub-block includes: According to the processing result of the residual module, the second feature map X is input to the channel relationship sub-block, where X∈R C×H×W , C represents the number of channel layers, H and W represent the height and width of the feature map respectively; In the channel relation sub-block: The second feature map X is reshaped to obtain feature Q, where the feature Q∈R C×N , C represents the number of channel layers, N = H × W, H and W represent the height and width of the feature map respectively; The second feature map X is reshaped and transposed to obtain feature K, where the feature K∈R C×N ; According to the feature K and the feature Q, matrix multiplication is performed to merge the features, and the channel attention map E is calculated using the softmax layer, wherein the calculation formula of the channel attention map E is: Among them, E∈R C×N , E ij Represents the influence of the pixel at position j on the channel at position i; Transpose the channel attention map E, and perform matrix multiplication calculation with the second feature map X to obtain a channel attention map processing result; Reshape the processing result of the channel attention map to obtain a channel feature map; The channel feature map is multiplied by the scale parameter β, and the second feature map X is added element by element to obtain the second feature map Y, where the second feature map Y∈R C×H×W , the calculation formula of the second feature map Y is: Among them, X i Represents the pixel value at the i-th position in the second feature map X.

7. The multi-scale feature remote sensing image semantic segmentation method according to claim 1, characterized in that: The step of fusing the features of the detail perception network and the global context network to obtain an enhanced feature representation comprises: According to the detail perception network, an initial detail feature map is obtained; According to the global context network, an initial global feature map is obtained; Performing a convolution operation on the initial detail feature map, adjusting the spatial size and feature size of the initial detail feature map, and obtaining a target detail feature map; Performing a convolution operation on the initial global feature map, adjusting the spatial size and feature size of the initial global feature map, and obtaining a target global feature map, wherein the spatial size and feature size of the target global feature map and the target detail feature map are compatible; Using the global context network, processing the target detail feature map to obtain a next layer of global feature map; Using the detail perception network, the target global feature map is processed to obtain a next layer of detail feature map; An enhanced feature representation is obtained according to the next-layer global feature map and the next-layer detail feature map.

8. The multi-scale feature remote sensing image semantic segmentation method according to claim 7, characterized in that: The step of integrating the enhanced feature representation using the polygonal feature refinement module to obtain a refined feature layer comprises: Superimposing the next layer of global feature map and the next layer of detail feature map to obtain initial integrated features; Performing a convolution operation on the integrated features to obtain target integrated features; The target integrated features are respectively subjected to matrix multiplication calculation with the features of the next layer of detail feature map and the next layer of global feature map after dilated spatial pyramid pooling to obtain a refined feature layer.

9. A multi-scale feature remote sensing image semantic segmentation system, characterized in that: Construction module, used to construct detail semantic module, global semantic module and multilateral feature refinement module; A detail semantic module, used to introduce a spatial relationship sub-block into the detail semantic module and remove the pooling operation to obtain a detail perception network, wherein the detail perception network represents a structure that can capture and preserve detail features in the image; A global semantic module, used for introducing a channel relationship sub-block into the global semantic module to obtain a global context network, wherein the global context network represents obtaining high-level semantic information when capturing global context information of an image; An enhanced feature representation module, used to fuse the features of the detail perception network and the global context network to obtain an enhanced feature representation; The polygonal feature refinement module is used to utilize the polygonal feature refinement module to integrate the enhanced feature representation to obtain a refined feature layer.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images