A semantic segmentation method for high-resolution remote sensing images based on rich scale feature enhancement

Abstract

This invention relates to a semantic segmentation method for high-resolution remote sensing images, specifically a method based on rich-scale feature enhancement. The method comprises the following steps: Step 1: Dividing the high-resolution remote sensing image into multiple sub-images according to information content levels and setting indices; Step 2: Extracting feature maps from different layers for each sub-image; Step 3: Using an attention mechanism to fuse inter-layer features of the feature maps from different layers to obtain the feature maps corresponding to each sub-image; then, restoring the spatial positions of the feature maps corresponding to each sub-image according to the indices set in Step 1; Step 4: Fusing intra-layer features of the feature maps corresponding to each sub-image to obtain a final feature map; then, performing linear interpolation and upsampling on the final feature map to obtain the final segmentation map. This invention effectively solves the problem of low segmentation accuracy in existing high-resolution remote sensing image semantic segmentation methods and is applicable to semantic segmentation of high-resolution remote sensing images.

Description

Technical Field

[0001] This invention relates to a semantic segmentation method for high-resolution remote sensing images, specifically a semantic segmentation method for high-resolution remote sensing images based on rich scale feature enhancement. Background Technology

[0002] In recent years, with the rapid development of satellite and airborne remote sensing technologies, a large number of high-resolution remote sensing images (hereinafter referred to as HSAR images) have been obtained. HSAR images can provide more spatial details and geometric feature information, and their accurate and efficient interpretation has broad application prospects in precision agriculture, urban planning, and efficient assessment and utilization of natural resources. Semantic segmentation, as a dense prediction task, can assign a category label to each pixel, that is, automatically realize accurate extraction of surface features and land cover classification, thus becoming an important research direction for HSAR image interpretation. However, practice shows that existing HSAR image semantic segmentation methods have the following problems due to their inherent limitations: First, when performing sub-image division, existing HSAR image semantic segmentation methods cannot accurately assess the amount of information contained in different locations in the entire image, resulting in their inability to adaptively process sparse and dense areas of ground features in the image, thus leading to low segmentation accuracy. Secondly, existing high-resolution remote sensing image semantic segmentation methods extract only single-layer or multi-layer feature maps from the entire image during feature extraction, making it difficult to adapt to the scale differences of ground objects in the image, which also leads to low segmentation accuracy. Therefore, it is necessary to invent a high-resolution remote sensing image semantic segmentation method based on scale-rich feature enhancement to solve the problem of low segmentation accuracy in existing high-resolution remote sensing image semantic segmentation methods. Summary of the Invention

[0003] To address the problem of low segmentation accuracy in existing high-resolution remote sensing image semantic segmentation methods, this invention provides a high-resolution remote sensing image semantic segmentation method based on rich scale feature enhancement.

[0004] This invention is achieved using the following technical solution:

[0005] A semantic segmentation method for high-resolution remote sensing images based on rich scale feature enhancement is implemented through the following steps:

[0006] Step 1: Send the high-resolution remote sensing image into the information quantification and grading module. The information quantification and grading module first calculates the amount of information contained in different locations in the high-resolution remote sensing image, then divides the amount of information into multiple levels according to the size, and then divides the high-resolution remote sensing image into multiple sub-images according to the information level and sets an index.

[0007] Step 2: Feed each sub-image into the encoder of the Res2Net backbone network. The encoder of the Res2Net backbone network extracts feature maps from different layers for each sub-image.

[0008] Step 3: The feature maps of different layers are fed into the inter-layer feature fusion module. The inter-layer feature fusion module uses an attention mechanism to fuse the feature maps of different layers to obtain the feature maps corresponding to each sub-image. Then, the spatial position of the feature map corresponding to each sub-image is restored according to the index set in Step 1.

[0009] Step 4: The feature maps corresponding to each sub-image are sent to the intra-layer feature fusion module. The intra-layer feature fusion module performs intra-layer feature fusion on the feature maps corresponding to each sub-image to obtain a final feature map. Then, the final feature map is linearly interpolated and upsampled to obtain the final segmentation map.

[0010] In step one, the amount of information contained at different locations in the high-resolution remote sensing image is calculated using the following formula:

[0011]

[0012] In the formula: This represents the amount of information contained at position (i,j) in the high-resolution remote sensing image I; This represents the probability that the pixel at position (i,j) in the high-resolution remote sensing image I belongs to the class c.

[0013] In step two, for the subgraph with the lowest information level, the feature maps of the lowest and highest layers are extracted; for the subgraph with the highest information level, feature maps of all layers are extracted.

[0014] In step four, 1×1 convolution, 3×3 convolution, BN layer, and ReLU layer are used sequentially to fuse the feature maps corresponding to each sub-image within each layer.

[0015] Compared with existing high-resolution remote sensing image semantic segmentation methods, the high-resolution remote sensing image semantic segmentation method based on rich scale feature enhancement described in this invention has the following advantages: First, when dividing sub-images, this invention calculates and classifies the information content contained in different locations within the entire image, achieving accurate assessment of the information content contained in different locations. This enables adaptive processing of sparse and dense areas of ground objects in the image, thereby effectively improving segmentation accuracy. Second, when extracting features, this invention extracts feature maps of different layers for each sub-image, thus adapting well to the scale differences of ground objects in the image, further improving segmentation accuracy. Figure 2 , Figure 3 As shown, by comparing the ground truth map of the high-resolution remote sensing image with the final segmentation map, it can be concluded that the present invention has a very high segmentation accuracy.

[0016] This invention effectively solves the problem of low segmentation accuracy in existing high-resolution remote sensing image semantic segmentation methods and is applicable to semantic segmentation of high-resolution remote sensing images. Attached Figure Description

[0017] Figure 1 This is a flowchart of the present invention.

[0018] Figure 2 This is a comparison diagram of the original image, ground truth image, and final segmentation image of a high-resolution remote sensing image. Figure 1 .

[0019] Figure 3 This is a comparison diagram of the original image, ground truth image, and final segmentation image of a high-resolution remote sensing image. Figure 2 . Detailed Implementation

[0020] A semantic segmentation method for high-resolution remote sensing images based on rich scale feature enhancement is implemented through the following steps:

[0021] Step 1: Send the high-resolution remote sensing image into the information quantification and grading module. The information quantification and grading module first calculates the amount of information contained in different locations in the high-resolution remote sensing image, then divides the amount of information into multiple levels according to the size, and then divides the high-resolution remote sensing image into multiple sub-images according to the information level and sets an index.

[0022] Step 2: Feed each sub-image into the encoder of the Res2Net backbone network. The encoder of the Res2Net backbone network extracts feature maps from different layers for each sub-image.

[0023] Step 3: The feature maps of different layers are fed into the inter-layer feature fusion module. The inter-layer feature fusion module uses an attention mechanism to fuse the feature maps of different layers to obtain the feature maps corresponding to each sub-image. Then, the spatial position of the feature map corresponding to each sub-image is restored according to the index set in Step 1.

[0024] Step 4: The feature maps corresponding to each sub-image are sent to the intra-layer feature fusion module. The intra-layer feature fusion module performs intra-layer feature fusion on the feature maps corresponding to each sub-image to obtain a final feature map. Then, the final feature map is linearly interpolated and upsampled to obtain the final segmentation map.

[0025] In step one, the amount of information contained at different locations in the high-resolution remote sensing image is calculated using the following formula:

[0026]

[0027] In the formula: This represents the amount of information contained at position (i,j) in the high-resolution remote sensing image I; This represents the probability that the pixel at position (i,j) in the high-resolution remote sensing image I belongs to the class c.

[0028] In step two, for the subgraph with the lowest information level, the feature maps of the lowest and highest layers are extracted; for the subgraph with the highest information level, feature maps of all layers are extracted.

[0029] In step four, 1×1 convolution, 3×3 convolution, BN layer, and ReLU layer are used sequentially to fuse the feature maps corresponding to each sub-image within each layer.

[0030] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention.

Claims

1. A semantic segmentation method for high-resolution remote sensing images based on rich scale feature enhancement, characterized in that: This method is implemented using the following steps: Step 1: Send the high-resolution remote sensing image into the information quantification and grading module. The information quantification and grading module first calculates the amount of information contained in different locations in the high-resolution remote sensing image, then divides the amount of information into multiple levels according to the size, and then divides the high-resolution remote sensing image into multiple sub-images according to the information level and sets an index. Step 2: Feed each sub-image into the encoder of the Res2Net backbone network. The encoder of the Res2Net backbone network extracts feature maps from different layers for each sub-image. Step 3: The feature maps of different layers are fed into the inter-layer feature fusion module. The inter-layer feature fusion module uses an attention mechanism to fuse the feature maps of different layers to obtain the feature maps corresponding to each sub-image. Then, the spatial position of the feature map corresponding to each sub-image is restored according to the index set in Step 1. Step 4: The feature maps corresponding to each sub-image are sent to the intra-layer feature fusion module. The intra-layer feature fusion module performs intra-layer feature fusion on the feature maps corresponding to each sub-image to obtain a final feature map. Then, the final feature map is linearly interpolated and upsampled to obtain the final segmentation map.

2. The semantic segmentation method for high-resolution remote sensing images based on rich scale feature enhancement according to claim 1, characterized in that: In step one, the amount of information contained at different locations in the high-resolution remote sensing image is calculated using the following formula: In the formula: P(x) represents the amount of information contained at position (i,j) in the high-resolution remote sensing image I; c i,j ) represents the probability that the pixel at position (i,j) in the high-resolution remote sensing image I belongs to the class c.

3. The semantic segmentation method for high-resolution remote sensing images based on rich scale feature enhancement according to claim 1, characterized in that: In step two, for the subgraph with the lowest information level, the feature maps of the lowest and highest layers are extracted; for the subgraph with the highest information level, feature maps of all layers are extracted.

4. The semantic segmentation method for high-resolution remote sensing images based on rich scale feature enhancement according to claim 1, characterized in that: In step four, 1×1 convolution, 3×3 convolution, BN layer, and ReLU layer are used sequentially to fuse the feature maps corresponding to each sub-image within each layer.

Images