A coarse-grained category information guided refinement method for remote sensing feature classification

Abstract

The present application relates to a kind of coarse granularity class information guide refinement remote sensing SAR image feature classification method, belong to remote sensing scene feature classification technical field.For the problems existing in the present SAR remote sensing image feature classification method, such as high-resolution SAR image is susceptible to noise interference, feature distribution is complex, target scale difference is significant, lack of color information, etc., leading to the phenomenon of boundary blur and class confusion, a more robust and discriminant classification method is proposed.The feature classification network model proposed in the present application is composed of three key modules: encoder module, decoder module and coarse-grained class information guide module.The method of the present application effectively alleviates the common noise interference in SAR image, scale inconsistency, color information loss and class boundary blur, significantly improves the accuracy of feature classification and the robustness of the model, and has good engineering practicability and wide application prospect.

Description

Technical Field

[0001] This invention relates to the field of SAR remote sensing image land cover classification technology, specifically to a refined remote sensing land cover classification method guided by coarse-grained category information. Background Technology

[0002] Remote sensing land cover classification aims to categorize pixels in an image into specific classes, enabling precise identification and analysis of terrain features and supporting intelligent applications of remote sensing data in disaster monitoring, agricultural production, urban planning, and military reconnaissance. Synthetic Aperture Radar (SAR) remote sensing images offer advantages such as all-weather, all-day imaging capabilities, high resolution, and wide coverage, allowing for multi-polarization and multi-angle observations, and have shown great promise in practical applications. However, because SAR imaging relies on the scattering characteristics between radar waves and land features, images often contain significant speckle noise and shadow interference, severely impacting visual quality and the accuracy of subsequent analysis tasks. Furthermore, high-resolution SAR images exhibit complex distributions of land cover types, significant differences in target scale, and a lack of color information, making the problems of blurred land cover boundaries and category confusion particularly prominent, further increasing the difficulty of land cover classification.

[0003] Existing research has shown that multi-scale feature fusion plays a crucial role in SAR image land cover classification, effectively preserving rich information from different land cover types, enhancing the model's discriminative ability, and improving the segmentation accuracy of edge regions. Combining feature extraction and fusion with contextual information helps alleviate problems such as blurred boundaries and classification errors. However, current research has focused less on how to enhance inter-class discriminability through prototype learning methods, and the application potential of prototype learning in SAR image land cover classification has not been fully explored and validated. Therefore, improving the model's robustness to multi-scale features and enhancing inter-class discriminative ability are key paths to solving the problems of blurred boundaries and class confusion in SAR remote sensing images.

[0004] Based on the above analysis of the characteristics of SAR remote sensing images and the challenges of land cover classification, this invention proposes a refined remote sensing land cover classification method guided by granular category information. This method designs a multi-scale feature fusion module to enhance contextual modeling capabilities, better preserve edge and structural information in the image, and improve robustness against noise interference. Simultaneously, it introduces a coarse-grained category information guidance mechanism, combined with dynamic semantic prototype representation, to effectively improve the ability to distinguish different land cover categories in SAR images, thereby achieving more accurate pixel-level land cover classification results. Summary of the Invention

[0005] This invention aims to address the shortcomings of existing technologies, particularly the boundary blurring and category confusion caused by problems such as a large amount of shadows and noise in SAR remote sensing image land cover classification, complex land cover distribution, significant differences in target scale, and lack of color information. It proposes a refined remote sensing land cover classification method guided by coarse-grained category information to effectively improve classification accuracy and model robustness.

[0006] The method of the present invention is achieved through the following technical solution.

[0007] A refined remote sensing land cover classification method guided by coarse-grained category information includes the following steps:

[0008] Step 1: Obtain the SAR remote sensing image ground feature classification dataset and divide it into training set and validation set;

[0009] Step 2: Preprocess the training set obtained in Step 1;

[0010] Step 3: Design a remote sensing image land cover classification network;

[0011] Step 4: Use the preprocessed training set obtained in Step 2 to train the remote sensing image land cover classification network designed in Step 3. During training, the network parameters are iteratively optimized. After training, the remote sensing image land cover classification network model is obtained.

[0012] Step 5: Input the remote sensing image to be segmented into the remote sensing image land cover classification network model obtained in Step 4 to obtain the remote sensing land cover classification result.

[0013] In step 1, the remote sensing images in the SAR remote sensing image land cover classification dataset are obtained from the SAR remote sensing image portion of the WHU-OPT-SAR dataset. The SAR remote sensing image land cover classification dataset is randomly divided into a training set and a validation set in an 8:2 ratio.

[0014] The specific method for preprocessing the training set in step 2 is as follows:

[0015] First, the remote sensing images in the training set, i.e. the original images, with a size of 5556×3704, are cut into 1389×926 slices in a 4×4 non-overlapping pattern. These slices are then randomly scaled within the range of [0.5, 2.0]. Next, the scaled slices are cut into square images with a size of 512×512. Finally, the resulting square images are randomly flipped horizontally or vertically to obtain the preprocessed training set.

[0016] In step 3, the designed remote sensing image land cover classification network includes an encoder module, a decoder module, and a coarse-grained category information guidance module; the decoder module is a multi-scale feature fusion decoder module.

[0017] The encoder module is used to input the preprocessed training set obtained in step 2 into the encoder structure based on the convolutional neural network to extract multi-scale feature representations. The encoder module uses a residual network as the encoder structure. The input image is sequentially processed by four residual modules to extract and downsample the features layer by layer to obtain multi-scale features.

[0018] The decoder module is a Unet structure decoder with the addition of the ASPP module, which includes three upsampling modules and one ASPP module; each upsampling module includes one deconvolution and two regular convolution operations; the multi-scale features obtained by the encoder module are first passed through the upsampling module to obtain the upsampled feature map;

[0019] The upsampled feature map is concatenated with the feature map of the corresponding scale in the encoder module along the channel dimension to obtain the concatenated feature. The concatenated feature and the deepest feature in the multi-scale feature extracted by the encoder module are input into the ASPP module to extract rich multi-scale context information and obtain the output feature of the decoder module.

[0020] The coarse-grained category information guidance module is used to generate a dynamic prototype from the output features of the decoder module, and then further compress the dynamic prototype to obtain a class embedding vector. Finally, the class embedding vector is used to weight the output of the decoder module to enhance the discriminability between different classes and obtain the enhanced features.

[0021] The specific method for generating the dynamic prototype is as follows: First, the output features of the decoder module are mapped to a category feature space to generate a class-related feature representation; then, this class-related feature representation is multiplied by the output features of the decoder module to generate a dynamic prototype that can represent the spatial distribution of different categories in the image, and can respond to changes in the content of the input image. The dynamic prototype is represented as follows:

[0022]

[0023] Where X represents the output feature of the decoder module. This represents a 1×1 convolution. It is a matrix multiplication operation. This represents the generated dynamic prototype.

[0024] The compressed dynamic prototype is:

[0025]

[0026] in, This represents the first fully connected layer. This represents the compressed dynamic prototype.

[0027] The enhanced features are:

[0028]

[0029] Where LN is layer normalization and ReLU is the activation function. This represents the obtained context information. This represents the Hadamard product with broadcast mechanism. This represents the enhanced features.

[0030] In step 4, training is performed using the cross-entropy function. The specific training steps include:

[0031] The SAR remote sensing images and their corresponding image segmentation labels in the preprocessed training set are processed by a remote sensing image land cover classification network to generate a dense prediction mask for each remote sensing image.

[0032] The dense prediction mask and the corresponding image segmentation label are input into the cross-entropy loss function to calculate the loss value;

[0033] The specific formula for the cross-entropy function is:

[0034]

[0035] in, N C is the total number of pixels in the SAR remote sensing image, and C is the number of categories. It is a pixel The true label, It is a pixel The predicted probability of belonging to category c.

[0036] In step 5, the specific method for obtaining the remote sensing land cover classification results is as follows:

[0037] The remote sensing image to be segmented is input into the trained remote sensing image land cover classification network model to obtain the segmentation results. The overall accuracy, average accuracy and average intersection-over-union ratio of the remote sensing image segmentation results are calculated and used as the evaluation results of the remote sensing image land cover classification network for the segmentation of remote sensing images.

[0038] Compared with the prior art, the present invention has the following beneficial technical effects:

[0039] This invention proposes a coarse-grained category-guided SAR remote sensing image land cover classification method. It integrates the Unet decoder with a multi-scale context-aware ASPP module, effectively enhancing the model's robustness to noise and its global context modeling capability. Simultaneously, a dynamic category prototype guidance mechanism is introduced, which enhances semantic features through class embedding vector weighting, thereby improving inter-class discriminability and representation accuracy. The overall method significantly improves classification accuracy while preserving boundary details, demonstrating good practicality and engineering application value.

[0040] This invention relates to a refined remote sensing SAR image land cover classification method guided by coarse-grained category information, belonging to the field of remote sensing scene land cover classification technology. Addressing the problems of high-resolution SAR images being susceptible to noise interference, complex land cover distribution, significant target scale differences, and lack of color information in existing SAR remote sensing image land cover classification methods, leading to blurred boundaries and category confusion, this invention proposes a more robust and discriminative classification method. The proposed land cover classification network model consists of three key modules: an encoder module, a decoder module, and a coarse-grained category information guidance module. First, a convolutional neural network is used to construct an encoder to extract multi-scale features from the input image. Then, a decoder based on a UNet structure performs layer-by-layer upsampling and skip connections to fuse the encoded features at different scales, achieving effective integration of high and low-level semantic information. Simultaneously, an ASPP (Atrous Spatial Pyramid Pooling) module is introduced to perform multi-scale dilated convolution processing on the deepest features of the encoder, extracting rich contextual information and further enhancing the model's global perception capability. Based on feature fusion, this invention designs a coarse-grained category information guidance module. This module dynamically generates category semantic prototypes by constructing the relationship between class space features and semantic space features, and performs weighted modulation on the semantic feature maps, thereby enhancing the discriminative power between different land cover categories and improving the model's ability to classify complex land cover scenes. This invention effectively alleviates common problems in SAR images such as noise interference, scale inconsistency, missing color information, and blurred category boundaries, significantly improving the accuracy of land cover classification and the robustness of the model, demonstrating good engineering practicality and broad application prospects. Attached Figure Description

[0041] Figure 1 Images and labels of the SAR remote sensing image dataset used in this example;

[0042] Figure 2 This is a schematic diagram of the SAR remote sensing image land cover classification prediction results for an example. Detailed Implementation

[0043] The method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0044] The method steps of this invention are as follows:

[0045] This invention aims to address the shortcomings of existing technologies, particularly the boundary blurring and category confusion caused by problems such as a large amount of shadows and noise in SAR remote sensing image land cover classification, complex land cover distribution, significant differences in target scale, and lack of color information. It proposes a refined remote sensing land cover classification method guided by coarse-grained category information to effectively improve classification accuracy and model robustness.

[0046] The method of the present invention is achieved through the following technical solution.

[0047] A refined remote sensing land cover classification method guided by coarse-grained category information includes the following steps:

[0048] Step 1: Obtain the SAR remote sensing image ground feature classification dataset and divide it into training set and validation set;

[0049] Step 2: Preprocess the training set obtained in Step 1;

[0050] Step 3: Design a remote sensing image land cover classification network;

[0051] Step 4: Use the preprocessed training set obtained in Step 2 to train the remote sensing image land cover classification network designed in Step 3. During training, the network parameters are iteratively optimized. After training, the remote sensing image land cover classification network model is obtained.

[0052] Step 5: Input the remote sensing image to be segmented into the remote sensing image land cover classification network model obtained in Step 4 to obtain the remote sensing land cover classification result.

[0053] In step 1, the remote sensing images in the SAR remote sensing image land cover classification dataset are obtained from the SAR remote sensing image portion of the WHU-OPT-SAR dataset. The SAR remote sensing image land cover classification dataset is randomly divided into a training set and a validation set in an 8:2 ratio.

[0054] The specific method for preprocessing the training set in step 2 is as follows:

[0055] First, the remote sensing images in the training set, i.e. the original images, with a size of 5556×3704, are cut into 1389×926 slices in a 4×4 non-overlapping pattern. These slices are then randomly scaled within the range of [0.5, 2.0]. Next, the scaled slices are cut into square images with a size of 512×512. Finally, the resulting square images are randomly flipped horizontally or vertically to obtain the preprocessed training set.

[0056] In step 3, the designed remote sensing image land cover classification network includes an encoder module, a decoder module, and a coarse-grained category information guidance module; the decoder module is a multi-scale feature fusion decoder module.

[0057] The encoder module is used to input the preprocessed training set obtained in step 2 into the encoder structure based on the convolutional neural network to extract multi-scale feature representations. The encoder module uses a residual network as the encoder structure. The input image is sequentially processed by four residual modules to extract and downsample the features layer by layer to obtain multi-scale features.

[0058] The decoder module is a Unet structure decoder with the addition of the ASPP module, which includes three upsampling modules and one ASPP module; each upsampling module includes one deconvolution and two regular convolution operations; the multi-scale features obtained by the encoder module are first passed through the upsampling module to obtain the upsampled feature map;

[0059] The upsampled feature map is concatenated with the feature map of the corresponding scale in the encoder module along the channel dimension to obtain the concatenated feature. The concatenated feature and the deepest feature in the multi-scale feature extracted by the encoder module are input into the ASPP module to extract rich multi-scale context information and obtain the output feature of the decoder module.

[0060] The coarse-grained category information guidance module is used to generate a dynamic prototype from the output features of the decoder module, and then further compress the dynamic prototype to obtain a class embedding vector. Finally, the class embedding vector is used to weight the output of the decoder module to enhance the discriminability between different classes and obtain the enhanced features.

[0061] The specific method for generating the dynamic prototype is as follows: First, the output features of the decoder module are mapped to a category feature space to generate a class-related feature representation; then, this class-related feature representation is multiplied by the output features of the decoder module to generate a dynamic prototype that can represent the spatial distribution of different categories in the image, and can respond to changes in the content of the input image. The dynamic prototype is represented as follows:

[0062]

[0063] Where X represents the output feature of the decoder module. This represents a 1×1 convolution. It is a matrix multiplication operation. This represents the generated dynamic prototype.

[0064] The compressed dynamic prototype is:

[0065]

[0066] in, This represents the first fully connected layer. This represents the compressed dynamic prototype.

[0067] The enhanced features are:

[0068]

[0069] Where LN is layer normalization and ReLU is the activation function. This represents the obtained context information. This represents the Hadamard product with broadcast mechanism. This represents the enhanced features.

[0070] In step 4, training is performed using the cross-entropy function. The specific training steps include:

[0071] The SAR remote sensing images and their corresponding image segmentation labels in the preprocessed training set are processed by a remote sensing image land cover classification network to generate a dense prediction mask for each remote sensing image.

[0072] The dense prediction mask and the corresponding image segmentation label are input into the cross-entropy loss function to calculate the loss value;

[0073] The specific formula for the cross-entropy function is:

[0074]

[0075] in, N C is the total number of pixels in the SAR remote sensing image, and C is the number of categories. It is a pixel The true label, It is a pixel The predicted probability of belonging to category c.

[0076] In step 5, the specific method for obtaining the remote sensing land cover classification results is as follows:

[0077] The remote sensing image to be segmented is input into the trained remote sensing image land cover classification network model to obtain the segmentation results. The overall accuracy, average accuracy and average intersection-over-union ratio of the remote sensing image segmentation results are calculated and used as the evaluation results of the remote sensing image land cover classification network for the segmentation of remote sensing images.

[0078] Example:

[0079] The embodiments of the present invention include the following steps:

[0080] Step 1: SAR Remote Sensing Image Data Processing. This embodiment uses SAR images and semantic labels from the open-source multi-source remote sensing image semantic segmentation dataset WHU-OPT-SAR, such as... Figure 1 As shown, the dataset contains 100 images, each 5556×3704 pixels in size. There are seven categories: farmland, forest, city, village, water body, road, and others. The dataset is randomly divided into training and validation sets in an 8:2 ratio. Based on the default data augmentation strategy in the mmsegmentation framework, the training dataset undergoes the following preprocessing and data augmentation steps:

[0081] Image resizing: To facilitate training and feature extraction, all images were non-overlappingly sliced ​​into 1389×926 pixels before training. During training, these images were randomly scaled by [0.5, 2.0] and then cut into 512×512 square images as input to the segmentation network.

[0082] Image normalization and data augmentation: Image normalization, by subtracting the mean from the pixel value of each channel and dividing by the standard deviation, makes the data distribution approximate a standard normal distribution, which can accelerate convergence and improve training results. After image normalization, the image undergoes a random flipping data augmentation step to enhance data diversity and improve the robustness of the model.

[0083] Step 2: Model building.

[0084] Construct the encoder module. Input the preprocessed training set obtained in step 1 into the encoder structure based on the convolutional neural network to extract multi-scale feature representations. The encoder module adopts a residual network as the encoder structure. The input image is sequentially passed through four residual modules for layer-by-layer feature extraction and downsampling to obtain multi-scale features.

[0085] A decoder module is constructed by adding an ASPP module to the Unet structure decoder, which includes three upsampling modules and one ASPP module. Each upsampling module includes one deconvolution and two regular convolution operations. The multi-scale features obtained by the encoder module are first passed through the upsampling module to obtain the upsampled feature map.

[0086] The upsampled feature map is concatenated with the feature map of the corresponding scale in the encoder module along the channel dimension to obtain the concatenated feature. The concatenated feature and the deepest feature in the multi-scale feature extracted by the encoder module are input into the ASPP module to extract rich multi-scale context information and obtain the output feature of the decoder module.

[0087] A coarse-grained category information guidance module is constructed. First, the output features of the decoder module are used to generate a dynamic prototype. Then, the dynamic prototype is further compressed to obtain class embedding vectors. Finally, the output of the decoder module is weighted using the class embedding vectors to enhance the discriminative power between different classes, resulting in enhanced features.

[0088] Step 3: Model Training. The remote sensing image dataset described in Step 1 is input into the remote sensing image land cover classification network constructed in Step 2 for training. The loss function in the deep semantic segmentation network is cross-entropy loss, and the AdamW optimization algorithm is used. The initial learning rate is 1.2 × 10⁻⁶. -6The momentum parameters were set to β1=0.9 and β2=0.999, and the weight decay coefficient was set to 0.01. The learning rate for classifying the head structure was 10 times that of other parts to accelerate the learning rate of the head. The model was trained for a total of 140 epochs. The ResNet encoder module used transfer learning to load the ResNet50 model parameters. After training, the model with the best performance on the validation set was selected as the final model.

[0089] Step 4: Predict land cover classification from remote sensing images. Using the final model obtained in Step 2, predict land cover classification from the tested SAR remote sensing image data and output the final results.

[0090] Compared to the Deeplabv3+ model, the method of this invention improves the MIoU, mAcc, and OA of the model trained on the WHU-OPT-SAR dataset by 2.26%, 3.5%, and 0.88%, respectively. The classification results are shown below. Figure 2 As can be seen from the classification results, the roads in Deeplabv3+ are interrupted and missing, the forest shape is irregular and easily confused with surrounding samples, while the roads in the method proposed in this invention are more complete and can classify more complete forest categories.

[0091] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for fine-grained remote sensing land cover classification guided by coarse-grained category information, characterized in that... Includes the following steps: Step 1: Obtain the SAR remote sensing image ground feature classification dataset and divide it into training set and validation set; Step 2: Preprocess the training set obtained in Step 1; Step 3: Design a remote sensing image land cover classification network; Step 4: Use the preprocessed training set obtained in Step 2 to train the remote sensing image land cover classification network designed in Step 3. During training, the network parameters are iteratively optimized. After training, the remote sensing image land cover classification network model is obtained. Step 5: Input the remote sensing image to be segmented into the remote sensing image land cover classification network model obtained in Step 4 to obtain the remote sensing land cover classification result; In step 3, the designed remote sensing image land cover classification network includes an encoder module, a decoder module, and a coarse-grained category information guidance module; The decoder module is a multi-scale feature fusion decoder module; The encoder module is used to input the preprocessed training set obtained in step 2 into the encoder structure based on the convolutional neural network to extract multi-scale feature representations. The encoder module uses a residual network as the encoder structure. The input image is sequentially processed by four residual modules to extract and downsample the features layer by layer to obtain multi-scale features. The decoder module is a Unet structure decoder with the addition of the ASPP module, which includes three upsampling modules and one ASPP module; each upsampling module includes one deconvolution and two regular convolution operations; the multi-scale features obtained by the encoder module are first passed through the upsampling module to obtain the upsampled feature map; The upsampled feature map is concatenated with the feature map of the corresponding scale in the encoder module along the channel dimension to obtain the concatenated feature. The concatenated feature and the deepest feature in the multi-scale feature extracted by the encoder module are input into the ASPP module to extract rich multi-scale context information and obtain the output feature of the decoder module. The coarse-grained category information guidance module is used to generate a dynamic prototype from the output features of the decoder module, and then further compress the dynamic prototype to obtain a class embedding vector. Finally, the class embedding vector is used to weight the output of the decoder module to enhance the discriminability between different classes and obtain the enhanced features.

2. The method for fine remote sensing land cover classification guided by coarse-grained category information according to claim 1, characterized in that: In step 1, the remote sensing images in the SAR remote sensing image land cover classification dataset are obtained from the SAR remote sensing image portion of the WHU-OPT-SAR dataset.

3. The method for fine-grained remote sensing land cover classification guided by coarse-grained category information according to claim 1, characterized in that: In step 1, the SAR remote sensing image ground feature classification dataset is randomly divided into a training set and a validation set in an 8:2 ratio.

4. The method for fine remote sensing land cover classification guided by coarse-grained category information according to claim 1, characterized in that: In step 1, the specific method for preprocessing the training set is as follows: First, the remote sensing images in the training set, i.e. the original images, with a size of 5556×3704, are cut into 1389×926 slices in a 4×4 non-overlapping pattern. These slices are then randomly scaled within the range of [0.5, 2.0]. Next, the scaled slices are cut into square images with a size of 512×512. Finally, the resulting square images are randomly flipped horizontally or vertically to obtain the preprocessed training set.

5. The method for fine remote sensing land cover classification guided by coarse-grained category information according to claim 1, characterized in that: The specific method for generating the dynamic prototype is as follows: First, the output features of the decoder module are mapped to a category feature space to generate a class-related feature representation; then, this class-related feature representation is multiplied by the output features of the decoder module to generate a dynamic prototype that can represent the spatial distribution of different categories in the image, and can respond to changes in the content of the input image. The dynamic prototype is represented as follows: Where X represents the output feature of the decoder module. This represents a 1×1 convolution. It is a matrix multiplication operation. This represents the generated dynamic prototype.

6. The method for fine remote sensing land cover classification guided by coarse-grained category information according to claim 5, characterized in that: The compressed dynamic prototype is: in, This represents the first fully connected layer. This represents the compressed dynamic prototype.

7. The method for fine remote sensing land cover classification guided by coarse-grained category information according to claim 6, characterized in that: The enhanced features are: Where LN is layer normalization and ReLU is the activation function. This indicates the second fully connected layer. This represents the third fully connected layer, which contains the obtained context information. This represents the Hadamard product with broadcast mechanism. This represents the enhanced features.

8. The method for fine remote sensing land cover classification guided by coarse-grained category information according to claim 1, characterized in that: In step 4, training is performed using the cross-entropy function. The specific training steps include: The SAR remote sensing images and their corresponding image segmentation labels in the preprocessed training set are processed by a remote sensing image land cover classification network to generate a dense prediction mask for each remote sensing image. The dense prediction mask and the corresponding image segmentation label are input into the cross-entropy loss function to calculate the loss value; The specific formula for the cross-entropy function is: in, N C is the total number of pixels in the SAR remote sensing image, and C is the number of categories. It is a pixel The true label, It is a pixel The predicted probability of belonging to category c.

9. The method for fine remote sensing land cover classification guided by coarse-grained category information according to claim 1, characterized in that: In step 5, the specific method for obtaining the remote sensing land cover classification results is as follows: The remote sensing image to be segmented is input into the trained remote sensing image land cover classification network model to obtain the segmentation results. The overall accuracy, average accuracy and average intersection-over-union ratio of the remote sensing image segmentation results are calculated and used as the evaluation results of the remote sensing image land cover classification network for the segmentation of remote sensing images.

Images