A remote sensing image small sample semantic segmentation method based on fusion of double models

Abstract

The application discloses a kind of remote sensing image small sample semantic segmentation methods of fusion double model, the method is suitable for providing only a small amount of pixel level annotation support sample, need to carry out segmentation to the remote sensing small sample segmentation scene of unobserved class in query remote sensing image, the application is generated from data by using latent diffusion model and visual language model, using self-supervised contrast pre-training carries out main training, realizes cross-modal semantic enhancement using visual language model combined with Softmax-GradCAM, realizes discriminative prototype alignment four levels based on discriminative prototype guide's semantic alignment module Construction remote sensing small sample semantic segmentation framework;So that the application can effectively improve the segmentation precision of unobserved class of remote sensing image, foreground positioning ability and cross-scene generalization performance under the condition of very few labeled samples.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent interpretation of remote sensing images, computer vision, few-shot learning and semantic segmentation, and specifically relates to a few-shot semantic segmentation method for remote sensing images that integrates two models. Background Technology

[0002] Remote sensing images are a crucial data source for acquiring information about the Earth's surface, enabling continuous observation of large areas. They have wide-ranging applications in fields such as land resource surveys, urban planning, ecological assessment, disaster monitoring, and land use management. Semantic segmentation of remote sensing images aims to classify different land cover categories, such as roads, buildings, vehicles, aircraft, ships, vegetation, and water bodies, at the pixel level. It is one of the core tasks in intelligent interpretation of remote sensing images. Through semantic segmentation technology, the fine spatial distribution and boundary structure of land cover targets can be obtained, providing fundamental data support for subsequent change detection, target statistics, scene understanding, and decision support.

[0003] In recent years, deep learning-based semantic segmentation methods have made significant progress in natural and remote sensing image processing. Convolutional neural networks, encoder-decoder structures, Transformers, and multi-scale feature fusion networks can automatically learn hierarchical feature representations in images, exhibiting stronger expressive power compared to traditional classification methods that rely on manually generated features. However, these methods typically require a large amount of high-quality pixel-level labeled data for training. For remote sensing images, pixel-level labeling not only requires specialized domain knowledge but is also time-consuming and labor-intensive, especially in high-resolution scenes, new regions, new sensors, or new category tasks, where the available labeled samples are often extremely limited.

[0004] Few-shot semantic segmentation offers a feasible approach to alleviate the aforementioned annotation bottleneck. This task typically employs a support-query paradigm, where only a small number of labeled support samples are provided during the inference phase, requiring the model to perform pixel-level predictions of the target category in the query image. Existing few-shot segmentation methods largely rely on pre-trained natural image backbones to extract features from support and query images, and establish the correspondence between support categories and query pixels through prototype matching, metric learning, or meta-learning strategies. While these methods have achieved some success in ordinary natural image scenarios, they still face numerous challenges when directly applied to remote sensing images.

[0005] First, there are significant domain differences between natural images and remote sensing images. Natural images are mostly viewed from eye level, with the subject typically occupying a large area of ​​the image. In contrast, remote sensing images are mostly viewed from above or orthographically, with varying target scales, a high proportion of background, and strong texture repetition. They are also significantly affected by imaging height, sensor type, spatial resolution, and geographical differences. Backbone networks pre-trained for natural image classification tasks tend to favor salient subjects and local textures in natural images, failing to adequately represent small-scale targets, dense targets, and complex spatial structures in remote sensing images. This leads to cross-domain representation mismatch in remote sensing scenes using small-sample segmentation models.

[0006] Secondly, targets of the same category in remote sensing scenes exhibit significant intra-class differences. The same ground features can appear completely different due to variations in imaging height, shooting angle, seasonal changes, regional differences, and target size. For example, targets belonging to the same aircraft category may show significant variations in scale, orientation, fuselage shape, and parking background; targets belonging to the same vehicle or ship category may also have differentiated textures due to differences in resolution and scene background. Under small sample conditions, a limited number of support samples are insufficient to cover the diverse appearance patterns of a category, leading to instability in the category prototypes obtained from the support samples, thereby weakening the reliability of support-query matching.

[0007] Secondly, remote sensing images often feature complex backgrounds where foreground targets are easily confused with similar backgrounds. High-resolution remote sensing images cover a wide area, and the background often includes roads, buildings, grasslands, water bodies, port facilities, and other land cover categories. Different categories may be highly similar in local texture and geometry. Since foreground pixels account for a small proportion of the image, the foreground supervision signal in the limited support samples is easily diluted by background information. When performing support-query feature matching, the model may incorrectly activate background regions that are similar in appearance to the target, resulting in incomplete segmentation boundaries, missed target classification, or misclassification of the background.

[0008] Therefore, existing technologies still have the following shortcomings in semantic segmentation of remote sensing images with small samples: First, there is a lack of high-quality unlabeled data generation and main intervention training mechanisms for the remote sensing domain, making it difficult to systematically alleviate the domain differences between the pre-trained backbone of natural images and remote sensing scenes; Second, there is a lack of semantic enhancement methods that can make full use of category text prompts and establish fine-grained text-visual associations, resulting in insufficient category representation of a small number of supporting samples; Third, there is a lack of prototype alignment mechanisms that can simultaneously consider supporting foreground integrity, foreground / background distinguishability, and query region discriminability, leading to easy false activation in complex backgrounds.

[0009] Therefore, it is necessary to propose a remote sensing image few-sample semantic segmentation method that integrates two models. Summary of the Invention

[0010] The purpose of this invention is to provide a remote sensing image small-sample semantic segmentation method that integrates two models to solve the technical problems mentioned in the background art.

[0011] To achieve the above objectives, the present invention provides the following technical solution: a remote sensing image small-sample semantic segmentation method integrating dual models, applicable to remote sensing small-sample segmentation scenarios where only a small number of support samples with pixel-level annotations are provided, and segmentation of unseen categories in the query remote sensing image is required, characterized by comprising the following steps: Step 1: Construct a few-sample segmentation task input consisting of support images, support masks, query images, and the names of the categories to be segmented. At the same time, convert the category names into category text prompts and convert the support masks or category layout maps into multi-channel semantic masks to provide category and spatial conditions for subsequent controllable data generation and cross-modal semantic guidance. Step 2: Using a latent diffusion model as a remote sensing image generator, the category text prompts and multi-channel semantic masks are injected into the diffusion denoising network as joint conditions to generate remote sensing images in the latent space that are semantically consistent with the remote sensing scene, have reasonable target layouts, and diverse appearances. The generated images are then semantically consistent using a visual language model to obtain high-quality unlabeled remote sensing pre-training samples. Step 3: Based on the high-quality unlabeled remote sensing pre-training samples, construct a self-supervised domain adaptation pre-training dataset. Use the momentum contrastive learning mechanism to perform unlabeled pre-training on the semantic segmentation backbone network, so that the backbone network learns the texture structure, target scale and top-down imaging features of the remote sensing images, and obtains the domain adaptation pre-training backbone. Step 4: Input the support image and query image into the domain adaptation pre-training backbone, extract multi-level support features and query features respectively, and use a visual language model to extract the text embeddings corresponding to the categories to be segmented and the visual features of the query image to construct a cross-modal semantic representation for the query image. Step 5: Model the fine-grained association between the category text embedding and the visual features of the query image as the optimal transport plan, solve the matching relationship between the text semantics and the visual region under global constraints, and input the aligned cross-modal response into Softmax-GradCAM to generate the pixel-level category semantic prior of the query image.

[0012] Step 6: Perform preliminary foreground prediction on the support features. Based on the coverage relationship between the support mask and the preliminary prediction results, encode the covered foreground regions and the insufficiently covered foreground regions into main support vectors and auxiliary support vectors, respectively, and aggregate the two to obtain enhanced support features containing complete foreground information. Step 7: Perform masked average pooling based on the enhanced support features and support mask to obtain the support foreground prototype and support background prototype respectively. Activate the query features using the support foreground prototype and support background prototype to obtain the query foreground prior mask and query background prior mask. Perform a difference operation on the two to eliminate the fuzzy region that simultaneously approximates the foreground and background, and generate a discriminative query foreground region. Step 8: Extract the query foreground prototype based on the discriminative query foreground region, and use the query foreground prototype, supporting foreground prototype and pixel-level category semantic prior together for query feature activation and segmentation prediction. Pixel-level segmentation results of the target category in the query image are obtained through end-to-end training.

[0013] According to the above technical solution, the specific method of step one is as follows: The category text prompt is obtained by combining the name of the category to be segmented, the description of the remote sensing imaging scene, and the description of the target appearance. The multi-channel semantic mask is used to represent the spatial layout of different land cover categories or target areas. The mask channels correspond one-to-one with the category semantics, which can provide local spatial structure constraints for the diffusion model without introducing additional manual annotation costs. The input to the few-shot segmentation task follows a scenario-based training paradigm, where each scenario contains at least one supporting sample and its pixel-level annotation, as well as at least one query sample to be predicted.

[0014] Based on the above technical solution, the specific method for step two is as follows: A) The latent diffusion model first maps remote sensing images or noise samples to a low-dimensional latent space using an image encoder; B) In the denoising U-Net, both textual conditions and semantic mask conditions are introduced. The textual conditions are used to constrain the global category semantics and scene attributes of the generated image, while the semantic mask conditions are used to constrain the spatial location, shape contour and category layout of the target. Both participate in feature modulation at multiple denoising scales, so that the generated image satisfies both semantic consistency and spatial structure consistency.

[0015] Based on the above technical solution, the specific method for step two is as follows: When the visual language model performs semantic consistency screening on the generated images, it extracts the visual embedding of the generated images and the text embedding of the corresponding category text prompts, calculates the similarity between the two, and retains the generated samples with similarity higher than a preset threshold. This screening process removes low-quality images that exhibit class drift, semantic inconsistency, or layout abnormalities, thereby improving the reliability of subsequent self-supervised domain adaptation pre-training data.

[0016] Based on the above technical solution, the specific method for step three is as follows: The self-supervised domain adaptation pre-training employs a momentum contrastive learning structure, including an online encoder, a momentum encoder, a projection head, and a dynamic negative sample queue. For two random augmented views of the same remote sensing image, the online encoder outputs a query representation and the momentum encoder outputs a key representation. The feature distance between augmented views from the same source is brought closer and the feature distance between different image samples is pushed further apart by the contrast loss. The momentum encoder parameters are updated by the online encoder parameters in an exponential momentum manner, and the dynamic negative sample queue is used to provide a stable and rich set of negative samples to avoid relying on ultra-large batch training.

[0017] Based on the above technical solution, the specific method for step four is as follows: The visual language model keeps its parameters frozen during the few-sample segmentation training phase and is used only as a category semantic prior extractor. The optimal transmission plan is based on the matching cost between text embedding and visual region features. Under global constraints, it calculates the cross-modal association matrix, enabling the categorical text semantics to be assigned to different visual regions of the query image in a fine-grained manner.

[0018] Based on the above technical solution, the specific method for step five is as follows: The Softmax-GradCAM performs gradient weighting on the query image features based on the aligned category response to obtain a pixel-level semantic prior map related to the category to be segmented.

[0019] Based on the above technical solution, the specific method for step six is ​​as follows: The main support vector is obtained by aggregating the reliable foreground region that overlaps with the preliminary prediction result and the support mask, and is used to characterize the target region that has been stably identified by the model. The auxiliary support vectors are obtained by aggregating the foreground regions in the support mask that are not fully covered by the initial prediction, and are used to supplement the target structures that are easily missed under small sample conditions; By jointly aggregating the main support vectors and auxiliary support vectors, a more complete representation of the support foreground features can be obtained than by single mask average pooling.

[0020] According to the above technical solution, the specific method of step seven is as follows: The supporting foreground prototype and supporting background prototype are obtained by performing masked average pooling on the supporting foreground region and supporting background region, respectively, by enhancing the supporting features: The two types of prototypes are then compared with the query features to obtain the query foreground prior mask and the query background prior mask. By subtracting the query background prior mask from the query foreground prior mask, the blurred areas that are highly similar to both the foreground and the background are suppressed, and the query foreground area that is more class discriminative is retained.

[0021] According to the above technical solution, the specific method of step eight is as follows: The query foreground prototype is obtained by weighted aggregation of query features in the discriminative query foreground region, and together with the supporting foreground prototype, it constitutes a bidirectional prototype guide. The pixel-level category semantic prior is used as an auxiliary attention weight to participate in query feature enhancement, enabling the segmentation network to accurately activate the target region and suppress similar background interference in remote sensing scenarios with very few supporting samples, significant changes in the scale and appearance of similar targets, and complex background regions. The overall training objective includes pixel-level segmentation loss and prototype matching constraint loss, which are weighted and summed before end-to-end optimization.

[0022] Compared with the prior art, the beneficial effects of the present invention are: 1) This invention introduces the conditional latent diffusion model into the data augmentation and domain adaptation pre-training process of remote sensing small sample semantic segmentation. It provides global category semantics through text prompts and provides local spatial layout constraints through multi-channel semantic masks, which can generate remote sensing images with consistent semantics, reasonable structure and diverse appearance. Combined with the visual language model screening mechanism, it can reduce the risk of semantic drift in generated samples and provide a reliable data source for unlabeled self-supervised pre-training.

[0023] 2) This invention utilizes momentum contrastive learning to perform self-supervised pre-training of the segmentation backbone network in the remote sensing domain, enabling the backbone network to learn the top-down texture, target scale distribution, and complex background structure of remote sensing images without additional manual annotation. This effectively alleviates the cross-domain representation bias when the pre-trained backbone of natural images is directly transferred to remote sensing scenes, and improves the feature representation ability in subsequent small sample segmentation tasks.

[0024] 3) This invention introduces category text prompts through a visual language model, and uses the text semantics of the category to be segmented as an external prior that is independent of the appearance changes of a single sample; further, it models text-visual alignment as the optimal transmission plan, establishes a fine-grained association between text embedding and local visual regions of the query image under global constraints, and combines Softmax-GradCAM to generate pixel-level semantic priors, thereby enhancing the semantic response of the target region in the query image.

[0025] 4) This invention designs a discriminative prototype-guided semantic alignment mechanism. It obtains a more complete support foreground representation by aggregating primary and secondary support vectors, and activates query features separately using support foreground / background prototypes. It then uses the difference between foreground and background priors to eliminate ambiguous regions, thereby generating a discriminative query foreground prototype. This mechanism effectively suppresses interference from complex backgrounds, improves the stability of support-query matching and the accuracy of target boundary segmentation, and is suitable for small-sample segmentation tasks of various remote sensing objects such as aircraft, vehicles, ships, buildings, and sports fields. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall method framework structure of the present invention; Figure 2 This is a schematic diagram of remote sensing image generation based on joint constraints of text prompts and semantic masks according to the present invention. Figure 3 This is a schematic diagram of the self-supervised domain adaptation pre-training process based on contrastive learning of the present invention; Figure 4 This is a schematic diagram of the cross-modal semantic enhancement module based on visual language prompts of the present invention; Figure 5 This is a schematic diagram of the semantic alignment module based on discriminative prototype guidance of the present invention; Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Example Please see Figure 1-5 This invention provides a technical solution: 1. A remote sensing image small sample semantic segmentation method integrating dual models, applicable to remote sensing small sample segmentation scenarios where only a small number of support samples with pixel-level annotations are provided, and segmentation of unseen categories in the query remote sensing image is required, characterized by comprising the following steps: Step 1: Construct a few-sample segmentation task input consisting of support images, support masks, query images, and the names of the categories to be segmented. At the same time, convert the category names into category text prompts and convert the support masks or category layout maps into multi-channel semantic masks to provide category and spatial conditions for subsequent controllable data generation and cross-modal semantic guidance. Step 2: Using a latent diffusion model as a remote sensing image generator, the category text prompts and multi-channel semantic masks are injected into the diffusion denoising network as joint conditions to generate remote sensing images in the latent space that are semantically consistent with the remote sensing scene, have reasonable target layouts, and diverse appearances. The generated images are then semantically consistent using a visual language model to obtain high-quality unlabeled remote sensing pre-training samples. Step 3: Based on the high-quality unlabeled remote sensing pre-training samples, construct a self-supervised domain adaptation pre-training dataset. Use the momentum contrastive learning mechanism to perform unlabeled pre-training on the semantic segmentation backbone network, so that the backbone network learns the texture structure, target scale and top-down imaging features of the remote sensing images, and obtains the domain adaptation pre-training backbone. Step 4: Input the support image and query image into the domain adaptation pre-training backbone, extract multi-level support features and query features respectively, and use a visual language model to extract the text embeddings corresponding to the categories to be segmented and the visual features of the query image to construct a cross-modal semantic representation for the query image. Step 5: Model the fine-grained association between the category text embedding and the visual features of the query image as the optimal transport plan, solve the matching relationship between the text semantics and the visual region under global constraints, and input the aligned cross-modal response into Softmax-GradCAM to generate the pixel-level category semantic prior of the query image.

[0029] Step 6: Perform preliminary foreground prediction on the support features. Based on the coverage relationship between the support mask and the preliminary prediction results, encode the covered foreground regions and the insufficiently covered foreground regions into main support vectors and auxiliary support vectors, respectively, and aggregate the two to obtain enhanced support features containing complete foreground information. Step 7: Perform masked average pooling based on the enhanced support features and support mask to obtain the support foreground prototype and support background prototype respectively. Activate the query features using the support foreground prototype and support background prototype to obtain the query foreground prior mask and query background prior mask. Perform a difference operation on the two to eliminate the fuzzy region that simultaneously approximates the foreground and background, and generate a discriminative query foreground region. Step 8: Extract the query foreground prototype based on the discriminative query foreground region, and use the query foreground prototype, supporting foreground prototype and pixel-level category semantic prior together for query feature activation and segmentation prediction. Pixel-level segmentation results of the target category in the query image are obtained through end-to-end training.

[0030] Specifically, the method for step one is as follows: The category text prompt is obtained by combining the name of the category to be segmented, the description of the remote sensing imaging scene, and the description of the target appearance. The multi-channel semantic mask is used to represent the spatial layout of different land cover categories or target areas. The mask channels correspond one-to-one with the category semantics, which can provide local spatial structure constraints for the diffusion model without introducing additional manual annotation costs. The input to the few-shot segmentation task follows a scenario-based training paradigm, where each scenario contains at least one supporting sample and its pixel-level annotation, as well as at least one query sample to be predicted.

[0031] Specifically, the method for step two is as follows: A) The latent diffusion model first maps remote sensing images or noise samples to a low-dimensional latent space using an image encoder; B) In the denoising U-Net, both textual conditions and semantic mask conditions are introduced. The textual conditions are used to constrain the global category semantics and scene attributes of the generated image, while the semantic mask conditions are used to constrain the spatial location, shape contour and category layout of the target. Both participate in feature modulation at multiple denoising scales, so that the generated image satisfies both semantic consistency and spatial structure consistency.

[0032] Specifically, the method for step two is as follows: When the visual language model performs semantic consistency screening on the generated images, it extracts the visual embedding of the generated images and the text embedding of the corresponding category text prompts, calculates the similarity between the two, and retains the generated samples with similarity higher than a preset threshold. This screening process removes low-quality images that exhibit class drift, semantic inconsistency, or layout abnormalities, thereby improving the reliability of subsequent self-supervised domain adaptation pre-training data.

[0033] Specifically, the method for step three is as follows: The self-supervised domain adaptation pre-training employs a momentum contrastive learning structure, including an online encoder, a momentum encoder, a projection head, and a dynamic negative sample queue. For two random augmented views of the same remote sensing image, the online encoder outputs a query representation and the momentum encoder outputs a key representation. The feature distance between augmented views from the same source is brought closer and the feature distance between different image samples is pushed further apart by the contrast loss. The momentum encoder parameters are updated by the online encoder parameters in an exponential momentum manner, and the dynamic negative sample queue is used to provide a stable and rich set of negative samples to avoid relying on ultra-large batch training.

[0034] Specifically, the method for step four is as follows: The visual language model keeps its parameters frozen during the few-sample segmentation training phase and is used only as a category semantic prior extractor. The optimal transmission plan is based on the matching cost between text embedding and visual region features. Under global constraints, it calculates the cross-modal association matrix, enabling the categorical text semantics to be assigned to different visual regions of the query image in a fine-grained manner.

[0035] Specifically, the method for step five is as follows: The Softmax-GradCAM performs gradient weighting on the query image features based on the aligned category response to obtain a pixel-level semantic prior map related to the category to be segmented.

[0036] Specifically, the method for step six is ​​as follows: The main support vector is obtained by aggregating the reliable foreground region that overlaps with the preliminary prediction result and the support mask, and is used to characterize the target region that has been stably identified by the model. The auxiliary support vectors are obtained by aggregating the foreground regions in the support mask that are not fully covered by the initial prediction, and are used to supplement the target structures that are easily missed under small sample conditions; By jointly aggregating the main support vectors and auxiliary support vectors, a more complete representation of the support foreground features can be obtained than by single mask average pooling.

[0037] Specifically, the method for step seven is as follows: The supporting foreground prototype and supporting background prototype are obtained by performing masked average pooling on the supporting foreground region and supporting background region, respectively, by enhancing the supporting features: The two types of prototypes are then compared with the query features to obtain the query foreground prior mask and the query background prior mask. By subtracting the query background prior mask from the query foreground prior mask, the blurred areas that are highly similar to both the foreground and the background are suppressed, and the query foreground area that is more class discriminative is retained.

[0038] Specifically, the method for step eight is as follows: The query foreground prototype is obtained by weighted aggregation of query features in the discriminative query foreground region, and together with the supporting foreground prototype, it constitutes a bidirectional prototype guide. The pixel-level category semantic prior is used as an auxiliary attention weight to participate in query feature enhancement, enabling the segmentation network to accurately activate the target region and suppress similar background interference in remote sensing scenarios with very few supporting samples, significant changes in the scale and appearance of similar targets, and complex background regions. The overall training objective includes pixel-level segmentation loss and prototype matching constraint loss, which are weighted and summed before end-to-end optimization.

[0039] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A remote sensing image small-sample semantic segmentation method integrating dual models, suitable for remote sensing small-sample segmentation scenarios where only a small number of support samples with pixel-level annotations are provided, and segmentation of unseen categories in the query remote sensing image is required, characterized in that, Includes the following steps: Step 1: Construct a few-sample segmentation task input consisting of support images, support masks, query images, and the names of the categories to be segmented. At the same time, convert the category names into category text prompts and convert the support masks or category layout maps into multi-channel semantic masks to provide category and spatial conditions for subsequent controllable data generation and cross-modal semantic guidance. Step 2: Using a latent diffusion model as a remote sensing image generator, the category text prompts and multi-channel semantic masks are injected into the diffusion denoising network as joint conditions to generate remote sensing images in the latent space that are semantically consistent with the remote sensing scene, have reasonable target layouts, and diverse appearances. The generated images are then semantically consistent using a visual language model to obtain high-quality unlabeled remote sensing pre-training samples. Step 3: Based on the high-quality unlabeled remote sensing pre-training samples, construct a self-supervised domain adaptation pre-training dataset. Use the momentum contrastive learning mechanism to perform unlabeled pre-training on the semantic segmentation backbone network, so that the backbone network learns the texture structure, target scale and top-down imaging features of the remote sensing images, and obtains the domain adaptation pre-training backbone. Step 4: Input the support image and query image into the domain adaptation pre-training backbone, extract multi-level support features and query features respectively, and use a visual language model to extract the text embeddings corresponding to the categories to be segmented and the visual features of the query image to construct a cross-modal semantic representation for the query image. Step 5: Model the fine-grained association between the category text embedding and the visual features of the query image as the optimal transport plan, solve the matching relationship between the text semantics and the visual region under global constraints, and input the aligned cross-modal response into Softmax-GradCAM to generate the pixel-level category semantic prior of the query image; Step 6: Perform preliminary foreground prediction on the support features. Based on the coverage relationship between the support mask and the preliminary prediction results, encode the covered foreground regions and the insufficiently covered foreground regions into main support vectors and auxiliary support vectors, respectively, and aggregate the two to obtain enhanced support features containing complete foreground information. Step 7: Perform masked average pooling based on the enhanced support features and support mask to obtain the support foreground prototype and support background prototype respectively. Activate the query features using the support foreground prototype and support background prototype to obtain the query foreground prior mask and query background prior mask. Perform a difference operation on the two to eliminate the fuzzy region that simultaneously approximates the foreground and background, and generate a discriminative query foreground region. Step 8: Extract the query foreground prototype based on the discriminative query foreground region, and use the query foreground prototype, supporting foreground prototype and pixel-level category semantic prior together for query feature activation and segmentation prediction. Pixel-level segmentation results of the target category in the query image are obtained through end-to-end training.

2. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step one is as follows: The category text prompt is obtained by combining the name of the category to be segmented, the description of the remote sensing imaging scene, and the description of the target appearance. The multi-channel semantic mask is used to represent the spatial layout of different land cover categories or target areas. The mask channels correspond one-to-one with the category semantics, which can provide local spatial structure constraints for the diffusion model without introducing additional manual annotation costs. The input to the few-shot segmentation task follows a scenario-based training paradigm, where each scenario contains at least one supporting sample and its pixel-level annotation, as well as at least one query sample to be predicted.

3. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step two is as follows: A) The latent diffusion model first maps remote sensing images or noise samples to a low-dimensional latent space using an image encoder; B) In the denoising U-Net, both textual conditions and semantic mask conditions are introduced. The textual conditions are used to constrain the global category semantics and scene attributes of the generated image, while the semantic mask conditions are used to constrain the spatial location, shape contour and category layout of the target. Both participate in feature modulation at multiple denoising scales, so that the generated image satisfies both semantic consistency and spatial structure consistency.

4. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step two is as follows: When the visual language model performs semantic consistency screening on the generated images, it extracts the visual embedding of the generated images and the text embedding of the corresponding category text prompts, calculates the similarity between the two, and retains the generated samples with similarity higher than a preset threshold. This screening process removes low-quality images that exhibit class drift, semantic inconsistency, or layout abnormalities, thereby improving the reliability of subsequent self-supervised domain adaptation pre-training data.

5. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step three is as follows: The self-supervised domain adaptation pre-training employs a momentum contrastive learning structure, including an online encoder, a momentum encoder, a projection head, and a dynamic negative sample queue. For two random augmented views of the same remote sensing image, the online encoder outputs a query representation and the momentum encoder outputs a key representation. The feature distance between augmented views from the same source is brought closer and the feature distance between different image samples is pushed further apart by the contrast loss. The momentum encoder parameters are updated by the online encoder parameters in an exponential momentum manner, and the dynamic negative sample queue is used to provide a stable and rich set of negative samples to avoid relying on ultra-large batch training.

6. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step four is as follows: The visual language model keeps its parameters frozen during the few-sample segmentation training phase and is used only as a category semantic prior extractor. The optimal transmission plan is based on the matching cost between text embedding and visual region features. Under global constraints, it calculates the cross-modal association matrix, enabling the categorical text semantics to be assigned to different visual regions of the query image in a fine-grained manner.

7. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step five is as follows: The Softmax-GradCAM performs gradient weighting on the query image features based on the aligned category response to obtain a pixel-level semantic prior map related to the category to be segmented.

8. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step six is ​​as follows: The main support vector is obtained by aggregating the reliable foreground region that overlaps with the preliminary prediction result and the support mask, and is used to characterize the target region that has been stably identified by the model. The auxiliary support vectors are obtained by aggregating the foreground regions in the support mask that are not fully covered by the initial prediction, and are used to supplement the target structures that are easily missed under small sample conditions; By jointly aggregating the main support vectors and auxiliary support vectors, a more complete representation of the support foreground features can be obtained than by single mask average pooling.

9. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step seven is as follows: The supporting foreground prototype and supporting background prototype are obtained by performing masked average pooling on the supporting foreground region and supporting background region, respectively, by enhancing the supporting features: The two types of prototypes are then compared with the query features to obtain the query foreground prior mask and the query background prior mask. By subtracting the query background prior mask from the query foreground prior mask, the blurred areas that are highly similar to both the foreground and the background are suppressed, and the query foreground area that is more class discriminative is retained.

10. The remote sensing image few-sample semantic segmentation method fused with dual models according to claim 1, characterized in that: The specific method for step eight is as follows: The query foreground prototype is obtained by weighted aggregation of query features in the discriminative query foreground region, and together with the supporting foreground prototype, it constitutes a bidirectional prototype guide. The pixel-level category semantic prior is used as an auxiliary attention weight to participate in query feature enhancement, enabling the segmentation network to accurately activate the target region and suppress similar background interference in remote sensing scenarios with very few supporting samples, significant changes in the scale and appearance of similar targets, and complex background regions. The overall training objective includes pixel-level segmentation loss and prototype matching constraint loss, which are weighted and summed before end-to-end optimization.

Images