A small sample remote sensing target detection method based on text semantic prior contrast learning

Abstract

The application discloses a small sample remote sensing target detection method based on text semantic prior contrast learning, and belongs to the field of remote sensing image processing and target detection, and solves the problems of misrecognition of easily confused categories and poor detection performance caused by high inter-class similarity of remote sensing targets in existing methods. The method first designs a remote sensing target feature prompt word, combines a CLIP text encoder to construct a class-level semantic relationship matrix as a text semantic prior; then, a detection network is constructed based on a Faster R-CNN and basic training is completed; subsequently, contrast learning fine-tuning training is carried out in combination with the text semantic prior, and the inter-class feature repulsion force is adaptively adjusted through prior weighting of negative samples; and finally, inference is completed by inputting an image to be detected. The method effectively captures the semantic relationship of remote sensing target categories and enhances the feature discrimination capability, and has excellent detection performance under the settings of 3, 5-shot and other small sample settings, thereby providing an efficient solution for small sample remote sensing target detection.

Description

Technical Field

[0001] This invention relates to a few-sample remote sensing target detection method based on textual semantic prior contrastive learning, belonging to the field of remote sensing image processing and target detection technology. Background Technology

[0002] Remote sensing target detection aims to locate and identify specific targets in aerial and satellite imagery, playing a crucial role in numerous practical applications such as environmental monitoring, traffic management, and military reconnaissance. In recent years, deep learning-based remote sensing target detection methods have become mainstream. These methods involve inputting remote sensing images into a neural network for feature extraction, followed by classification and bounding box regression, ultimately outputting the target category and its corresponding location information.

[0003] However, the performance of deep learning models heavily relies on large-scale, high-quality labeled datasets, and obtaining sufficient labeled data is costly and time-consuming. To alleviate data scarcity and the labeling burden, few-shot object detection has emerged, using a small number of labeled samples to locate and classify new categories of objects. Existing few-shot object detection methods are mainly divided into two categories: meta-learning-based and transfer learning-based methods. Meta-learning methods learn task-independent knowledge through meta-training to quickly adapt to new tasks, but they suffer from problems such as complex training processes and sensitivity to the quality of supporting samples. Transfer learning methods, on the other hand, adopt a two-stage training paradigm: first training the detector on data-rich basic categories, and then fine-tuning it on data-scarce new categories. Due to its simplicity, training stability, and compatibility with standard detection methods, it has become the dominant paradigm in few-shot object detection research in recent years.

[0004] Despite the achievements of transfer learning methods, the high inter-class similarity of remote sensing targets remains a significant challenge, leading to misidentification of easily confused categories. Many remote sensing categories share highly similar geometric structures, spatial layouts, and surrounding environments, making accurate detection difficult even with current state-of-the-art methods. Existing methods for detecting remote sensing targets in small sample sizes largely rely on visual features and treat all inter-class relationships equally, making them ill-suited to handling severe confusion between semantically similar categories in small-sample scenarios.

[0005] Language models can encode rich semantic knowledge from large-scale text corpora, capturing fine-grained category semantics and inter-class relationships that are difficult to learn from limited visual samples alone. This provides valuable textual semantic priors for small-sample remote sensing target detection. How to effectively utilize the semantic priors provided by language models to alleviate the detection challenges caused by inter-class similarity and improve the detection performance of new categories in small-sample remote sensing target detection has become an urgent technical problem to be solved. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing small-sample remote sensing target detection methods, which suffer from misidentification of easily confused categories and poor detection performance due to high inter-class similarity. This invention provides a small-sample remote sensing target detection method based on textual semantic prior contrastive learning. This method constructs a textual semantic prior and combines it with contrastive learning to adaptively adjust the inter-class feature repulsion, expand the feature distance between easily confused categories, and enhance feature discrimination ability, thereby improving the detection performance of new categories of remote sensing targets in small-sample scenarios.

[0007] The technical solution of this invention is: a few-shot remote sensing target detection method based on textual semantic prior contrastive learning, the steps of which include: The first step is to construct remote sensing target feature cue words and extract textual semantic priors: Design remote sensing target feature cue words for remote sensing images. These cue words encode both satellite view priors and structural appearance information of the remote sensing target. The template is: "a high-satellite view of the category of the target showing the appearance of characteristic spatial structure", where "category of the target" is replaced with the specific remote sensing target category name.

[0008] A frozen CLIP text encoder is used to project the feature cue words of each remote sensing target category into a shared semantic embedding space to obtain class-level semantic embeddings. The specific calculation method is as follows:

[0009] in, Indicates the category of the i-th remote sensing target. Feature prompts indicating the corresponding category Indicates a frozen CLIP text encoder. This indicates a normalization operation. This represents the normalized semantic embedding of the i-th category.

[0010] Based on class-level semantic embedding, semantic similarity between categories is calculated. A class-level semantic relationship matrix is ​​constructed as a text semantic prior. Semantic similarity is calculated using the angle similarity formula, as follows:

[0011] in, Indicate category and The normalized angular similarity between pairs indicates a stronger semantic relevance; by calculating the similarity between all category pairs, a remote sensing class-level semantic relationship matrix is ​​obtained. , where N is the total number of target categories.

[0012] The second step is to construct a small-sample remote sensing target detection network and perform basic training: The network adopts a two-stage training paradigm, is built on Faster R-CNN, uses ResNet-50 as the backbone network and is pre-trained on the ImageNet dataset. The network includes a backbone network, a detection head and a text semantic contrast learning module.

[0013] In the basic training phase, the network is trained using a basic category dataset, which contains a large number of well-labeled basic category samples. Through basic training, the network learns visual feature representations with generalization ability and basic detection capabilities.

[0014] The third step involves fine-tuning the training by combining comparative learning with prior semantic knowledge of the text: A class-balanced few-shot fine-tuning dataset is constructed, where each class contains only K labeled samples (K=3,5,10,20). During the fine-tuning phase, the backbone network is frozen, and only the detection head and contrastive learning module are trained. The contrastive learning module is integrated into the RoI branch of the detection head, and the specific implementation process is as follows: Class prototype construction: The RoI features corresponding to the real target are projected into the contrast embedding space through a lightweight contrast head. A memory prototype library is maintained for each category to store representative feature embeddings. The class prototype is calculated as follows:

[0015] in, This represents the memory buffer for category c. This represents the feature embedding in the buffer. This represents the class prototype of category c.

[0016] Prior weighted negative sample repulsion: Based on textual semantic prior, the repulsion force between class features is adjusted, and the temperature-scaled cosine similarity between the feature embedding and the class prototype is calculated.

[0017] in, This represents the normalized feature embedding of the i-th RoI. Let represent the normalized prototype of the j-th category, and τ represent the temperature parameter used to control the sharpness of the similarity distribution.

[0018] A text semantic prior contrastive loss function is constructed, with the weights of negative samples determined by the class-level semantic relation matrix. The loss function is as follows:

[0019] Where N represents the number of RoIs in the mini-batch, This represents the true label of the i-th RoI. = The negative sample weights are determined by the true class. The semantic similarity with the negative category j is determined.

[0020] The network is trained to adapt to the detection of new object categories by weighting and summing the contrast loss with the classification loss and regression loss of the detection network as the total loss function.

[0021] Step 4, Remote sensing target detection inference: The remote sensing image to be detected is input into the trained network. Image features are extracted through the backbone network, the detection head generates region proposals and performs classification and bounding box regression, the contrastive learning module enhances the feature discrimination ability, and finally the target detection results are output through non-maximum suppression (confidence threshold 0.05, IoU threshold 0.5), including target category, location and confidence. Attached Figure Description

[0022] Figure 1 Network structure diagram of the method of this invention; Figure 2 A framework diagram for text semantic prior contrastive learning; Figure 3 Example test results; Figure 4 Example: Heat map of test results.

[0023] Beneficial effects 1. This invention designs target feature prompts for remote sensing images and constructs a class-level semantic relationship matrix as a text semantic prior by combining a frozen CLIP text encoder. This prior is independent of visual samples and can effectively capture the inherent semantic relationship between remote sensing target categories, providing key guidance for solving the misidentification problem caused by high inter-class similarity. 2. By combining textual semantic prior with contrastive learning, a prior weighted negative sample rejection mechanism is proposed. The mechanism adaptively adjusts the feature rejection force between classes based on the semantic similarity between classes. It applies a stronger rejection force to easily confused classes with similar semantics, thereby expanding their feature distance. At the same time, it maintains a moderate separation for classes with large semantic differences, which significantly enhances the feature discrimination ability. 3. A two-stage training paradigm is adopted. The basic training stage learns generalized visual features, while the fine-tuning stage combines textual semantic priors with comparative learning to adapt to new categories. The training process is stable and highly compatible, achieving excellent performance even under challenging few-sample settings such as 3-shot and 5-shot tests. Experimental verification shows that this method outperforms existing few-sample object detection methods and effectively reduces the false recognition rate between easily confused categories, providing an efficient and feasible technical solution for few-sample remote sensing object detection. Detailed Implementation

[0024] The present invention will be further described below with reference to the embodiments.

[0025] Example A few-sample remote sensing target detection method based on textual semantic prior contrastive learning is proposed. Taking a practical application scenario of few-sample remote sensing target detection as an example, the overall framework of this method is as follows: Figure 1 As shown, the steps of this method include: The first step is to construct remote sensing target feature cues and extract textual semantic priors: Based on the imaging characteristics of targets in remote sensing imagery, a target feature cue word template was designed as "a high-satellite view of the category of the target showing the appearance of characteristic spatial structure". For specific target categories (such as bridge, overcome, stadium, etc.), "category of the target" was replaced with the corresponding category name to generate feature cue words for that category. A pre-trained CLIP model (ViT-B / 32 version) was used with a text encoder. Feature cue words for all target categories were input into the encoder to obtain the original semantic embeddings, which were then normalized to obtain class-level semantic embeddings. The encoder parameters were kept frozen throughout the training process to avoid the influence of scarce small-sample visual data or noise.

[0026] Calculate the semantic similarity between all category pairs using the angular similarity formula. Construct a class-level semantic relationship matrix. This matrix is ​​a symmetric matrix with element values ​​between [0,1], which intuitively reflects the semantic relationships between categories and serves as a text semantic prior for subsequent comparative learning.

[0027] The second step is to construct a small-sample remote sensing target detection network and perform basic training: The network is based on Faster R-CNN architecture, with a backbone network using ResNet-50 combined with FPN (Feature Pyramid Network). The detection head includes classification and regression branches, and the contrastive learning module is integrated into RoIHead. The basic training phase uses basic category samples (15 categories) from the DIOR dataset, which is preprocessed to a uniform resolution. Through basic training, the network learns the visual feature representations of basic categories and object detection capabilities.

[0028] The third step involves fine-tuning the training by combining comparative learning with prior semantic knowledge of the text: A balanced few-shot fine-tuning dataset was constructed. For the 5 new classes and 15 basic classes in the DIOR dataset, 3, 5, 10, and 20 labeled samples were selected for each class as K-shot settings (K=3,5,10,20), forming a class-balanced fine-tuning dataset. During the fine-tuning phase, the parameters of the backbone network were frozen, and only the detection head and contrastive learning module were trained.

[0029] The specific implementation of the comparative learning module: Class Prototype Construction: The contrastive head consists of two fully connected layers and a ReLU activation function, projecting RoI features (1024 dimensions) into a 256-dimensional contrastive embedding space. A memory prototype library with a capacity of 100 is maintained for each class, storing the most recently updated feature embeddings. The class prototype is obtained by calculating the mean of the features in the library. ,in .

[0030] Prior weighted negative sample exclusion: The temperature parameter τ is set to 0.07, and the temperature-scaled cosine similarity between the RoI feature embedding and all class prototypes is calculated. The weights of negative samples are determined based on the class-level semantic relationship matrix. = Construct the text semantic prior contrast loss as , Total loss is

[0031] in, For classifying losses, To regress the loss, The text semantic prior contrast loss is used.

[0032] Step 4, Remote sensing target detection inference: The remote sensing image to be detected is input into the trained network, the weight file obtained from the training is used for inference, non-maximum suppression (NMS) is applied, the confidence threshold is set to 0.05, the IoU threshold is set to 0.5, and the final target detection result is output.

[0033] Taking remote sensing small sample target detection as an example, on the DIOR public dataset, a comparison of various existing remote sensing small sample detection methods is shown in Table 1. It can be seen that the method proposed in the embodiment has better detection performance.

[0034] Table 1. Performance Indicators of Various Existing Remote Sensing Small Sample Target Detection Methods and the Proposed Method

[0035] In summary, the above are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A few-sample remote sensing target detection method based on textual semantic prior contrastive learning, characterized in that, include: The first step is to construct remote sensing target feature prompts and extract text semantic priors: By designing targeted remote sensing target feature prompts, class-level semantic embeddings are extracted by combining the frozen CLIP text encoder, and a class-level semantic relationship matrix is ​​constructed based on angle similarity calculation to provide semantic guidance for subsequent comparative learning; The second step is to construct a small-sample remote sensing target detection network and perform basic training: by constructing a class prototype memory to store representative feature embeddings, and by adaptively adjusting the negative sample weights based on the text semantic relationship matrix, the dynamic adjustment of the inter-class feature repulsion force is achieved. The third step involves fine-tuning the training by combining contrastive learning with textual semantic priors: in the basic training phase, the network is trained on a large-scale basic category dataset to obtain generalized visual features; in the fine-tuning phase, the backbone network is frozen, and only the detection head and contrastive learning module are trained to adapt to new categories; in the inference phase, the final detection result is output through non-maximum suppression.

2. The few-sample remote sensing target detection method based on textual semantic prior contrastive learning as described in claim 1, characterized in that, in the first step... include: 1.1 Design a remote sensing target feature cue word template. The template is "a high-satellite view of the category of the target showing the appearance of characteristic spatial structure". Replace "category of the target" with the specific remote sensing target category name to generate corresponding category feature cue words. Encode the feature cue words using a frozen CLIP text encoder, and obtain the class-level semantic embedding after normalization. The calculation method is as follows: ； in, Indicates the category of the i-th remote sensing target. Feature prompts indicating the corresponding category Indicates a frozen CLIP text encoder. This indicates a normalization operation. Represents the normalized semantic embedding of the i-th category; 1.

2. Based on class-level semantic embedding, calculate the angular similarity between categories and construct a class-level semantic relationship matrix. The similarity calculation formula is as follows: ； in, Indicate category and The normalized angular similarity between pairs indicates a stronger semantic relevance; by calculating the similarity between all category pairs, a remote sensing class-level semantic relationship matrix is ​​obtained. , where N is the total number of target categories.

3. The few-sample remote sensing target detection method based on textual semantic prior contrastive learning as described in claim 1, characterized in that, The second step includes: 2.1 Project the RoI features corresponding to the real target onto the contrast embedding space through a lightweight contrast head. The contrast head consists of two fully connected layers and a ReLU activation function, which projects the RoI features onto the contrast embedding space of a preset dimension. 2.

2. Maintain a memory prototype library for each category to store representative feature embeddings. The class prototype is obtained by calculating the mean value of the features in the library, as shown in the following formula: ； in, This represents the memory buffer for category c. This represents the feature embedding in the buffer. Represents the class prototype of category c; 2.3 Calculate the temperature-scaled cosine similarity between the feature embedding and the class prototype, using the following formula: ； in, This represents the normalized feature embedding of the i-th RoI. Let represent the normalized prototype of the j-th category, and τ represent the temperature parameter used to control the sharpness of the similarity distribution; 2.4 Construct a text semantic prior contrast loss function. The weights of negative samples are determined by the class-level semantic relation matrix. The loss function is as follows: ； Where N represents the number of RoIs in the mini-batch, This represents the true label of the i-th RoI. = The negative sample weights are determined by the true class. The semantic similarity with negative category j is determined; 2.5 The total loss function is a weighted sum of the contrastive loss and the classification and regression losses of the detection network, as shown in the following formula: ； in, For classifying losses, To regress the loss, The text semantic prior contrast loss is used.

4. The few-sample remote sensing target detection method based on textual semantic prior contrastive learning as described in claim 1, characterized in that, The third step includes: 3.1 Basic Training Stage: Construct a few-sample remote sensing target detection network based on Faster R-CNN, using ResNet-50 as the backbone network and pre-trained on the ImageNet dataset. The network includes a backbone network, a detection head, and a text semantic contrast learning module. The network is trained using a basic category dataset, which contains a large number of well-labeled basic category samples. Through basic training, the network learns visual feature representations with generalization ability and basic detection capabilities. 3.2 Fine-tuning training phase: Construct a class-balanced few-shot fine-tuning dataset, with only K labeled samples for each class (K=3,5,10,20); freeze the backbone network parameters and train only the detection head and contrastive learning module to adapt the network to the detection of new class objects; 3.3 Inference Stage: The remote sensing image to be detected is input into the trained network. Image features are extracted through the backbone network, the detection head generates region proposals and performs classification and bounding box regression, the contrastive learning module enhances the feature discrimination ability, and finally the target detection results are output through non-maximum suppression, including target category, location and confidence.

Images