Small sample target detection method, system and equipment based on decoupling prototype and medium

Abstract

The invention relates to a small sample target detection method and device based on a decoupling prototype and a medium, belongs to the technical field of crossing of remote sensing image processing and computer vision, and can effectively separate interference features such as category semantics, angles and backgrounds of remote sensing targets. A double-branch prototype decoupling structure of'category semantic branch + angle geometric branch 'is constructed, and a decoupling loss constraint based on cosine similarity is introduced, so that a category prototype and an angle prototype are approximately orthogonal in a feature space; and then the RPN and the detection head are cooperatively guided through double prototypes, and a two-stage training strategy is combined, so that efficient migration of basic class knowledge to new class small samples is realized, and the small sample target detection performance and robustness in a complex remote sensing scene are improved.

Description

Technical Field

[0001] This invention relates to a small-sample target detection method, device, and medium based on a decoupled prototype, belonging to the interdisciplinary field of remote sensing image processing and computer vision. Background Technology

[0002] In target detection scenarios with limited sample remote sensing images, prototype learning is currently the mainstream technical approach. Its core idea is to extract aggregated representations (i.e., "category prototypes") of similar target features from a "support set" containing a small number of labeled samples, and use these category prototypes as semantic anchors to guide target localization and recognition in the "query set" (the image to be detected). For example, the existing Prototype-Guided CNN (P-CNN) generates category prototypes from the support set and integrates them into a Region Proposal Network (RPN) to filter candidate boxes containing targets, thus alleviating to some extent the performance degradation caused by insufficient sample data.

[0003] However, existing prototype-based target detection methods for small-sample remote sensing images still suffer from three major drawbacks, making it difficult to meet the requirements of practical engineering applications in terms of detection accuracy and robustness:

[0004] Entangled Prototype Representations: Category semantics and interfering features are mixed, leading to a sharp drop in generalization ability for small samples. Existing methods typically generate category prototypes that simultaneously carry both "category semantic features" (such as ship deck structure, aircraft wing proportions) and "interfering features" (such as target angular geometry, background texture). Since targets in remote sensing images can be tilted arbitrarily from 0° to 180° (such as ships sailing at 45°, aircraft flying sideways), if the angular information is not separated from the category prototype, the prototypes of targets of the same type but at different angles will have extremely high dispersion in the feature space—the prototype difference between a 0° ship and a 45° ship may exceed the prototype difference between a 0° ship and a 0° vehicle. In small sample scenarios such as 3-shot / 5-shot, the model cannot distinguish between "angular differences" and "category differences," leading to a large number of misclassifications or missed detections.

[0005] Feature decoupling is lacking: semantic and interfering features are not separated, and knowledge transfer in small samples fails. Small sample detection relies on the transfer of knowledge from the base class (large-scale samples) to the new class (small number of samples), but existing methods do not decouple the "category semantic features" from the "remote sensing scene interference features (angle, background)". Remote sensing images often have complex backgrounds (such as clouds, mountains, and vegetation) and low contrast between the target and the background, making the background features of the support set samples prone to contaminating the category prototype. For example, in a 3-shot scenario, if the support set samples for the new class "small bridge" contain a large amount of vegetation background, the generated prototype will be dominated by vegetation features, easily misclassifying mountain vegetation as a bridge during detection, resulting in a high false detection rate.

[0006] Poor Prototype-Guided Adaptability: Lacking optimization for remote sensing scenarios, candidate box selection accuracy is insufficient. Existing methods use a "single hybrid prototype" to guide RPN candidate box selection, which cannot adapt to the characteristics of remote sensing targets such as "arbitrary angles" and "background complexity." On the one hand, traditional horizontal anchor points have low matching accuracy with tilted targets (such as ships at 135°), significantly increasing the candidate box omission rate; on the other hand, background textures (such as water ripples and vegetation textures) are similar to the prototype, easily generating a large number of misselected candidate boxes, increasing the computational burden on the detection head and reducing detection accuracy.

[0007] In summary, existing prototype-based remote sensing image target detection methods with small sample sizes struggle to fully balance detection accuracy, robustness, and generalization ability when faced with typical remote sensing scenarios characterized by "targets at arbitrary angles + complex backgrounds + small sample sizes". Summary of the Invention

[0008] In view of this, the present invention provides a small sample target detection method, system, device and medium based on decoupled prototypes, which can effectively separate the category semantics of remote sensing targets from interference features such as angle and background, fundamentally alleviating the problem of "angle differences being misidentified as category differences and background features contaminating the category prototype under small sample conditions".

[0009] To achieve the above objectives, the technical solution of the present invention is as follows.

[0010] A few-sample target detection method based on a decoupled prototype includes the following steps:

[0011] Acquire remote sensing images containing targets, label the location, category, and binary mask of the targets, divide the images into basic classes and new classes, and divide the new classes into support sets and query sets according to a preset ratio;

[0012] The images of the base class and the new class are preprocessed, including pixel value normalization and mean standard deviation normalization, and the processed images and labels are converted into tensors compatible with deep learning frameworks.

[0013] Load the pre-trained shared backbone network, freeze some layer weights, and fine-tune the later layer weights to adapt to the remote sensing image features.

[0014] At the output of the shared backbone network, a category semantic branch and an angle geometry branch are constructed. The same type of target features of the support set are input into the two branches respectively: the category semantic branch generates category prototypes, and the angle geometry branch extracts angle features through multi-directional convolution and global pooling and aggregates them to generate angle prototypes.

[0015] Calculate the cosine similarity between the category prototype and the angle prototype, construct the decoupling loss, and minimize the decoupling loss to make the two types of prototypes approach orthogonality in the feature space;

[0016] Channel-level aggregation is performed on all category prototypes to obtain foreground category aggregate prototypes, and the complementary weights of the network classification branch are generated by inputting the output region of the fully connected layer. The feature map output by the backbone network is used to extract the angle features of the candidate boxes within the candidate box coverage area through the angle geometry branch. The similarity between the angle features of the candidate boxes and the angle prototypes is calculated to obtain the angle matching degree of the candidate boxes.

[0017] The original classification scores of the region generation network and the angle matching degree are weighted and fused according to preset weights to filter high-confidence candidate boxes;

[0018] The filtered candidate boxes are aligned with regions of interest to obtain fixed-dimensional region of interest features. These features are then processed at the channel level with the corresponding category prototype. During the training phase, a smooth loss function is used to regress the horizontal bounding box. The target category probability and bounding box coordinates are output through a fully connected layer.

[0019] Construct the total loss function, including detection loss and decoupling loss, calculate the loss between the predicted results and the actual results, and backpropagate to update the model parameters;

[0020] A two-stage training strategy is adopted: the first stage uses the support set and query set of the base class for training, and saves the optimal model weights and the prototype of the base class; the second stage uses a mixed dataset of a subset of the base class and the complete set of the new class for training, dynamically updates the prototype of the new class, and saves the optimal model weights of the new class.

[0021] The remote sensing image to be detected is preprocessed, input into the trained model, and the detection result is output.

[0022] The category semantic branch comprises a convolutional layer, a batch normalization layer, an activation function, and a global average pooling layer connected in sequence; the angular geometry branch comprises a convolutional layer, a multi-directional convolutional layer, a batch normalization layer, an activation function, and a global average pooling layer connected in sequence; the decoupling loss is obtained by calculating the absolute value of the cosine similarity between the prototype of the category and its corresponding angular prototype for each category, and by calculating the cosine similarity for each category, which is used to constrain the category prototype and the angular prototype to be approximately orthogonal in the feature space; the complementary weights of the region generation network classification branch are obtained by aggregating the foreground category prototype through a fully connected layer, and are used to adjust the scoring weights of the region generation network for the foreground target.

[0023] The candidate box angle matching degree is calculated by the cosine similarity between the candidate box angle features extracted from the query set feature map through the angle geometry branch and the angle prototype.

[0024] The channel-level operation involves element-wise multiplication of the region of interest features and the category prototype along the channel dimension to enhance category-related semantic features.

[0025] In the two-stage training strategy, the first stage of training uses basic class data, and the second stage of training uses mixed data of a subset of basic classes and the entire set of new classes, and dynamically updates the prototype of the new classes through a moving average strategy.

[0026] This invention also provides a small-sample target detection system based on a decoupled prototype, comprising:

[0027] The dual-branch prototype decoupling module is used to generate category prototypes and angle prototypes through category semantic branches and angle geometry branches, respectively.

[0028] The cosine loss constraint module is used to calculate the cosine similarity between the category prototype and the angle prototype, construct the decoupling loss, and make the two types of prototypes approach orthogonality in the feature space.

[0029] The prototype collaborative guidance module is used to aggregate prototypes and angle prototypes to collaboratively guide the candidate box filtering of the region generation network using foreground categories;

[0030] The detection head and total loss optimization module are used to classify candidate boxes and regress bounding boxes, and calculate the total loss.

[0031] The two-stage training module is used to execute a two-stage training strategy and optimize model parameters.

[0032] In the dual-branch prototype decoupling module, both the category semantic branch and the angle geometry branch are constructed based on the output of the shared backbone network. The category semantic branch focuses on extracting category semantic features, while the angle geometry branch focuses on extracting angle geometry features.

[0033] The prototype collaborative guidance module generates complementary weights for network classification branches by aggregating the prototype output region through foreground categories, and generates the original classification score of the network by weighting and fusing the region through angle matching degree.

[0034] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the present invention.

[0035] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the present invention.

[0036] Beneficial effects

[0037] 1. This invention constructs a dual-branch prototype decoupling structure of "category semantic branch + angle geometric branch" and introduces a decoupling loss constraint based on cosine similarity to make the category prototype and angle prototype approximately orthogonal in the feature space. Then, by using dual prototypes to collaboratively guide the RPN and the detection head, combined with a two-stage training strategy, it achieves efficient transfer of basic class knowledge to small samples of new classes, thereby improving the performance and robustness of small sample target detection in complex remote sensing scenarios.

[0038] 2. In this invention, decoupling prototypes enhances category generalization ability. Specifically, this is achieved through the structural separation of category semantic branches and angular geometric branches, and by introducing decoupling loss to constrain the orthogonality of category prototypes and angular prototypes. The cosine similarity of targets of the same class but different angles in the category prototype space can be maintained at a high level (e.g., ≥0.9), while the similarity between different classes is significantly lower. Thus, in extremely small sample scenarios such as 3-shot, the model can effectively avoid the problem of "angular differences being misjudged as category differences." Even if the target is in an arbitrary tilted posture, it can still achieve accurate identification through stable category prototypes, significantly reducing the false negative rate.

[0039] 3. In this invention, dual-prototype collaboration enhances detection robustness. Specifically, foreground category aggregation prototypes are used to enhance the RPN's response to foreground regions, and angle prototypes are introduced into the candidate box selection process to measure angle matching, resulting in a significant improvement in the recall rate of candidate boxes for tilted targets. Furthermore, combined with channel attention enhancement guided by category prototypes in the detection head and smooth L1 regression, high-quality detection results can be stably output even in complex backgrounds (such as clouds, water reflections, vegetation textures, etc.). In typical 10-shot scenarios on remote sensing datasets, the novel target detection mAP of this invention achieves a high level in the current field, demonstrating significant advantages over mainstream methods such as Meta R-CNN and TFA.

[0040] 4. This invention possesses strong scene adaptability and meets engineering requirements. By explicitly decoupling category semantics and angular geometric features, this invention can automatically learn the tilt direction representation of targets through the angular geometric branch in small sample scenarios without relying on additional manual angle annotation or explicit angle regression branches. It utilizes a pure category prototype to accurately identify targets with arbitrary tilt attitudes from 0° to 180°, fundamentally solving the problem of "angle differences being misjudged as category differences," and fully adapting to the highly flexible angular distribution of remote sensing targets. Simultaneously, the category prototype effectively filters background features during construction, retaining only semantic information highly correlated with the target category. Therefore, under complex background conditions such as cloud cover, mountainous terrain, and water reflection, it can effectively suppress interference from background textures, significantly reducing the false detection rate and demonstrating promising engineering application prospects. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention.

[0042] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0043] The present invention will be further described in detail below with reference to specific embodiments.

[0044] This invention provides a small-sample target detection method based on a decoupled prototype, the specific process of which is as follows: Figure 1 As shown, it includes the following steps:

[0045] Step 1: Collect remote sensing images containing targets, label the target location, category and binary mask, divide the images into basic class and new class, and then divide the new class into support set and query set according to preset ratio;

[0046] Step 2: Normalize the pixel values ​​and standard deviation of all base and new class images, convert the standardized images and corresponding labels into tensors compatible with deep learning frameworks, and create training data loader and validation data loader.

[0047] Step 3: Load the pre-trained backbone network on a general image dataset as a shared backbone network, freeze the weights of some layers in the early stage of the network, and only fine-tune the weights of the later layers to adapt to the features of the remote sensing images.

[0048] Step 4: Build a category semantic branch and an angle geometry branch at the output of the shared backbone network. Input the features of the same type of target in the support set into the two branches respectively: the category semantic branch takes the mean of the output feature vector to generate the category prototype; the angle geometry branch extracts the angle features through multi-directional convolution and global pooling, and aggregates the angle feature vectors of the same type of target to generate the angle prototype.

[0049] Step 5: For each category, calculate the cosine similarity between the prototype of that category and its corresponding angle prototype, and use the cosine similarity of each category as the decoupling loss to construct the decoupling loss. By minimizing the decoupling loss, the two prototypes are made to approach orthogonality in the feature space, thereby achieving the separation of category semantics and angle geometric features.

[0050] Step 6: Perform channel-level aggregation on all category prototypes to obtain the foreground category aggregated prototype, and input the complementary weights of the classification branch of the Region Generation Network (RPN) output by the fully connected layer; extract the candidate box angle features in the candidate box coverage area through the angle geometry branch of the feature map output by the backbone network for the query set, calculate the similarity between the candidate box angle features and the angle prototype, and obtain the angle matching degree of each candidate box.

[0051] Step 7: Combine the original RPN classification scores and angle matching scores with preset weights to filter high-confidence candidate boxes;

[0052] Step 8: Perform region of interest alignment on the filtered candidate boxes to obtain fixed-dimensional region of interest (RoI) features. Perform channel-level operations on the RoI features and the corresponding category prototypes. Use a smooth loss function to regress the horizontal bounding box. Build a fully connected layer to output the target category probability and bounding box coordinates.

[0053] Step 9: Construct the total loss function, which includes detection loss and decoupling loss. Calculate the loss between the predicted result and the actual result, backpropagate the loss, and update the model parameters.

[0054] Step 10: Adopt a two-stage training strategy: In the first stage, train using the support set and query set of the base class, and save the model weights with the best mean average precision (mAP) and the prototype of the base class; In the second stage, train using a mixed dataset of "base class subset + new class full set", dynamically update the prototype of the new class, and save the model weights with the best mAP of the new class.

[0055] Step 11: Preprocess the remote sensing image to be detected according to the process in Step 2. Input the trained model, perform feature extraction, candidate box selection, classification and bounding box regression, output and save the detection results.

[0056] In this embodiment, in step 1, an image annotation tool is used to annotate the target. In the binary mask, the pixel value of the target area is set to 1 and the pixel value of the background area is set to 0. Each basic class retains a sufficient number of annotated images to cover diverse imaging scenarios, and each new class retains a small number of annotated images to simulate small sample scenarios. The ratio of the new class support set to the query set is determined according to the needs of small sample experiments. The query set contains mixed images of the basic class and the new class.

[0057] Specifically, publicly available remote sensing datasets (DIOR, NWPU-VHR10) and real-world scene images were collected using satellite remote sensing platforms and airborne remote sensing equipment. Samples containing targets such as aircraft, ships, and bridges were selected, and image processing tools were used to uniformly adjust the image resolution and size to meet the model input requirements. The LabelMe annotation tool was used to complete target annotation: target bounding boxes were annotated to determine their location, category labels were added to distinguish target types (ensuring consistency with subsequent basic and new class classifications), and a binary mask of the target was generated (target area pixel value is 1, background area is 0), which is used to focus on target feature learning during the training process.

[0058] From the labeled general dataset, base classes and new classes are divided (15:5): a sufficient number of classes are selected as base classes, with each class retaining enough labeled images to cover diverse imaging angles and background environments; a small number of classes are selected as new classes, with each class retaining only a small number of labeled images to simulate small sample scenarios. The new classes are then divided into support and query sets according to a preset ratio—the support set is used for subsequent prototype generation, and the query set uses a mixture of base and new class images for model training and validation, as well as final detection performance evaluation, ensuring a comprehensive test of the model's adaptability to both base and new classes.

[0059] In step 2, all images of the base class and new class are iterated through, and a pixel value normalization tool is used to map the image pixel values ​​from the range of 0-255 to the interval [0,1] to avoid excessively large pixel values ​​interfering with model training. The mean and standard deviation of the pixels in the base class training set images are calculated, and all images are standardized based on this mean and standard deviation using the formula: Standardized pixel value = (Original pixel value - Mean) / Standard deviation, so that the image data meets the numerical distribution requirements for model training. The tensor transformation function of the PyTorch framework is used to convert the standardized images and their corresponding labels (bounding box coordinates, class, mask) into PyTorch tensors; a training data loader and a validation data loader are created, a reasonable batch size is set, the data random shuffling function is enabled, and multi-threaded data reading is enabled to improve the data loading efficiency during training.

[0060] In step 3, a ResNet101 network pre-trained on the ImageNet dataset is loaded into the deep learning framework as the shared backbone network of the model. This network has multi-level feature extraction capabilities and can efficiently capture the deep semantic and detailed features of remote sensing images. A "partial weight freezing + local fine-tuning" strategy is adopted: the weights of several layers in the early stage of the network are frozen to retain the general image feature extraction capability, and only the weights of the later layers are fine-tuned. This allows the backbone network to adapt to the unique features of remote sensing images (such as target scale differences and complex background textures) while avoiding overfitting and slow convergence problems caused by training from scratch.

[0061] The preprocessed target instance blocks of the support set are input into the backbone network. After layers of operations such as convolution, batch normalization, and activation, a fixed-dimensional deep feature map is output, providing a feature foundation for the subsequent generation of category prototypes and angle prototypes. The preprocessed complete image of the query set is input into the backbone network, and multi-scale feature maps are output from different network layers. This provides a suitable basic feature support for candidate box generation and region of interest (RoI) feature extraction of the Region Generation Network (RPN), ensuring that the feature input requirements of subsequent modules match the output characteristics of the backbone network.

[0062] In step 4, the category semantic branch consists of a convolutional layer, a batch normalization layer, an activation function, and a global pooling layer. The convolutional layer sets the number of input and output channels according to the channel characteristics of the backbone network output features. The global pooling layer compresses the multi-dimensional feature map into a fixed-dimensional feature vector. The angular geometry branch consists of a convolutional layer, a multi-directional convolutional layer, a batch normalization layer, an activation function, and a global pooling layer. The multi-directional convolutional layer responds only to the edge and contour features of the corresponding angle through multiple sets of directional convolutional kernels.

[0063] Specifically, an angular geometry branch is built at the output of the shared backbone network, consisting of a 1×1 convolutional layer, a multi-directional 3×3 convolutional layer, a batch normalization layer, a ReLU activation function, and a GAP layer. The 1×1 convolutional layer configures adaptation parameters based on the channel characteristics of the backbone network output features to achieve feature channel dimensionality reduction. The multi-directional 3×3 convolutional layer, through multiple sets of directional convolutional kernels, responds only to the edge and contour features corresponding to the angle in the image, accurately extracting the target angle geometry information. The batch normalization layer and the ReLU activation function respectively achieve feature normalization and introduce non-linear transformations, while the GAP layer compresses the multi-dimensional feature map into a fixed-dimensional feature vector. The features of target instance blocks of the same category within the support set are input into this branch. After processing by multi-directional 3×3 convolution, batch normalization, ReLU, and GAP, the angular feature vectors of each instance block in the discrete directional channels are obtained. The arithmetic mean of the angular feature vectors of all instance blocks under the same category is taken to obtain the angular prototype of that category. The response of this angular prototype in each directional channel characterizes the tilting posture distribution of the category over a set of discrete dominant directions, without requiring explicit angle labeling and minimizing the inclusion of category semantic features.

[0064] The architecture follows a logical sequence of "channel transformation – feature optimization – dimensionality compression," consisting of a 1×1 convolutional layer, a batch normalization layer, a ReLU activation function, and a global average pooling (GAP) layer. The 1×1 convolutional layer configures the appropriate number of input and output channels based on the channel dimensionality characteristics of the backbone network's output features, achieving channel dimensionality reduction and category semantic feature transformation. This maps the general features output by the backbone network to category-specific features. The batch normalization layer standardizes the convolutional output features, accelerating model training convergence by stabilizing the feature distribution. The ReLU activation function introduces a non-linear transformation, enhancing the network's ability to express category-specific features of remote sensing targets (such as aircraft wing proportions and ship deck structures), avoiding insufficient feature discrimination caused by linear transformations. The GAP layer, through global average pooling, compresses multi-dimensional feature maps into fixed-dimensional feature vectors, eliminating the interference of spatial dimension on category semantic information and ensuring that the output features focus solely on the essential attributes of the category. This branch inputs the features of all target instance blocks of the same category within the support set. After processing at each level, each instance block outputs a feature vector of fixed dimension. The arithmetic mean of the feature vectors of all instance blocks under the same category is then taken to obtain the "category prototype" of that category. This aggregation process effectively integrates common semantic features of similar targets while filtering out personalized interference features such as angular geometry and background texture, ensuring that the category prototype only carries category-specific semantic information, and providing a reliable semantic anchor for accurate identification of target categories in subsequent detection tasks.

[0065] Step 5: Calculate the cosine similarity between each category prototype and the corresponding angle prototype, construct the decoupling loss, and minimize the decoupling loss to make the two types of prototypes approach orthogonality in the feature space, thereby achieving the separation of category semantics and angle geometric features;

[0066] Specifically, the dimensions of the angle prototype are unified to match those of the category prototype through convolution operations. Then, the absolute value of the cosine similarity between each category prototype and its corresponding angle prototype is calculated. The decoupling loss formula is as follows: Where K is the total number of categories and M is the number of angle channels.

[0067] In step 6, a cosine loss constraint is introduced to achieve dual-prototype decoupling. To completely separate category semantics and angular geometric features, a dual-prototype decoupling mechanism is constructed using a cosine loss constraint: first, the prototype of each category is calculated. With each angle prototype The cosine similarity—this similarity directly reflects the degree of feature entanglement between the two prototypes; the closer the value is to 1, the more severe the feature confusion. Based on this, a decoupling loss is constructed. The formula is:

[0068]

[0069] Where K is the total number of categories and M is the number of angle channels. During model training, the goal is to minimize the decoupling loss by forcing the category prototypes and angle prototypes to approach an orthogonal state in the feature space (cosine similarity close to 0). This fundamentally severs the association between category semantics and angle geometric features, ensuring that subsequent RPN candidate box selection, detection head classification, and other processes are guided only by pure category semantics, thus avoiding detection errors caused by angle interference.

[0070] Step 7 optimizes candidate box generation by constructing a prototype collaborative guidance module. To improve the accuracy of RPN candidate box filtering, a dual-prototype collaborative guidance mechanism is constructed: first, channel-level aggregation is performed on the category prototypes of all base classes and new classes to obtain the "foreground category aggregated prototype". This integrates semantic information from all foreground categories, strengthening the model's foundation for foreground target recognition. A fully connected layer is built, into which the foreground category aggregation prototype is input, and the complementary weights of the RPN classification branch are output to dynamically adjust the RPN's scoring weights for foreground targets, reducing the probability of misjudging background regions.

[0071] Simultaneously, for the feature map output by the query set backbone network, the angular features of the candidate boxes are extracted in the coverage area of ​​each candidate box through the angular geometry branch. The cosine similarity between the angular features of the candidate boxes and the prototypes of each angular feature is calculated to obtain the angular matching degree of each candidate box. The original classification score (foreground / background score) of the RPN is weighted and fused with the angular matching degree according to the preset weight to generate the final score of the candidate boxes. High-confidence candidate boxes are selected to retain effective candidate boxes with high angular matching degree and foreground semantics, while low-confidence background candidate boxes are filtered out, laying the foundation for accurate classification by the subsequent detection head.

[0072] Step 8 involves constructing a detection head for classification and bounding box regression. Based on the Faster R-CNN detection head structure, RoI Align is performed on the filtered candidate boxes, mapping candidate boxes of different sizes to fixed-dimensional RoI features to ensure consistency in feature size for subsequent calculations. The RoI features are then multiplied by the corresponding category prototype using a channel-level multiplication operation, as shown in the following formula:

[0073]

[0074] in, Represents the basic features of the candidate bounding boxes after RoI Align processing. The category prototype representing the k-th target (has been decoupled by the previous two branches and constrained by cosine loss, and the angle and background interference have been removed, retaining only the category-specific semantics). This represents element-wise multiplication along the channel dimension. The input dimension of the fully connected layer matches the expanded dimension of the RoI features, and the output dimension includes the class probability corresponding to "number of classes + 1" (including the background class) and the horizontal bounding box coordinates of each target class. This operation weights and strengthens the "class-related semantics" in the RoI features through the class prototype—if a channel of the RoI feature has a high semantic match with the corresponding channel of the class prototype, the feature response of that channel is enhanced after multiplication; otherwise, irrelevant features are suppressed, thereby highlighting the target class attributes and reducing the interference of non-class features on the classification results.

[0075] The smooth L1 loss function is used to regress the target bounding box, and the horizontal bounding box that fits the remote sensing target is output to ensure that the bounding box and the target outline are accurately matched. A fully connected layer is built, and the adaptation parameters are set according to the RoI feature dimension and output requirements. Finally, the target's category probability and horizontal bounding box coordinates are output. The whole process relies on the pure category semantic information obtained by decoupling in the early stage, without introducing angular geometric branches to participate in the calculation, thus avoiding angular interference affecting the detection results.

[0076] In step 9, the formula for the total loss function is:

[0077]

[0078] in The loss includes RPN classification loss, RPN regression loss, and detector head classification loss. The RPN classification loss and detector head classification loss use the cross-entropy loss function to measure the model's classification error for "foreground / background" and "target category." The RPN regression loss and detector head regression loss use the smooth L1 loss function to reduce the error sensitivity of bounding box coordinate regression. The decoupling loss weight coefficients are determined through validation set tuning. In this embodiment... To balance the performance of the detection task with the decoupling effect of the dual prototypes, and to avoid excessive decoupling loss affecting the detection accuracy or too little decoupling loss causing feature entanglement.

[0079] Step 10 involves two stages of training: pre-training of the base class and fine-tuning of the new class. Both stages employ the stochastic gradient descent (SGD) optimizer. The initial learning rate for the first stage is set to the order of 0.001, and the initial learning rate for the second stage is set to the order of 0.0001. During training, the learning rate decays at preset intervals. The second stage uses a moving average strategy to update the prototype of the new class, achieving dynamic updates of the prototype through a weighted fusion of the mean features of the old prototype and the new samples.

[0080] Specifically, in the pre-training phase of the basic classes, training is conducted using only the support and query sets of the basic classes. The support set selects a sufficient number of instance blocks for each class to ensure prototype representativeness, while the query set randomly selects images covering all categories of the basic classes in each iteration. An SGD optimizer is employed, with initial learning rate, momentum, and weight decay (to suppress overfitting). Training is performed for 20 epochs, with the learning rate decreasing proportionally every 5 epochs. This gradual reduction of the learning rate ensures stable convergence of the model in the later stages of training. During training, the mAP of the basic class object detection is evaluated on the validation set every fixed number of iterations. The model weights with the highest mAP and their corresponding basic class category and angle prototypes are saved to ensure the model possesses good basic feature extraction and detection capabilities.

[0081] In the fine-tuning phase for new classes, a hybrid training dataset is constructed consisting of a subset of basic classes and a complete set of new classes. The subset of basic classes randomly selects samples from the basic class training set to prevent the model from forgetting basic class knowledge. The complete set of new classes includes all support and query set samples for all new classes to adapt to the small sample size of new class features. The SGD optimizer is used again, but the initial learning rate is reduced (to avoid overfitting due to the small number of new class samples). Training is performed for 10 epochs, with the learning rate decreasing proportionally every 3 epochs. In each iteration, a "moving average" strategy is used (new prototype = 0.7 × old prototype + 0.3 × mean of new sample features) to dynamically update the category prototype and angle prototype of the new class, allowing the prototype to gradually adapt to the new class features. At the same time, the mAP of new class detection is evaluated on the validation set after a fixed number of iterations, and the model weight with the highest mAP of the new class is saved to ensure that the model can quickly adapt to the new class features while retaining the basic class detection capability.

[0082] In step 11, the preprocessing of the remote sensing image to be detected according to the standard procedure is as follows: the image size is adjusted to fit the model input; standardization is performed based on the mean and standard deviation of the basic class training set; it is converted into a tensor compatible with the deep learning framework and batch dimension is added; and it is loaded into the computing device to accelerate subsequent operations. The preprocessed image is input into the trained shared backbone network, and multi-scale feature maps are output through the specified network layers. At the same time, the previously saved basic class and new class decoupled prototypes are loaded to ensure that the prototypes and the features of the image to be detected are in the same feature space, providing semantic and angular priors for subsequent guidance. The multi-scale feature maps are input into the prototype collaborative guidance module. The RPN first generates initial candidate boxes, adjusts the foreground score of the candidate boxes by aggregating prototypes through foreground categories, and obtains the angle matching degree by combining the similarity between the angle features of the candidate boxes and the angle prototype. The final score of the candidate boxes is calculated according to the rule "final score = preset weight × RPN original score + preset weight × maximum angle matching degree", and high-confidence candidate boxes are selected to remove background interference. The selected candidate bounding boxes undergo RoI Alignment to extract fixed-dimensional RoI features. These features are then compared with prototypes of all categories to match the target category. Channel-level weighting is used to enhance category semantic features. A smooth L1 loss function is employed to regress the horizontal bounding box coordinates. Non-maximum suppression is then used to remove overlapping candidate boxes, retaining the optimal detection result for each target. Finally, the detection results (target category, bounding box coordinates, and confidence score) are visualized using image annotation tools. Target locations and related information are annotated on the original image and saved in both image format (for intuitive visualization) and structured file format (for subsequent statistical analysis), completing the entire remote sensing image target detection process.

[0083] This invention also provides a few-shot target detection system based on decoupled prototypes, used to execute the method described in this invention. It extracts features using a shared backbone network, generates a "category semantic prototype" and an "angular geometric prototype" through two complementary branches, and then uses decoupling loss constraints to bring the two prototypes to near orthogonality in the feature space. Subsequently, these two decoupled prototypes jointly guide candidate box selection and target classification regression, thereby improving detection accuracy and angular robustness in few-shot scenarios. The system of this invention includes a dual-branch prototype decoupling module, a cosine loss constraint module, a prototype collaborative guidance module, a detection head and total loss optimization module, and a two-stage training module.

[0084] Specifically, in the dual-branch prototype decoupling module, ResNet101 pre-trained on the ImageNet dataset is used as a shared backbone network, and "category semantic branch" and "angular geometry branch" are built in parallel at its output to achieve independent generation and feature decoupling of category semantic prototype and angular geometry prototype:

[0085] Category Semantic Branch: The category semantic branch consists of a 1×1 convolutional layer, a batch normalization layer, a ReLU activation function, and a global average pooling (GAP) layer. It focuses on extracting semantic features relevant to the target category and suppressing interference components unrelated to angle, background, etc. The 1×1 convolutional layer configures the number of input and output channels according to the channel dimension of the deep feature map of the backbone network, achieving channel dimensionality reduction and category semantic feature mapping, transforming general features into more discriminative features. The batch normalization layer standardizes the convolutional output, stabilizing the feature distribution and accelerating model convergence. The ReLU activation function introduces a non-linear transformation, enhancing the network's ability to characterize fine-grained category differences (e.g., differences in ship deck structure, differences in aircraft wing proportions). The GAP layer performs global average pooling on the spatial dimension, compressing the multi-dimensional feature map into a fixed-dimensional vector, centrally preserving global semantic information at the category level. After processing all target instance blocks of the same category within the support set sequentially through a shared backbone network and a category semantic branch, several category feature vectors can be obtained. The arithmetic mean of all feature vectors corresponding to the category is then performed to obtain the category prototype. The category prototype mainly carries the stable semantic features of the category, effectively reducing the interference of individual sample angular geometry and background differences on the overall prototype.

[0086] Angular geometry branch: The angular geometry branch consists of a 1×1 convolutional layer, a multi-directional 3×3 convolutional layer, a batch normalization layer, a ReLU activation function, and a GAP layer. It is used to accurately characterize the angular geometry and geometric structure information of the target, while minimizing the removal of categorical semantic components. The 1×1 convolutional layer performs channel dimensionality reduction on the deep features output by the backbone network, reducing the computational cost of subsequent angular feature extraction. The multi-directional 3×3 convolutional layer, through the design of multiple sets of directional convolutional kernels (e.g., several dominant directions covering 0° to 180°), only produces significant responses to edges and contours consistent with or close to that direction, thereby capturing the target's tilt direction, major axis orientation, and other geometric pose information. The batch normalization layer and the ReLU activation function are used to stabilize the feature distribution during training and enhance non-linear expressive power, respectively. The GAP layer compresses the feature maps of multi-directional responses into fixed-dimensional vectors to represent the angular geometric features. Target instance blocks of the same category within the support set are sequentially input into the shared backbone network and the angular geometry branch. After processing by the above structure, multiple angular feature vectors are obtained. The arithmetic mean of all angular feature vectors corresponding to the category is taken to obtain the angular prototype of the category. The response intensity of the angular prototype on the multi-directional convolutional channel reflects the dominant tilting posture distribution of the category in several discrete directions, without the need to pre-divide the angle interval or label the precise angle. The angular prototype mainly describes the tilting direction and other posture information of the target contour, and tries not to carry the semantic features of the category, so as to form an orthogonal relationship with the category prototype in the feature space in the subsequent decoupling constraint.

[0087] In the cosine loss constraint module, to completely separate category semantics and angular geometric information, a "decoupling loss" is constructed based on cosine similarity, forcing the category prototype and angular prototype to be orthogonal in the feature space. First, a 1×1 convolution is used to unify the dimension of the angular prototype from 512 to 1024, ensuring it resides in the same feature space as the category prototype. Then, the absolute value of the cosine similarity between each category prototype and each angular prototype is calculated using the cosine similarity function. This absolute value reflects the degree of feature entanglement between the two types of prototypes (the closer the value is to 1, the more severe the entanglement). Finally, the decoupling loss formula is constructed:

[0088]

[0089] Where K is the total number of categories and M is the number of angle channels. During model training, by minimizing the decoupling loss, the cosine similarity between the category prototype and the angle prototype is made close to 0, thus achieving complete decoupling of the two types of features.

[0090] In the prototype collaborative guidance module, to address the shortcomings of traditional RPNs that rely solely on general features and single-category prototypes and lack adaptability to tilted targets and complex backgrounds, this invention proposes a dual-prototype collaborative guidance mechanism of "category prototype + angle prototype" to jointly optimize the generation and selection process of candidate boxes: All base class and new class category prototypes are aggregated along the channel dimension (e.g., by averaging or weighted summation) to form a "foreground category aggregate prototype." This aggregate prototype integrates the shared semantic features of all foreground categories, providing stable and generalizable semantic priors in foreground / background discrimination. This foreground category aggregate prototype is fed into a weight predictor composed of fully connected layers, outputting complementary weights for the RPN classification branch. This weights are used to dynamically adjust the original foreground score of the RPN, thereby improving the score of regions containing real targets and suppressing pseudo-high-scoring responses in pure background regions.

[0091] Secondly, for the feature map extracted from the query set image by the backbone network, the angle geometry branch is used to extract position-level / candidate box-level angle features. The cosine similarity between the angle feature of the position (or candidate box) and each angle prototype is calculated to obtain the angle matching degree corresponding to each position (or candidate box). For the candidate box proposed by the RPN, the angle matching degree of the candidate box can be calculated based on the similarity between the angle features in its coverage area and the angle prototype. Subsequently, the original foreground / background classification score of the RPN and the angle matching degree are weighted and fused according to preset weights to obtain the final score of the candidate box. Through this dual-prototype collaborative scoring method, high-confidence candidate boxes that simultaneously satisfy "semantic matching + angle matching" can be preferentially retained, while a large number of background candidate boxes that are only semantically vaguely similar or only geometrically coincidentally similar can be filtered out, thereby improving the recall rate of candidate boxes for tilted targets and reducing background interference.

[0092] The detection head in the detection head and total loss optimization module is designed as follows: Based on the Faster R-CNN detection head structure, it performs RoI Align operation on the candidate boxes selected by the prototype collaborative guidance module, uniformly mapping candidate boxes of different sizes to fixed-dimensional RoI features to ensure feature size consistency for subsequent calculations. By performing channel-level operations on RoI features and corresponding category prototypes, the semantic response related to the category in the RoI features is further enhanced. Since the category prototypes have been completely stripped of angular geometric interference through the previous two-branch decoupling and cosine loss constraint, only category-specific semantic information is retained. Therefore, this operation can accurately improve the target category classification accuracy and avoid interference from angular differences in category judgment. At the same time, the smooth L1 loss function is used to regress the target bounding box, outputting a horizontal bounding box adapted to the remote sensing target to ensure that the bounding box accurately fits the target contour. A fully connected layer is also built, and the adaptation parameters are set according to the RoI feature dimension and output requirements. Finally, the target category probability and horizontal bounding box coordinates are output. The entire process relies on the pure category semantic information obtained from the previous decoupling, without the need to introduce angular geometric branches to participate in the calculation.

[0093] A fully connected layer is built, and the input, intermediate, and output dimensions are set according to the RoI feature dimensions and output requirements (the output dimension includes the number of categories, the probability corresponding to the background class, and the bounding box coordinate parameters). Finally, the target category probability and bounding box coordinates are output.

[0094] The total loss is: the total loss function consists of "detection loss" and "decoupling loss", and the formula is as follows: .in, It includes RPN classification loss, RPN regression loss, and detector head classification loss. The RPN classification loss and detector head classification loss use the cross-entropy loss function to measure the model's classification error for "foreground / background" and "target category". The RPN regression loss and detector head regression loss use the smooth L1 loss function to reduce the error sensitivity of bounding box coordinate regression and avoid outlier interference. The decoupling loss weight coefficient was determined to be 0.1 through validation set tuning. This is used to balance detection performance and the decoupling effect of the dual prototypes, avoiding excessive decoupling loss that affects detection accuracy or excessive decoupling loss that leads to feature entanglement.

[0095] In the two-stage training module, to achieve efficient transfer of knowledge from basic classes to new classes and avoid overfitting and knowledge forgetting issues during the fine-tuning stage of new classes, a phased progressive training strategy is designed to ensure the model's feature learning and generalization capabilities in small-sample scenarios. The first stage (basic class pre-training) uses only the support set and query set of the basic classes for training. The support set selects sufficient instance blocks for each class to ensure the representativeness of the class prototypes, and the query set is randomly sampled in each iteration to cover all basic classes, avoiding data bias affecting feature learning. The optimizer used is SGD, with an initial learning rate set to 0.001, combined with conventional momentum and weight decay mechanisms to suppress overfitting. The training cycle is set to 20 epochs, with the learning rate decaying every 5 epochs according to a preset ratio to ensure stable convergence of the model in the later stages of training. During training, the total loss function is optimized to enable the model to simultaneously master the ability to extract the decoupled prototype of the basic class, as well as the basic detection functions of RPN and detection head. After training, the model weights with the best mean average precision (mAP) on the validation set are saved, along with the corresponding class prototypes and angle prototypes of the basic class, laying the foundation for subsequent adaptation to new classes.

[0096] The second stage (fine-tuning of new classes) constructs a hybrid training dataset consisting of a subset of basic classes and a complete set of new classes. The subset of basic classes randomly selects samples from the basic class training set (to prevent the model from forgetting basic class knowledge), while the complete set of new classes includes all support and query set samples for all new classes (to adapt to the small sample features of new classes). The SGD optimizer is used again, but the initial learning rate is reduced (to adapt to the small sample scenario of new classes and reduce the risk of overfitting). The training cycle is set to 10 epochs, with the learning rate decaying every 3 epochs according to a preset ratio. In each iteration, a "moving average" strategy is used to dynamically update the category prototype and angle prototype of the new class. By weighted fusion of the old prototype and the mean of the new sample features, the prototype gradually fits the new class features. Every fixed number of iterations, the mAP of the new class detection is evaluated on the validation set, and the optimal model weights for the new class mAP and the updated new class prototype are saved to ensure that the model retains the basic class detection capability while efficiently adapting to the new class features.

[0097] This application also provides an electronic device. Figure 2The structure of an electronic device provided in an embodiment of the present invention is illustrated. For example, the electronic device 20 may include a processor 21, a memory 22, and a transmission device 23. The processor is used to execute the small sample target detection method based on a decoupled prototype mentioned in the above embodiments. The processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The transmission device can be connected to the processor and the memory via wired or wireless means. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the small sample target detection method based on a decoupled prototype in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the small sample target detection method based on a decoupled prototype in the above method embodiments. The memory may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and this remote memory may be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The one or more modules stored in the memory, when executed by the processor, perform the small-sample target detection method based on a decoupled prototype in the embodiments.

[0098] In another aspect, this application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into the device. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the small sample target detection method based on a decoupled prototype described in this application.

[0099] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A small-sample target detection method based on a decoupled prototype, characterized in that, Includes the following steps: Acquire remote sensing images containing targets, label the location, category, and binary mask of the targets, divide the images into basic classes and new classes, and divide the new classes into support sets and query sets according to a preset ratio; The images of the base class and the new class are preprocessed, including pixel value normalization and mean standard deviation normalization, and the processed images and labels are converted into tensors compatible with deep learning frameworks. Load the pre-trained shared backbone network, freeze some layer weights, and fine-tune the later layer weights to adapt to the remote sensing image features. At the output of the shared backbone network, a category semantic branch and an angle geometry branch are constructed. The same type of target features of the support set are input into the two branches respectively: the category semantic branch generates category prototypes, and the angle geometry branch extracts angle features through multi-directional convolution and global pooling and aggregates them to generate angle prototypes. Calculate the cosine similarity between the category prototype and the angle prototype, construct the decoupling loss, and minimize the decoupling loss to make the two types of prototypes approach orthogonality in the feature space; Channel-level aggregation is performed on all category prototypes to obtain foreground category aggregation prototypes, and the complementary weights of the network classification branches are generated by inputting the output region of the fully connected layer. For the feature map output by the backbone network of the query set, the angle features of the candidate boxes are extracted within the candidate box coverage area through the angle geometry branch. The similarity between the angle features of the candidate boxes and the angle prototype is calculated to obtain the angle matching degree of the candidate boxes. The original classification scores of the region generation network and the angle matching degree are weighted and fused according to preset weights to filter high-confidence candidate boxes; The filtered candidate boxes are aligned with regions of interest to obtain fixed-dimensional region of interest features. These features are then processed at the channel level with the corresponding category prototype. During the training phase, a smooth loss function is used to regress the horizontal bounding box. The target category probability and bounding box coordinates are output through a fully connected layer. Construct the total loss function, including detection loss and decoupling loss, calculate the loss between the predicted results and the actual results, and backpropagate to update the model parameters; A two-stage training strategy is adopted: the first stage uses the support set and query set of the base class for training, and saves the optimal model weights and the prototype of the base class; the second stage uses a mixed dataset of a subset of the base class and the complete set of the new class for training, dynamically updates the prototype of the new class, and saves the optimal model weights of the new class. The remote sensing image to be detected is preprocessed, input into the trained model, and the detection result is output.

2. The method according to claim 1, characterized in that, The category semantic branch includes a convolutional layer, a batch normalization layer, an activation function, and a global average pooling layer connected in sequence; the angular geometry branch includes a convolutional layer, a multi-directional convolutional layer, a batch normalization layer, an activation function, and a global average pooling layer connected in sequence; the decoupling loss is obtained by calculating the absolute value of the cosine similarity between the prototype of the category and its corresponding angular prototype for each category, and by calculating the cosine similarity for each category, which is used to constrain the category prototype and the angular prototype to be approximately orthogonal in the feature space; the complementary weights of the region generation network classification branch are obtained by aggregating the foreground category prototype through a fully connected layer, and are used to adjust the scoring weights of the region generation network for the foreground target.

3. The method according to claim 1 or 2, characterized in that, The candidate box angle matching degree is calculated by the cosine similarity between the candidate box angle features extracted from the query set feature map through the angle geometry branch and the angle prototype.

4. The method according to claim 1 or 2, characterized in that, The channel-level operation involves element-wise multiplication of the region of interest features and the category prototype along the channel dimension to enhance category-related semantic features.

5. The method according to claim 1, characterized in that, In the two-stage training strategy, the first stage of training uses basic class data, and the second stage of training uses mixed data of a subset of basic classes and the entire set of new classes, and dynamically updates the prototype of the new classes through a moving average strategy.

6. A small-sample target detection system based on a decoupled prototype, characterized in that, include: The dual-branch prototype decoupling module is used to generate category prototypes and angle prototypes through category semantic branches and angle geometry branches, respectively. The cosine loss constraint module is used to calculate the cosine similarity between the category prototype and the angle prototype, construct the decoupling loss, and make the two types of prototypes approach orthogonality in the feature space. The prototype collaborative guidance module is used to aggregate prototypes and angle prototypes to collaboratively guide the candidate box filtering of the region generation network using foreground categories; The detection head and total loss optimization module are used to classify candidate boxes and regress bounding boxes, and calculate the total loss. The two-stage training module is used to execute a two-stage training strategy and optimize model parameters.

7. The system according to claim 6, characterized in that, In the dual-branch prototype decoupling module, both the category semantic branch and the angle geometry branch are constructed based on the output of the shared backbone network. The category semantic branch focuses on extracting category semantic features, while the angle geometry branch focuses on extracting angle geometry features.

8. The system according to claim 6 or 7, characterized in that, The prototype collaborative guidance module generates complementary weights for network classification branches by aggregating the prototype output region through foreground categories, and generates the original classification score of the network by weighting and fusing the region through angle matching degree.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 5.

Images