Method and system for generalized small sample target detection of ocean remote sensing with controlled knowledge transfer

Abstract

The application discloses a kind of controlled knowledge migration marine remote sensing generalized small sample target detection method and system, the method includes the following steps: multi-scale feature extraction is carried out to remote sensing image, and multi-scale feature representation is generated;Multi-scale feature representation is generated candidate region feature by regional suggestion network, and candidate region feature is respectively inputted parameter independent base class feature extraction branch and new class feature extraction branch, and base class feature and new class feature are respectively generated;Base class feature and new class feature are carried out two-level fusion processing, and fusion feature is generated;With knowledge distillation loss and mutual information regularization loss, double constraints are applied to fusion feature, knowledge distillation loss constraint consistency between fusion feature and base class feature, and mutual information regularization loss reduces the redundant correlation between new class feature and base class feature;Fusion feature is respectively inputted classification branch and regression branch, and detection result is obtained.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a controlled knowledge transfer method and system for generalized small-sample target detection in marine remote sensing. Background Technology

[0002] Constructing a wide-area, timely intelligent monitoring system for marine and coastal targets has become a crucial technological requirement for ensuring maritime traffic and operational safety, supporting the development and utilization of marine resources, and improving the level of refined marine management. In typical marine areas, dense maritime activities and diverse target types place higher demands on the automated identification and continuous monitoring of marine targets. Utilizing optical remote sensing imagery to detect and identify targets such as ships, port facilities, offshore oil and gas platforms, and near-shore islands and reefs is an important technological means to achieve dynamic marine spatial supervision.

[0003] Driven by advancements in computer vision and remote sensing information processing technologies, deep learning-based remote sensing target detection methods have made rapid progress in recent years. These methods typically rely on training with a large number of labeled samples and learn the appearance features and spatial distribution patterns of targets to achieve automatic identification of multiple target categories. Such methods achieve good detection results for target categories with relatively abundant samples. However, in practical marine remote sensing applications, the distribution of remote sensing target categories often exhibits a significant imbalance. Some common targets have relatively abundant labeled samples, while a large number of target categories with significant application value have limited labeled samples. To alleviate the performance degradation caused by insufficient samples, related research has begun to introduce knowledge transfer techniques, transferring the discriminative abilities learned in categories with abundant samples to categories with scarce samples, thereby improving detection performance under limited sample conditions.

[0004] Although knowledge transfer-based object detection methods have improved the recognition performance of categories with scarce samples to some extent, they still show significant shortcomings in complex marine remote sensing scenarios. For example... Figure 1 As shown, target detection in complex marine remote sensing scenarios often faces challenges such as the coexistence of multiple target categories, significant differences in target scale, and uneven distribution of sample numbers across categories. For example... Figure 1 As shown in (a), in complex port scenes, targets such as ships, bridges, and storage tanks often coexist in the same image; such as Figure 1 As shown in (b), the coexistence of large and small targets leads to significant scale variations; Figure 1As shown in (c), the number of samples across different categories typically exhibits a significant long-tail distribution, with the category with the larger sample size designated as the base class and the category with the smaller sample size designated as the new class. Under these conditions, existing knowledge transfer methods tend to simultaneously introduce background-related features and scale priors from the category with sufficient samples during the transfer process. When the transfer process lacks differentiation, the feature boundaries between different categories are easily weakened, leading to unstable discriminative ability for the category with scarce samples and even affecting the original detection performance of the category with sufficient samples. In practical applications of multi-class joint detection, these problems further restrict the practicality of knowledge transfer-based methods. Summary of the Invention

[0005] This invention proposes a controlled knowledge transfer method and system for generalized small-sample target detection in marine remote sensing, which solves the problem that existing technologies easily introduce background-related features and scale priors during the knowledge transfer process, leading to unstable class discrimination ability due to scarce samples.

[0006] To address the aforementioned technical problems, this invention provides a controlled knowledge transfer-based method for generalized small-sample target detection in marine remote sensing, comprising the following steps:

[0007] Step S1: Extract multi-scale features from the remote sensing image using a multi-scale feature extraction network to generate a multi-scale feature representation;

[0008] Step S2: The multi-scale feature representation is processed by a region proposal network to generate candidate region features. The candidate region features are then input into a parameter-independent base class feature extraction branch and a new class feature extraction branch to generate base class features and new class features, respectively.

[0009] Step S3: The base class features and the new class features are fused in two stages through the cross-class feature adaptive fusion module to generate fused features. The first stage adaptively adjusts the cross-class knowledge transfer amount based on cosine similarity, and the second stage adaptively adjusts the feature fusion ratio based on normalized weights.

[0010] Step S4: Apply dual constraints to the fused feature using knowledge distillation loss and mutual information regularization loss. The knowledge distillation loss constrains the consistency between the fused feature and the base class feature, and the mutual information regularization loss reduces the redundant correlation between the new class feature and the base class feature.

[0011] Step S5: Input the fused features into the classification branch and the regression branch respectively to obtain the detection results.

[0012] Preferably, in step S1, a multi-scale feature extraction network is constructed by embedding a multi-scale dilated convolutional attention mechanism in the lateral connection paths of the feature pyramid network. The multi-scale dilated convolutional attention mechanism includes:

[0013] The input feature map is projected by a 1×1 convolution and split along the channel dimension into a query tensor, a key tensor, and a value tensor.

[0014] The query tensor, the key tensor, and the value tensor are each divided along the channel dimension. N Each subspace corresponds to a different void ratio, wherein the bond tensor is spatially sparsely sampled in each subspace through void convolution with different void ratios to obtain receptive fields of different scales.

[0015] Within each subspace, attention responses are calculated based on scaled dot product attention, and the attention outputs of each subspace are weighted and fused using adaptive weights based on global average pooling to generate multi-scale attention features.

[0016] Preferably, the multi-scale attention features are injected into the lateral connection path through gated residual connections:

[0017] ;

[0018] In the formula, Output feature map; The input feature map; The multi-scale attention features; The gating factor; This is an element-wise multiplication operation.

[0019] Preferably, the adaptive adjustment of cross-category knowledge transfer based on cosine similarity includes the following steps:

[0020] The base class features With the new class features The system is divided into G subspaces along the channel dimension. Within each subspace, the cosine similarity between the base class features and the new class features is calculated, resulting in a G-dimensional similarity vector. ;

[0021] Based on the similarity vector The new class of features is weighted group by group at the channel group level, and a compensation coefficient 1 is constructed based on the similarity vector. The base class features are weighted group by group at the channel group level, and then the two weighted results are added together to obtain the transfer features. :

[0022] ;

[0023] In the formula, This is an element-wise multiplication operation; For the first The base class features corresponding to each candidate region; For the first New class features corresponding to each candidate region; For the first The candidate region in the first g Cosine similarity over each channel subspace; For the first Migration features corresponding to each candidate region;

[0024] Within different channel subspaces, when the cosine similarity of the corresponding subspaces is low, the transfer contribution of the base class features in the current subspace increases; when the cosine similarity of the corresponding subspaces is high, the transfer contribution of the base class features in the current subspace decreases.

[0025] Preferably, the adaptive adjustment of the feature fusion ratio based on normalized weights includes the following steps:

[0026] Independent linear transformations are applied to the base class features and the new class features respectively, and their respective response values ​​are generated by the Sigmoid activation function.

[0027] The two response values ​​are normalized to obtain the base class fusion weight. Weighting for integration with new categories ;

[0028] Based on the base class fusion weight and the new class fusion weight, the transfer feature and the new class feature are weighted and summed to generate the fusion feature:

[0029] ;

[0030] In the formula, For the first The fusion features corresponding to each candidate region; For the first The base class fusion weights corresponding to each candidate region; For the first The new fusion weights corresponding to each candidate region.

[0031] Preferably, the mutual information regularization loss is defined as:

[0032] ;

[0033] In the formula, Define the mutual information regularization loss; The number of samples for the new class feature; The number of samples representing base class features; The cosine similarity function; This refers to temperature hyperparameters. For the first Feature vectors of new class feature samples; For the first Feature vectors of base class feature samples; The similarity threshold;

[0034] When the cosine similarity between the new class feature and the base class feature exceeds the similarity threshold A gradient penalty is applied when the similarity falls below the threshold. It does not participate in gradient updates.

[0035] Preferably, steps S1 to S4 employ a two-stage training strategy, including:

[0036] Base class pre-training stage: The backbone network and the multi-scale feature extraction network are trained end-to-end using the base class dataset. The optimization objectives include region proposal loss, classification loss and bounding box regression loss.

[0037] New Class Fine-tuning Stage: The parameters of the backbone network, the multi-scale feature extraction network, and the base class feature extraction branch are frozen. The number of output nodes in the classification head of the classification branch is expanded from the number of base class categories to the sum of the number of base class categories and the number of new class categories. Parameter updates are performed only on the classification branch, the regression branch, and the cross-class feature adaptive fusion module. The total loss function is:

[0038] ;

[0039] in, This is the total loss function; The region is advised to bear losses; For classification loss; For bounding box regression loss; and These are the loss weight coefficients for knowledge distillation loss and mutual information regularization loss, respectively.

[0040] Preferably, the base class feature extraction branch and the new class feature extraction branch have the same structure and their weight parameters are independent of each other. Each feature extraction branch includes two cascaded fully connected layers. During the new class fine-tuning stage, the weight parameters of the base class feature extraction branch are kept frozen, and the base class features participate in the two-level fusion process as anchor reference features.

[0041] Preferably, the adaptive weights based on global average pooling are generated in the following manner:

[0042] Global average pooling is performed on the input feature map, followed by 1×1 convolution mapping and then the weight coefficients for each subspace are generated using the Softmax normalization function. The attention outputs of each subspace are then weighted and summed.

[0043] ;

[0044] in, For the first Attention output of each subspace; The total number of subspaces; For the first The weight coefficients corresponding to each subspace; This represents the multi-scale attention features after weighted fusion.

[0045] This invention also provides a generalized small-sample target detection system for remote sensing images based on controlled knowledge transfer, implemented based on the aforementioned controlled knowledge transfer method for generalized small-sample target detection in marine remote sensing, comprising:

[0046] Multi-scale feature extraction module: Extracts multi-scale feature representations from remote sensing images through a multi-scale dilated convolutional attention mechanism in the lateral connection path of the feature pyramid network;

[0047] Candidate region feature generation module: The multi-scale feature representation is processed by a region proposal network to generate candidate region features, and the candidate region features are respectively input into a parameter-independent base class feature extraction branch and a new class feature extraction branch to generate base class features and new class features;

[0048] Cross-category feature adaptive fusion module: performs two-level fusion processing on the base class features and the new class features. The first level adaptively adjusts the cross-category knowledge transfer amount based on cosine similarity, and the second level adaptively adjusts the feature fusion ratio based on normalized weights to generate fused features.

[0049] Dual regularization constraint module: constrains the consistency between the fused feature and the base class feature through knowledge distillation loss, and reduces the redundant correlation between the new class feature and the base class feature through mutual information regularization loss;

[0050] The target detection output module inputs the fused features into the classification branch and the regression branch respectively. The classification branch performs joint classification prediction of the base class and the new class for the candidate region, and the regression branch performs bounding box coordinate regression for the candidate region, outputting the class labels and localization results of the base class target and the new class target in the remote sensing image.

[0051] The advantages of this invention include at least the following:

[0052] 1. By adaptively adjusting the scope and intensity of the transfer from base class knowledge to new classes, the performance degradation of the base class caused by insufficient transfer and the limited discrimination ability of the new class caused by excessive transfer are effectively avoided. Thus, the goal of synergistic optimization of maintaining stable base class detection performance and effectively improving new class detection ability is achieved in the unified model.

[0053] 2. By embedding a multi-scale dilated convolutional attention mechanism in the lateral connection path of the feature pyramid network, the semantic consistency and representation stability between features of different scales are improved by combining multi-scale perception and attention modulation. This enables the model to obtain multi-scale feature representations with strong robustness and transferability in complex ocean and coastal remote sensing scenarios, providing a stable and reliable feature foundation for subsequent knowledge transfer processes.

[0054] 3. A two-level adaptive control mechanism is adopted to selectively control the cross-category feature fusion process. The first level is based on cosine similarity to adaptively adjust the cross-category knowledge transfer content, and the second level is based on normalized weights to adaptively adjust the feature fusion ratio. This can guide the transfer of common features with transfer value and suppress the interference of irrelevant or redundant features, thereby avoiding the weakening of the base class features' ability to distinguish new classes and maintaining the stability of the class discrimination boundary under small sample conditions. Attached Figure Description

[0055] Figure 1 This is a schematic diagram illustrating a typical problem in the generalized small-sample target detection task of marine remote sensing. Figure 1 (a) represents the coexistence of multiple target categories in a complex background scene. Figure 1 (b) Shows the scale differences between different categories of objectives. Figure 1 (c) represents the distribution of long-tail samples between the base class and the new class.

[0056] Figure 2 This is a flowchart illustrating the overall architecture of the detection model in this embodiment of the invention.

[0057] Figure 3 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the adaptive dilated convolutional attention submodule structure applied to a single feature layer in an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram of the cross-category feature regulation and fusion process of the feature fusion perception module in an embodiment of the present invention;

[0060] Figure 6 This table presents the comparative experimental results of the model of this invention with existing methods under different class divisions and different K-shot settings on the NWPU VHR-10.v2 dataset.

[0061] Figure 7 The comparison results of the model of this embodiment with existing methods on the DIOR dataset are shown.

[0062] Figure 8 This is a visualization example of the detection results of the model in this embodiment of the invention and a comparison with existing methods. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0064] This invention proposes a controlled knowledge transfer-based generalized small-sample target detection method for marine remote sensing. Under limited sample conditions, this method achieves stable maintenance of base class detection performance and synergistic improvement of new target discrimination capabilities through fine-tuning of the knowledge transfer process. For example... Figure 2 As shown, the overall architecture of the detection model in this embodiment of the invention includes a backbone network, an adaptive feature pyramid, and a detection head. The backbone network is used to extract multi-layer features from the input remote sensing image and outputs C2, C3, C4, and C5 feature layers. The adaptive feature pyramid enhances and fuses the multi-layer features, generating P2, P3, P4, P5, and P6 multi-scale feature layers. The detection head includes a pre-training stage and a fine-tuning stage: in the pre-training stage, after the multi-scale features are processed by the region proposal network to generate candidate regions, class prediction and location regression are completed; in the fine-tuning stage, the candidate region features are respectively input into the base class feature extraction branch and the new class feature extraction branch, and after cross-class feature adaptive fusion, the class prediction result and the location regression result are output. Furthermore, during the fine-tuning training process, double regularization constraints can also be combined to adjust the knowledge transfer process between the base class and the new class. Figure 3 As shown, the detection process includes the following steps:

[0065] Step S1: Extract multi-scale features from the remote sensing image using a multi-scale feature extraction network to generate a multi-scale feature representation.

[0066] To address the problem that existing feature pyramid networks are susceptible to scale differences in multi-scale target feature extraction from marine and coastal remote sensing images, leading to instability of base class features when transferred to new target classes, this invention proposes an Adaptive Feature Pyramid Network (AFPN). This network introduces an Adaptive Dilated Convolutional Attention (Adaptive-DCA) submodule into the lateral connection paths of multiple scale branches in the traditional Feature Pyramid Network (FPN) to generate multi-scale feature representations that are friendly to subsequent transfer tasks and semantically consistent. This provides a stable and decoupled feature foundation for subsequent feature fusion perception modules and dual regularization constraint modules. Figure 4 This demonstrates the internal structure of the submodule at a single feature layer.

[0067] Specifically, given an input feature map The Adaptive-DCA module first projects the input feature map using a 1×1 convolution and then splits it into query tensors along the channel dimension. Key tensors AND-value tensor :

[0068] ;

[0069] Then query the tensor Key tensors AND-value tensor Divided along the channel dimension Each subspace is mapped through a linear layer and then input into a multi-head self-attention mechanism, while a preset hole ratio is assigned to each subspace. In this embodiment, the void ratio is set to an increasing sequence {1, 2, 4} to construct a receptive field hierarchy that expands progressively from local to global. Within each subspace, key features... Sparse sampling is performed in the spatial dimension using dilated convolutions with corresponding dilation rates to expand the receptive field, while the query... Sum The standard linear projection remains unchanged. The small-hole-rate branch focuses on capturing local texture and fine-grained object information, while the large-hole-rate branch encodes long-range context and global structural information. Within each subspace, the attention response is computed based on scaled dot product attention.

[0070] ;

[0071] ;

[0072] in This represents the subspace index. The attention outputs of each subspace are weighted and fused using adaptive weights based on global average pooling. Specifically, for the input feature map... A global average pooling operation is performed, followed by 1×1 convolution mapping and then the Softmax normalization function is used to generate the weight coefficients for each subspace. Finally, the attention outputs of each subspace are weighted and summed.

[0073] ;

[0074] ;

[0075] In the formula, For the first Attention output of each subspace; The total number of subspaces; For the first The weight coefficients corresponding to each subspace; The weighted and fused multi-scale attention features are then uniformly mapped by a linear transformation and further input into a multilayer perceptron consisting of linear layers, GELU activation units, and another linear layer to obtain enhanced output features.

[0076] The multi-scale attention features output by the Adaptive-DCA submodule are used to enhance the feature representation of the corresponding scale layers. To reintroduce the enhanced features into the lateral connection paths of the adaptive feature pyramid network, such as... Figure 2 As shown, the fused multi-scale attention features are combined with the original input features through a gated residual connection structure:

[0077] ;

[0078] In the formula, Output feature map; Input feature map; Multi-scale attention features; The gating factor; This is an element-wise multiplication operation.

[0079] Gating factor The contribution of enhanced features is adaptively adjusted based on the statistical properties of the fused features, thereby enhancing semantic information while maintaining the stability of the pyramid structure. This is achieved by... Figure 4 The Adaptive-DCA submodule shown is deployed at multiple different scale layers, and according to... Figure 2The method shown re-injects the enhanced features into the lateral connection path of the adaptive feature pyramid network. AFPN can generate multi-scale features that are more robust to scale changes and semantically consistent, providing a semantically consistent and stable feature foundation for the subsequent feature fusion perception module and dual regularization constraint module. This enables the simultaneous improvement of base class performance and new class discrimination ability in generalized few-sample object detection.

[0080] Step S2: The multi-scale feature representation is processed by the region proposal network to generate candidate region features. The candidate region features are then input into the parameter-independent base class feature extraction branch and the new class feature extraction branch to generate base class features and new class features respectively.

[0081] The multi-scale feature representations generated after processing by the feature extractor and adaptive feature pyramid network are input into the Region Proposal Network (RPN) to generate candidate regions. After region alignment, the candidate regions are transformed into region feature representations with uniform dimensions and mapped to a uniform feature space. Subsequently, these region feature representations are input into the Base Fully Connected Subnetwork (Base_FCS) and the Novel Fully Connected Subnetwork (Novel_FCS), respectively. Base_FCS and Novel_FCS have the same network structure, and each branch includes two sequentially connected fully connected mapping layers, but their parameters are set independently and do not share parameters. By performing linear transformations on the candidate region features through these two branches, feature representations representing the base class discriminative information can be obtained. and feature representations used to characterize new class discriminative features During the fine-tuning training phase for the new class, Base_FCS is initialized using the parameters obtained from step S1 and remains frozen during the fine-tuning process to ensure... As stable anchoring reference features, they participate in subsequent fusion processing; Novel_FCS uses the Xavier method for parameter initialization, and new class features... The learnable state is maintained during fine-tuning so that the new class feature representation can fully characterize the new class discriminative information.

[0082] Step S3: The base class features and new class features are fused in two stages through the cross-class feature adaptive fusion module to generate fused features. The first stage adaptively adjusts the cross-class knowledge transfer amount based on cosine similarity, and the second stage adaptively adjusts the feature fusion ratio based on normalized weights.

[0083] like Figure 5As shown, this embodiment of the invention proposes a Feature Fusion Perception (FFP) module for adjusting the transfer content and ratio of cross-class features. This module takes the region feature representation obtained by aligning candidate regions and mapping them to a unified feature space as input features, and first forms base class features. With new class features And generate transfer features through similarity calculation. Subsequently, the transfer features and the new class features are further fused to obtain the fused features. The fused features are then input into subsequent detection heads for category prediction and location regression. The module employs a two-level adaptive control mechanism to achieve controlled fusion of cross-category knowledge, thereby generating highly discriminative and stable fused features within a unified feature space. This supports controlled knowledge transfer. Different colors in the diagram are used to illustrate the distinguishing relationships between base class features, new class features, similar features, and enhanced features.

[0084] In the first-level adaptive adjustment stage, the cross-class feature similarity of corresponding candidate regions is measured by calculating the cosine similarity between base class features and new class features. To improve adjustment accuracy, the base class features and new class features are divided into G subspaces along the channel dimension, and the cosine similarity is calculated in each subspace to obtain the first-level adaptive adjustment. i The G-dimensional similarity vector corresponding to each candidate region , of which g Each component Indicates the first i The candidate region in the first g Cosine similarity over each channel subspace:

[0085] ;

[0086] Based on similarity vector At the channel group level, the new class features are weighted group by group, and a compensation coefficient is constructed based on the similarity vector. At the channel group level, the base class features are weighted group by group, and then the two weighted results are added together to obtain the transfer features. :

[0087] ;

[0088] In the formula, This is an element-wise multiplication operation; For the first The base class features corresponding to each candidate region; For the first New class features corresponding to each candidate region; For the first The candidate region in the first g Cosine similarity over each channel subspace; For the first The migration features corresponding to each candidate region.

[0089] The number of channels G is a preset value or is adaptively determined based on the feature dimensions.

[0090] Since the compensation coefficient and similarity have an inverse relationship, in different channel subspaces, when the cosine similarity of the corresponding subspace is low, it indicates that the difference between the new class features and the base class features is large. At this time, the participation of the base class features is correspondingly increased, thereby enhancing the transfer of base class knowledge to stabilize the new class representation. When the similarity is high, it indicates that the new class features have fully absorbed the base class information. At this time, the compensation effect of the base class features is automatically weakened to maintain the discriminative independence of the new class features and achieve adaptive adjustment of the knowledge transfer content.

[0091] After completing the content transfer adjustment, the second-level adaptive adjustment stage begins. Independent linear transformations are applied to the base class features and the new class features, and their respective response values ​​are generated using the Sigmoid activation function. These response values ​​are then normalized to obtain the fusion weights. The fusion weights are used to weightedly combine the knowledge transfer features and the new class features to achieve adaptive adjustment of the fusion ratio, while satisfying the normalization constraint that the sum of the values ​​must be 1.

[0092] ;

[0093] ;

[0094] In the formula, σ(·) is the Sigmoid activation function; and These are the linear transformation weight matrices corresponding to the base class features and the new class features, respectively; For the first The base class fusion weights corresponding to each candidate region; For the first The new fusion weights corresponding to each candidate region.

[0095] Finally, the knowledge transfer features and new class features are weighted and combined according to the normalized fusion weights to generate a fused feature representation:

[0096] ;

[0097] In the formula, For the first The fusion features corresponding to each candidate region.

[0098] Through the above two-level adaptive adjustment process, the FFP module coordinates the cross-category feature interaction at two levels: knowledge transfer content and feature fusion ratio. This distinguishes it from the traditional method of feature fusion based solely on single-level weight allocation, generating structurally stable and discriminative fusion features. This provides a consistent and decoupled feature representation basis for the subsequent dual regularization constraint module.

[0099] Step S4: Apply dual constraints to the fused features using knowledge distillation loss and mutual information regularization loss. Knowledge distillation loss constrains the consistency between the fused features and the base class features, while mutual information regularization loss reduces the redundant correlation between the new class features and the base class features.

[0100] To achieve controlled knowledge transfer, this embodiment of the invention employs a dual regularization constraint mechanism to separately control the stability and discriminativeness of features. The knowledge distillation loss uses frozen base class features. For reference, regarding fusion features Imposing consistency constraints ensures that the model maintains the stability of base class feature representations during training with new class samples, thereby suppressing unnecessary changes to base class discriminative information. The knowledge distillation loss is defined as:

[0101] ;

[0102] In the formula, For knowledge distillation loss; The number of samples representing base class features; For the first The base class features corresponding to each candidate region; For the first The fusion features corresponding to each candidate region.

[0103] To further prevent the new class features from being over-assimilated by the base class semantics during the transfer process, this embodiment of the invention also introduces mutual information regularization loss. This loss imposes a conditional penalty on the similarity between the new class features and the base class features, limiting the excessive convergence of the new class features to the base class discriminative structure. Simultaneously, it excludes the new class features from gradient updates when the similarity is below a set threshold, thus maintaining the necessary degrees of freedom in the feature space for the new class features. The mutual information regularization loss is defined as follows:

[0104] ;

[0105] In the formula, For mutual information regularization loss; The number of samples for the new class feature; The cosine similarity function; This is a temperature hyperparameter used to adjust the smoothness of the similarity distribution; its value is a positive number, and the preferred value range is [value range missing]. In this embodiment, the value is taken as 0.05. In other embodiments, it can be adjusted according to the feature distribution or training stability; For the first Feature vectors of new class feature samples; For the first Feature vectors of base class feature samples; This is a similarity threshold used to gate the cross-class similarity between new class features and base class features. During training, the similarity threshold... Update according to preset decay nodes; whenever the training iteration count reaches a preset decay node, update the current similarity threshold to a preset proportion of the previous stage threshold. For any new class feature... Features of the base class When cosine similarity Greater than the current similarity threshold At that time, Xiang A value greater than 0 corresponds to the feature pair participating in gradient constraints; when the cosine similarity is less than or equal to the current similarity threshold... When the term is 0, the corresponding feature pair does not participate in the gradient update of this constraint term.

[0106] In one specific embodiment, the initial value of the similarity threshold is set to 0.5, and it is updated in a stepwise manner when the training steps reach 1000 and 1500, with each update being 0.1 times the threshold of the previous stage. Since the similarity threshold is relatively large in the early stages of training, there are fewer cross-class feature pairs involved in the constraint, corresponding to a weaker constraint strength. As training progresses, the similarity threshold gradually decreases at each preset decay node, and the number of feature pairs involved in the constraint gradually increases, thereby achieving a gradual adjustment of the cross-class feature dependency relationship from weak to strong. This constraint does not push the new class features out of the base class feature space as a whole, but only restricts the adjustment of cross-class similarity exceeding the threshold, maintaining the continuity of the feature space structure while suppressing semantic assimilation, and avoiding instability caused by excessive separation during optimization.

[0107] Step S5: Input the fused features into the classification branch and the regression branch respectively to obtain the detection results.

[0108] The fused features are input into the classification branch and the regression branch, respectively. The classification branch performs joint classification prediction of the base class and the new class for the candidate region, while the regression branch performs bounding box coordinate regression on the candidate region, thereby outputting the class labels and localization results of the base class and the new class targets in the remote sensing image. The system can accurately detect the new class targets using only a small number of new class samples, while maintaining the detection performance of the base class targets.

[0109] This invention employs a two-stage training strategy to achieve end-to-end model optimization:

[0110] like Figure 2 As shown, in the base class pre-training stage, the model is trained using only the base class dataset to obtain stable and scale-consistent multi-scale feature representations. Specifically, the backbone network ResNet101 and the adaptive feature pyramid network are optimized end-to-end by minimizing the overall detection loss. The optimization objectives include region proposal loss, classification loss, and bounding box regression loss.

[0111] ;

[0112] In the formula, To detect the loss; The region is advised to bear losses; Classification loss; The bounding box regression loss.

[0113] This stage provides a stable and transferable feature foundation for subsequent learning of new feature classes.

[0114] During the fine-tuning phase for new classes, the parameters of the backbone network, adaptive feature pyramid network, and base class feature extraction branch are frozen. The number of output nodes in the classification head of the classification branch is expanded from the number of base class categories to the sum of the number of base class categories and the number of new class categories to adapt to the joint class space composed of base and new classes. Parameter updates are performed only on the classification branch, regression branch, and cross-class feature adaptive fusion module. The model introduces a dual constraint mechanism consisting of knowledge distillation loss and mutual information constraints, by jointly adjusting the weight of the knowledge distillation loss. Mutual information constraint weights The model gradually enhances the discriminative power of new class features while maintaining the stability of the base class discriminative structure, thus achieving a controlled balance between the two class constraints during training. The total loss function in the new class fine-tuning stage is defined as:

[0115] ;

[0116] in, This is the total loss function; The region is advised to bear losses; For classification loss; For bounding box regression loss; and These are the loss weight coefficients for knowledge distillation loss and mutual information regularization loss, respectively, and their values ​​are non-negative real numbers. In this embodiment, and The values ​​are 0.01 and 0.25 respectively.

[0117] By jointly optimizing the above objective function, the model completes the transfer of knowledge to new classes and the formation of discriminative ability while maintaining the stability of base class features. This achieves the synergistic optimization of base class and new class features, thereby achieving the technical effect of controlled knowledge transfer.

[0118] To verify the effectiveness of the method proposed in this invention, experimental evaluations were conducted on two publicly available mainstream remote sensing datasets. The NWPU VHR-10.v2 dataset contains multi-class high-resolution remote sensing images, with target categories originating from various surface environments, including typical remote sensing scenes such as coastal urban built-up areas, transportation and engineering facilities, and their surrounding areas. Targets exhibit differences in scale and imaging angle. The DIOR dataset contains multi-source aerial remote sensing images, covering marine and coastal remote sensing scenes such as coastlines, port facilities, islands, reefs, and maritime activity targets, with complex backgrounds and diverse target morphologies. The base class and new class partitioning methods and K-shot settings in the experiments all followed publicly available standard experimental partitioning protocols in the field of generalized small-sample target detection. Specifically, partitioning methods 1 and 2 were used on the NWPU VHR-10.v2 dataset; and partitioning methods 1, 2, 3, and 4 were used on the DIOR dataset. All of the above partitioning methods are experimental configurations known to those skilled in the art and directly reproducible. In the experiment, FRW, Meta-RCNN, FsDetView, P-CNN, TFAw / cos, DeFRCN, G-FSDet, and GE-FSOD were selected as comparison methods. Among them, although the task definition of GE-FSOD is generalized small sample target detection, it only reports the detection results of new classes.

[0119] like Figure 6As shown, the comparison results between the embodiments of the present invention and existing methods under NWPU VHR-10.v2 dataset partition 1 and partition 2, with 3-shot, 5-shot, 10-shot, and 20-shot settings, include the mean accuracy (mAP) of the base class, new class, and all classes, respectively. As the K-shot setting increases from 3-shot to 20-shot, the detection accuracy of the new class and the detection accuracy of all classes in the embodiments of the present invention under partition 1 and partition 2 generally show an improving trend, while the detection performance of the base class remains relatively stable. This indicates that the proposed method can effectively enhance the discrimination ability of new class targets under the condition of increased sample size. Taking G-FSDet as an example, under partition 1, the accuracy of new class detection in the embodiments of the present invention is improved by 17.99%, 16.33%, 8.90%, and 5.01% under 3-shot, 5-shot, 10-shot, and 20-shot conditions, respectively, and the accuracy of detection for all classes is improved by 6.45%, 7.00%, 4.45%, and 2.59%, respectively. Under partition 2, the accuracy of new class detection is improved by 4.69%, 8.61%, 7.24%, and 4.02%, respectively, and the accuracy of detection for all classes is improved by 1.93%, 2.79%, 3.40%, and 1.45%, respectively. The above results show that the embodiments of the present invention can effectively improve the detection capability of new classes while maintaining stable base class detection performance.

[0120] like Figure 7 As shown, this embodiment of the invention compared with existing methods under 3-shot, 5-shot, 10-shot, and 20-shot settings in DIOR dataset partitions 1, 2, 3, and 4. The results include the mean average accuracy (mAP) of the base class, new class, and all classes. Experimental results show that under complex scenarios and large class differences, such as the DIOR dataset, this embodiment of the invention tends to maintain stable base class detection performance under extremely low sample conditions, while the improvement in new class detection capability gradually becomes apparent as the number of samples increases. Specifically, under the 3-shot condition, the base class detection performance remains at a high and stable level in all class partitions; while under the 10-shot and 20-shot conditions, the improvement in new class detection accuracy is more significant. Taking partition 1 as an example, under the 10-shot and 20-shot conditions, the new class detection accuracy of this embodiment of the invention reaches 38.16% and 40.76%, respectively, which are improvements of 0.70% and 0.93% compared to G-FSDet. The above results show that this embodiment of the invention can achieve an effective balance between base class preservation and new class enhancement in complex remote sensing scenarios.

[0121] like Figure 8As shown, the qualitative analysis results further verify the superiority of the method in the embodiments of the present invention. When dealing with bridges and ships with large aspect ratios, the method in the embodiments of the present invention can provide more accurate detection results. Furthermore, the method in the embodiments of the present invention can not only detect some targets missed in the annotation, but also maintain consistent category discrimination results, while existing methods suffer from category confusion in complex scenarios. In scenarios where base class targets and new class targets appear simultaneously, the method in the embodiments of the present invention, while maintaining stable base class detection performance, provides more accurate detection results for new class targets, fully demonstrating the applicability and effectiveness of the controlled knowledge transfer strategy in diverse remote sensing environments.

[0122] Based on the above method, this invention also provides a remote sensing image generalized small sample target detection system based on controlled knowledge transfer, which includes the following functional modules:

[0123] Multi-scale feature extraction module: In the lateral connection path of the feature pyramid network, multi-scale feature representations are extracted from remote sensing images through multi-scale dilated convolutional attention mechanism, generating a feature base that is robust to scale changes and semantically consistent.

[0124] Candidate region feature generation module: The multi-scale feature representation is processed by the region proposal network to generate candidate region features. The candidate region features are then input into the parameter-independent base class feature extraction branch and new class feature extraction branch to generate base class features and new class features.

[0125] Cross-class feature adaptive fusion module: performs two-level fusion processing on base class features and new class features. The first level adaptively adjusts the cross-class knowledge transfer amount based on cosine similarity, and the second level adaptively adjusts the feature fusion ratio based on normalized weights to generate fused features.

[0126] The dual regularization constraint module constrains the consistency between fused features and base class features through knowledge distillation loss, and reduces the redundant correlation between new class features and base class features through mutual information regularization loss, thereby achieving controlled knowledge transfer.

[0127] The target detection output module inputs the fused features into the classification branch and the regression branch respectively. The classification branch performs joint classification prediction of the base class and the new class for the candidate region, and the regression branch performs bounding box coordinate regression for the candidate region, outputting the class labels and localization results of the base class and the new class targets in the remote sensing image.

[0128] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; only preferred embodiments of the present invention are illustrated. The descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. As long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0129] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this invention should be determined by the appended claims.

Claims

1. A controlled knowledge transfer-based generalized small-sample target detection method for marine remote sensing, characterized in that, Includes the following steps: Step S1: Extract multi-scale features from the remote sensing image using a multi-scale feature extraction network to generate a multi-scale feature representation; Step S2: The multi-scale feature representation is processed by a region proposal network to generate candidate region features. The candidate region features are then input into a parameter-independent base class feature extraction branch and a new class feature extraction branch to generate base class features and new class features, respectively. Step S3: The base class features and the new class features are fused in two stages through the cross-class feature adaptive fusion module to generate fused features. The first stage adaptively adjusts the cross-class knowledge transfer amount based on cosine similarity, and the second stage adaptively adjusts the feature fusion ratio based on normalized weights. The adaptive adjustment of cross-category knowledge transfer based on cosine similarity includes the following steps: The base class features With the new class features The system is divided into G subspaces along the channel dimension. Within each subspace, the cosine similarity between the base class features and the new class features is calculated, resulting in a G-dimensional similarity vector. ; Based on the similarity vector The new class of features is weighted group by group at the channel group level, and a compensation coefficient 1 is constructed based on the similarity vector. The base class features are weighted group by group at the channel group level, and then the two weighted results are added together to obtain the transfer features. : ； In the formula, This is an element-wise multiplication operation; For the first The base class features corresponding to each candidate region; For the first New class features corresponding to each candidate region; For the first The candidate region in the first g Cosine similarity over each channel subspace; For the first Migration features corresponding to each candidate region; Within different channel subspaces, when the cosine similarity of the corresponding subspaces is low, the transfer contribution of the base class features in the current subspace increases; when the cosine similarity of the corresponding subspaces is high, the transfer contribution of the base class features in the current subspace decreases. The adaptive adjustment of the feature fusion ratio based on normalized weights includes the following steps: Independent linear transformations are applied to the base class features and the new class features respectively, and their respective response values ​​are generated by the Sigmoid activation function. The two response values ​​are normalized to obtain the base class fusion weight. Weighting for integration with new categories ; Based on the base class fusion weight and the new class fusion weight, the transfer feature and the new class feature are weighted and summed to generate the fusion feature: ； In the formula, For the first The fusion features corresponding to each candidate region; For the first The base class fusion weights corresponding to each candidate region; For the first New fusion weights corresponding to each candidate region; Step S4: Apply dual constraints to the fused feature using knowledge distillation loss and mutual information regularization loss. The knowledge distillation loss constrains the consistency between the fused feature and the base class feature, and the mutual information regularization loss reduces the redundant correlation between the new class feature and the base class feature. Step S5: Input the fused features into the classification branch and the regression branch respectively to obtain the detection results.

2. The method for controlled knowledge transfer-based generalized small-sample target detection in marine remote sensing according to claim 1, characterized in that: In step S1, a multi-scale feature extraction network is constructed by embedding a multi-scale dilated convolutional attention mechanism in the lateral connection paths of the feature pyramid network. The multi-scale dilated convolutional attention mechanism includes: The input feature map is projected by a 1×1 convolution and split along the channel dimension into a query tensor, a key tensor, and a value tensor. The query tensor, the key tensor, and the value tensor are each divided along the channel dimension. N Each subspace corresponds to a different void ratio, wherein the bond tensor is spatially sparsely sampled in each subspace through void convolution with different void ratios to obtain receptive fields of different scales. Within each subspace, attention responses are calculated based on scaled dot product attention, and the attention outputs of each subspace are weighted and fused using adaptive weights based on global average pooling to generate multi-scale attention features.

3. The method for controlled knowledge transfer-based generalized small-sample target detection in marine remote sensing according to claim 2, characterized in that: The multi-scale attention features are injected into the lateral connection path through gated residual connections: ； In the formula, Output feature map; The input feature map; The multi-scale attention features; The gating factor; This is an element-wise multiplication operation.

4. The method for controlled knowledge transfer-based generalized small-sample target detection in marine remote sensing according to claim 1, characterized in that: The mutual information regularization loss is defined as: ； In the formula, Define the mutual information regularization loss; The number of samples for the new class feature; The number of samples representing base class features; The cosine similarity function; This refers to temperature hyperparameters. For the first Feature vectors of new class feature samples; For the first Feature vectors of base class feature samples; The similarity threshold; When the cosine similarity between the new class feature and the base class feature exceeds the similarity threshold A gradient penalty is applied when the similarity falls below the threshold. It does not participate in gradient updates.

5. The method for controlled knowledge transfer-based generalized small-sample target detection in marine remote sensing according to claim 1, characterized in that: Steps S1 to S4 employ a two-stage training strategy, including: Base class pre-training stage: The backbone network and the multi-scale feature extraction network are trained end-to-end using the base class dataset. The optimization objectives include region proposal loss, classification loss and bounding box regression loss. New Class Fine-tuning Stage: The parameters of the backbone network, the multi-scale feature extraction network, and the base class feature extraction branch are frozen. The number of output nodes in the classification head of the classification branch is expanded from the number of base class categories to the sum of the number of base class categories and the number of new class categories. Parameter updates are performed only on the classification branch, the regression branch, and the cross-class feature adaptive fusion module. The total loss function is: ； in, This is the total loss function; The region is advised to bear the loss; For classification loss; For bounding box regression loss; For knowledge distillation loss; For mutual information regularization loss; and These are the loss weight coefficients for knowledge distillation loss and mutual information regularization loss, respectively.

6. The method for controlled knowledge transfer-based generalized small-sample target detection in marine remote sensing according to claim 5, characterized in that: The base class feature extraction branch and the new class feature extraction branch have the same structure and their weight parameters are independent of each other. Each feature extraction branch includes two cascaded fully connected layers. During the new class fine-tuning stage, the weight parameters of the base class feature extraction branch are kept frozen, and the base class features participate in the two-level fusion process as anchor reference features.

7. The method for controlled knowledge transfer-based generalized small-sample target detection in marine remote sensing according to claim 2, characterized in that: The adaptive weights based on global average pooling are generated in the following manner: Global average pooling is performed on the input feature map, followed by 1×1 convolution mapping and then the weight coefficients for each subspace are generated using the Softmax normalization function. The attention outputs of each subspace are then weighted and summed. ； in, For the first Attention output of each subspace; The total number of subspaces; For the first The weight coefficients corresponding to each subspace; This represents the multi-scale attention features after weighted fusion.

8. A controlled knowledge transfer-based generalized small-sample target detection system for marine remote sensing, implemented based on the controlled knowledge transfer-based generalized small-sample target detection method for marine remote sensing as described in any one of claims 1-7, characterized in that, include: Multi-scale feature extraction module: Extracts multi-scale feature representations from remote sensing images through a multi-scale dilated convolutional attention mechanism in the lateral connection path of the feature pyramid network; Candidate region feature generation module: The multi-scale feature representation is processed by a region proposal network to generate candidate region features, and the candidate region features are respectively input into a parameter-independent base class feature extraction branch and a new class feature extraction branch to generate base class features and new class features; Cross-category feature adaptive fusion module: performs two-level fusion processing on the base class features and the new class features. The first level adaptively adjusts the cross-category knowledge transfer amount based on cosine similarity, and the second level adaptively adjusts the feature fusion ratio based on normalized weights to generate fused features. Dual regularization constraint module: constrains the consistency between the fused feature and the base class feature through knowledge distillation loss, and reduces the redundant correlation between the new class feature and the base class feature through mutual information regularization loss; The target detection output module inputs the fused features into the classification branch and the regression branch respectively. The classification branch performs joint classification prediction of the base class and the new class for the candidate region, and the regression branch performs bounding box coordinate regression for the candidate region, outputting the class labels and localization results of the base class target and the new class target in the remote sensing image.

Images