A small sample target detection method based on attribute semantic alignment

Abstract

The application discloses a small sample target detection method based on attribute semantic alignment. The method comprises the following steps: acquiring an input image, and extracting visual features of a region of interest in the image by using a target detection backbone network; acquiring fine-grained attribute features generated by a large language model, and constructing a hypergraph structure with the attribute features as nodes and categories as hyperedges; generating optimized attribute representations by hypergraph propagation, and aggregating category prototypes of various categories to form attribute semantic manifolds and the like; the application combines fine-grained attribute priors generated by a large language model to construct a hypergraph network, designs a double-branch network of attribute semantic manifold modeling and attribute-guided background correction, effectively alleviates the semantic drift problem under extreme small sample conditions, and adaptively eliminates negative bias interference caused by unlabeled foreground targets, so as to enhance the detection robustness and classification accuracy of the model for new category targets.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a few-shot object detection method based on attribute semantic alignment. Background Technology

[0002] Object detection, a core task in computer vision, aims to accurately locate and classify objects of interest in images. In recent years, deep learning-based object detection technologies have demonstrated enormous application potential in fields such as autonomous driving, security monitoring, and medical image diagnosis. However, traditional object detection models typically rely on large-scale, high-quality labeled data for training. In many practical applications, acquiring massive amounts of labeled data often faces challenges such as high costs or limited data availability (e.g., rare species, rare diseases). Therefore, the Few-Shot Object Detection (FSOD) task has emerged, with the core objective of enabling models to quickly learn and identify new object categories using only a very small number of labeled samples (typically only 1 to 30 per class).

[0003] While existing few-shot object detection methods based on a two-stage fine-tuning paradigm have made some progress, classification ambiguity remains a core bottleneck limiting performance in extremely low-shot regimes. The first major challenge of existing technologies lies in the "semantic drift" problem. Due to the extreme scarcity of available visual samples, the model relies on only limited visual features to construct class representations for new classes during the fine-tuning phase. This representation based on scarce visual information often suffers from severe statistical bias, causing the feature representations of new target classes to easily shift towards semantically similar base classes. Furthermore, most existing methods lack effective external semantic guidance, making it difficult to distinguish fine-grained categories with similar appearance features in the absence of explicit structured relationship modeling.

[0004] The second major challenge facing existing technologies is the "negative bias" phenomenon. In real-world training environments for few-shot object detection, images often contain multiple potential targets, but typically only a very small number are assigned true labels. This means that a large number of unlabeled foreground objects, including both base and new class objects, inevitably exist in the background region of the image. Existing training strategies often treat these unlabeled foreground objects uniformly as background, resulting in erroneous negative gradients during optimization. These erroneous supervision signals distort the decision boundary, causing the model to tend to misclassify true foreground objects as background, further exacerbating the instability of class representation and severely impairing the model's detection robustness.

[0005] In summary, existing few-sample object detection techniques are still insufficient in addressing semantic drift caused by visual distribution bias and negative background bias caused by missing annotations, making it difficult to achieve high-precision classification and detection in extremely low-sample environments. Therefore, those skilled in the art urgently need to propose a more robust few-sample object detection method to effectively utilize multi-dimensional prior information to overcome the aforementioned technical bottlenecks. Summary of the Invention

[0006] In view of this, the purpose of this invention is to propose a few-shot target detection method based on attribute semantic alignment. This invention combines fine-grained attribute priors generated by a large language model to construct a hypergraph network, and designs a dual-branch network for attribute semantic manifold modeling and attribute-guided background correction. This effectively alleviates the semantic drift problem under extremely few-shot conditions and adaptively eliminates the negative bias interference caused by unlabeled foreground targets, thereby enhancing the model's robustness in detecting new target categories and its classification accuracy.

[0007] According to one aspect of the present invention, a few-shot target detection method based on attribute semantic alignment is provided, the method comprising: The input image is acquired, and the visual features of the region of interest in the image are extracted using an object detection backbone network. We acquire fine-grained attribute features generated by a large language model, construct a hypergraph structure with the attribute features as nodes and categories as hyperedges, generate optimized attribute representations through hypergraph propagation, and aggregate them to obtain category prototypes for each category, forming an attribute semantic manifold. The visual features of the region of interest are projected onto the semantic space to obtain semantic embedding features, and the semantic embedding features are then subjected to multiple visual-semantic alignment with the category prototypes and attribute representations in the attribute semantic manifold. During the model fine-tuning stage, the semantic embedding features of the region of interest initially identified as background are calculated and the semantic confidence between the category prototype of the new category is calculated. Based on the semantic confidence, the label of the background region of interest is adaptively corrected to a pseudo-label foreground, excluded sample, or reliable background to correct the background supervision signal. The network is jointly optimized by combining object detection loss and attribute semantic manifold loss.

[0008] The aforementioned technical solution aims to address the semantic drift problem caused by the extreme scarcity of visual samples in few-shot object detection, as well as the negative bias problem caused by unlabeled foreground objects being incorrectly used as background during training due to missing annotations. The solution proposes a detection method based on attribute semantic alignment. By introducing fine-grained attribute priors generated by a large language model, an attribute semantic manifold is constructed to guide multiple semantic alignments of visual features. Simultaneously, adaptive label correction is applied to background regions based on semantic confidence. Finally, robust few-shot object detection is achieved within a joint optimization framework.

[0009] The essential feature of this plan is that, First, a hypergraph structure with attributes as nodes and categories as hyperedges is constructed to organize fine-grained attribute features obtained from a large language model. Optimized attribute representations and category prototypes are generated through hypergraph propagation, forming an attribute semantic manifold. This design differs fundamentally from existing methods that rely solely on a few labeled samples to construct visual prototypes. The hypergraph propagation mechanism captures higher-order semantic relationships between nodes and hyperedges, achieving a balance between inter-class sharing and intra-class specialization in attribute representations. Simultaneously, the attribute priors provided by the large language model transform category representations from data-driven statistical estimations into knowledge-guided semantic modeling. The resulting effect is that even under extreme conditions with only 1 to 5 labeled samples per class, the category prototypes maintain a stable semantic structure, effectively suppressing semantic drift—the shift of new category representations towards semantically similar base classes—which cannot be achieved simply by increasing visual data augmentation or improving classifier boundaries.

[0010] Second, a multi-layered visual-semantic alignment mechanism was established, including cosine prototype alignment loss, prototype contrastive classification loss, and attribute-level alignment loss, projecting visual features onto an attribute-semantic manifold space for constraint. This mechanism is not a single-dimensional feature matching, but rather establishes alignment targets at three levels: "individual-prototype," "prototype-global," and "attribute-distribution." Prototype alignment encourages similar visual features to converge towards the correct attribute center of gravity, contrastive classification increases the discriminative power of different categories in the manifold space, and attribute-level alignment constrains the consistency of visual response distribution at a fine-grained semantic level. This hierarchical alignment strategy provides visual features with continuous standardization from structured semantic priors, overcoming the problem of insufficient alignment targets under a single loss function. It allows for relatively accurate semantic embeddings to be formed early in the fine-tuning process, laying a reliable foundation for subsequent background correction.

[0011] Third, a three-state background correction strategy based on semantic confidence is proposed. During the fine-tuning stage, the semantic confidence score between regions of interest initially identified as background and the prototype of the new category is calculated. Based on the relationship between this score and a preset threshold, the region is adaptively classified as pseudo-labeled foreground, excluded sample, or reliable background. The innovation of this design lies in its avoidance of the weighted penalty or hard threshold filtering strategies commonly used in existing methods. Instead, it introduces an intermediate state of "excluded sample," directly shielding semantically ambiguous high-suspicious regions from negative sample supervision, thus avoiding the distortion of the decision boundary by erroneous negative gradients. Simultaneously, semantically highly matched unlabeled regions are assigned pseudo-labels and converted into positive samples for training, achieving a progression from "eliminating interference" to "mining positive samples." This mechanism significantly eliminates the negative bias interference caused by unlabeled foreground targets without introducing additional model complexity, improving the model's detection robustness in environments with incomplete labeling.

[0012] In summary, the solution deeply integrates external language knowledge with visual learning, effectively mitigating the semantic drift problem under extremely small sample conditions, while adaptively eliminating the negative bias interference caused by unlabeled foreground targets.

[0013] In some embodiments, fine-grained attribute features generated by a large language model are obtained, and a hypergraph structure with the attribute features as nodes and categories as hyperedges is constructed. Optimized attribute representations are generated through hypergraph propagation, and category prototypes for each category are aggregated accordingly to form an attribute semantic manifold. Specifically, this includes: We use a large language model to generate fine-grained text attribute descriptions for each category, and then encode them into attribute embedding vectors using a text encoder. Cluster the attribute embedding vectors and use the cluster centers as compact initial attribute node features; An association matrix is ​​constructed based on the attribution relationship between attribute nodes and categories, and the hypergraph structure is constructed accordingly. By using a composite propagation operator that includes a hypergraph propagation matrix, an attribute node pairwise similarity matrix, and a self-connection matrix, the attribute node features are propagated and updated hierarchically to obtain the optimized attribute representation.

[0014] The aforementioned technical solution aims to address the problem that category prototype construction in small-sample object detection heavily relies on sparse visual samples, leading to representational bias and semantic drift. The solution proposes an attribute semantic manifold construction method based on hypergraph propagation. By introducing fine-grained attribute priors generated by a large language model, it organizes high-order semantic associations between attributes and categories using a hypergraph structure, and generates optimized attribute representations and category prototypes through a composite propagation mechanism.

[0015] The essential feature of this plan is that, First, a process of "large language model generation - text encoding - clustering compression" is adopted to obtain compact initial attribute node features. Existing methods often directly use category names or simple attribute words as semantic priors, resulting in coarse information granularity and a lack of balance between intra-class diversity and inter-class distinguishability. This approach first uses a large language model to generate multiple fine-grained text attribute descriptions for each category, producing rich semantic materials covering dimensions such as visual appearance, structure, and scene. Then, a text encoder maps these descriptions to a high-dimensional semantic space, and K-means clustering is performed using cosine similarity as a metric, with the cluster centers serving as compact attribute nodes. This process is not a simple attribute collection, but rather an adaptive compression of semantic redundancy through clustering. This ensures that each attribute node maintains its semantic integrity while avoiding feature space expansion and computational redundancy caused by a large number of similar descriptions, providing a concise and information-dense node foundation for subsequent graph structure modeling.

[0016] Second, a hypergraph structure with attributes as nodes and categories as hyperedges was constructed, and a composite propagation operator was designed, comprising a hypergraph propagation matrix, an attribute-node pairwise similarity matrix, and a self-connection matrix. Ordinary graph structures can only model pairwise associations between attributes, while the hypergraph treats each category as a hyperedge that can simultaneously connect multiple attribute nodes, aligning with the semantic essence that "a category is defined by multiple attributes." Based on the node degree matrix and hyperedge degree matrix derived from the attribute-category association matrix, the constructed hypergraph propagation matrix can capture the higher-order semantic propagation path of "node-hyperedge-node," allowing attribute updates to converge information from other attributes of the same category via category hyperedges. Furthermore, the composite propagation operator integrates the direct semantic associations encoded by the attribute pairwise similarity matrix and the node-specific characteristics preserved by the self-connection matrix, ensuring that the propagation process considers both higher-order structural constraints and lower-level semantic similarity and node independence. The resulting effect is that the attribute representation after hierarchical propagation and updating not only strengthens the semantic consistency between similar attributes, but also retains the necessary distinguishability between different categories of attributes. Finally, the category prototype generated by weighted aggregation has stronger semantic stability and discriminative power, effectively alleviating the semantic drift problem caused by relying on limited visual features to construct prototypes under extremely small sample conditions.

[0017] In summary, the design of the composite propagation operator enables attribute learning to benefit simultaneously from the constraints of high-order hypergraph structures, pairwise semantic similarity guidance, and self-connectivity preservation. The resulting category prototypes have significant advantages in semantic stability and category discriminability.

[0018] In some embodiments, after obtaining the optimized attribute representation, the method further includes: A residual mapping network is introduced to adaptively adjust the visual distribution of the final layer attribute features after hierarchical propagation, generating the final optimized attribute representation; Based on the association matrix, the final optimized attribute representation is weighted and aggregated to generate the category prototypes for all categories.

[0019] The above technical solution aims to address the potential distributional discrepancies between the attribute representations generated by hypergraph propagation and the visual feature space, and to effectively aggregate discriminative category prototypes from the optimized attribute representations. After hierarchical hypergraph propagation, the solution introduces a residual mapping network to adaptively adjust the visual distribution of attribute features, and generates category prototypes through degree-normalized weighted aggregation based on the association matrix.

[0020] The key feature of this scheme lies in introducing a residual mapping network for adaptive adjustment of visual distribution, based on the semantic-level attribute representation generated by hypergraph propagation. The hypergraph propagation process primarily optimizes attribute features within the textual semantic space, and its output distribution may deviate from the subsequent visual embedding space. Directly using this attribute representation for visual-semantic alignment would increase the difficulty of alignment due to the distributional differences between the two modalities, weakening the effectiveness of semantic supervision. This scheme introduces a lightweight mapping network using residual connections. Its forward computation involves adding its own nonlinear transformation to the final layer attribute features, followed by normalization. The innovations of this design are: firstly, the residual structure uses an identity mapping as its basis, ensuring that the semantic structure learned through hypergraph propagation is not destroyed during adjustment, only adding the small offset required for visual distribution adaptation; secondly, this separation of "semantic preservation and distribution adaptation" allows the attribute representation to retain the rich semantic information provided by the large language model while conforming to the statistical characteristics of visual feature distribution. The resulting effect is that the adjusted attribute representation is in a more compatible feature subspace with the visual embedded features, which reduces the difficulty of optimizing subsequent loss functions such as cosine alignment and contrastive classification, and provides a more suitable semantic anchor for visual-semantic alignment.

[0021] In generating category prototypes, this scheme uses a degree normalization approach to weighted aggregate the final optimized attribute representations based on the attribute-category association matrix. This step involves weighting and normalizing all attribute representations associated with each category by the inverse of their hyperedge degree, making the category prototype the semantic centroid of its associated attribute nodes. Compared to directly averaging attributes or using attention weighting, the degree normalization strategy considers the unequal number of attributes across different categories, preventing categories with more attributes from dominating the norm or distribution of their prototypes, and ensuring comparable representational strength across all categories in the manifold space. This balanced prototype construction method allows even small-sample new categories with a limited number of associated attributes to obtain robust prototype representations matching their semantic richness, preventing representation collapse even with very few labeled samples.

[0022] In summary, the introduction of residual mapping networks bridges the gap in modality distribution while maintaining semantic integrity, and degree-normalized weighted aggregation generates balanced and discriminative category prototypes for all categories.

[0023] In some embodiments, the multiple visual-semantic alignment is achieved by combining at least two of the following attribute semantic manifold losses: Cosine prototype alignment loss is used to narrow the distance between the semantic embedding feature and the category prototype of its corresponding target category in the manifold space. Prototype contrastive classification loss is used to increase the distance between the semantic embedding features and the class prototypes of non-target categories, thereby enhancing global discriminability; The attribute-level alignment loss is used to constrain the response distribution of the semantic embedding features on fine-grained attribute nodes, so that it is consistent with the true attribute distribution of the target category.

[0024] The aforementioned technical solution aims to address the issues of insufficient alignment between visual features and category semantics in small-sample object detection, and the difficulty of simultaneously achieving intra-class compactness and inter-class discriminability with a single constraint. The solution proposes a multi-visual-semantic alignment mechanism, which applies hierarchical constraints to visual embedding features within the attribute-semantic manifold space by combining cosine prototype alignment loss, prototype contrastive classification loss, and attribute-level alignment loss.

[0025] The key feature of this approach lies in its construction of a composite alignment loss system encompassing three levels of constraints. Existing methods typically employ only a single-dimensional semantic alignment strategy, such as requiring visual features to be close to the category text embedding or performing only category-level metric learning, lacking a coordinated specification of feature distribution across the three levels of global structure, individual localization, and fine-grained semantics. This approach decomposes visual-semantic alignment into three complementary levels: Cosine prototype alignment loss operates between individual samples and their target category prototypes, minimizing the cosine distance to guide visual features toward the correct semantic center of gravity, addressing the intra-class compactness problem; Prototype contrastive classification loss operates on the global relationship between a single sample and all category prototypes, incorporating the similarity between the semantic embedding and the entire prototype matrix into cross-entropy optimization after temperature scaling, forcing the model to actively reject non-target category prototypes while distinguishing the target category, addressing the inter-class discriminability problem; Attribute-level alignment loss operates at a finer-grained attribute node level, constraining the response distribution of the semantic embedding on all attribute nodes with the true attribute distribution of the target category using KL divergence, ensuring that visual features are not only correct in "which category they belong to," but also consistent in their attribute interpretation of "why they belong to that category." The resulting effect is that the three losses complement each other in terms of optimization objectives: prototype alignment provides a direct attraction signal, contrastive classification provides a global repulsion signal, and attribute-level alignment provides a fine-grained distribution normalization signal. Their synergistic effect ensures that visual embedding features, within the attribute semantic manifold, not only closely surround the correct category prototype but also maintain clear semantic boundaries with other categories. Furthermore, they possess interpretable response patterns along the attribute dimension, effectively mitigating the problems of feature representation ambiguity and category confusion caused by the scarcity of visual samples.

[0026] Furthermore, all three losses can be calculated based on a unified mapping attribute prototype and mapping attribute nodes, sharing the same set of semantic anchor parameters, thus avoiding the anchor inconsistency problem commonly found in multi-objective optimization. The introduction of hyperparameter weight coefficients allows for flexible adjustment of the contribution ratio of each loss term, adapting to different dataset characteristics and new category semantic distributions, thereby improving the method's versatility and deployment flexibility.

[0027] In summary, this technical solution overcomes the limitations of a single loss function in terms of constraint dimensions, enabling visual features to obtain structured and highly interpretable semantic anchoring within the attribute semantic manifold.

[0028] In some embodiments, the labels of the background region of interest are adaptively corrected based on the semantic confidence level, and the correction decision rules specifically include: When the highest score of the semantic confidence is greater than the preset pseudo-label threshold, and the difference between the highest score and the second highest score is greater than the preset boundary threshold, the label of the region is corrected to the corresponding new category pseudo-label. When the highest score of the semantic confidence is between the preset pseudo-label threshold and the preset mask threshold, and the difference condition is met, the region is marked as an excluded sample so that it does not participate in the calculation of negative sample loss. In other cases, the original background label of the area is retained.

[0029] The aforementioned technical solution aims to address the problem of negative bias interference caused by unlabeled foreground objects being incorrectly used as background during the fine-tuning stage of small-sample object detection. The solution proposes a three-state background correction strategy based on semantic confidence. By evaluating the semantic similarity between the region of interest initially identified as background and the prototype of the new category, it adaptively classifies the region as pseudo-labeled foreground, excluded sample, or reliable background based on a preset threshold.

[0030] The key feature of this scheme lies in its dual-judgment mechanism, which includes both a highest score threshold and a boundary difference threshold, and the introduction of an intermediate state of "excluded samples," forming a three-state routing rule. Existing methods typically employ two strategies when dealing with unlabeled foreground targets in the background region: one is to uniformly apply a small loss weight to all background regions, and the other is to directly convert high-confidence regions into pseudo-labels based on a single threshold. The former cannot distinguish between the true background and potential missed targets, and erroneous negative gradients will continue to interfere with training; the latter, when the threshold is set too high, misses a large number of usable positive samples, and when it is set too low, may introduce noisy pseudo-labels.

[0031] This scheme first introduces a boundary difference constraint, requiring that the difference between the highest and second-highest similarity scores be greater than a preset boundary threshold. The essence of this constraint is to ensure that semantic decisions have a clear direction; subsequent corrections are only triggered when the semantically embedded features significantly favor a specific new category rather than being ambiguous across multiple categories. This design effectively filters out ambiguous areas with unclear semantic features, avoiding erroneous corrections caused by features being at category boundaries.

[0032] Building upon this, the scheme uses the highest similarity score as the primary criterion, setting pseudo-label thresholds and background mask thresholds. When the highest score exceeds the pseudo-label threshold and meets the boundary constraints, it indicates that the semantic matching between this region and a new category has high confidence and high directionality. This region is then identified as a missed target in the new category, assigned a corresponding pseudo-label, and converted into a positive sample for training. This operation not only eliminates erroneous negative gradients in this region but also transforms them into valuable positive supervision signals, effectively expanding training resources even under extreme conditions where only 1 to 5 labeled samples are available per category.

[0033] When the highest score falls between the pseudo-label threshold and the background mask threshold and satisfies the boundary constraints, it indicates that the region has some foreground semantic suspicion but is not yet certain. If it is treated as a positive sample, it may introduce noise; if it is treated as a negative sample, it will generate erroneous gradients. This scheme marks it as an "excluded sample" and directly masks it in subsequent negative sample supervision calculations. The introduction of this intermediate state is a key innovation that distinguishes it from existing binarization processing. It cuts off the interference path of semantically ambiguous regions on the optimization of decision boundaries with a "not judging if there is suspicion" strategy, and avoids the distortion of the classifier boundary by erroneous negative gradients.

[0034] In other cases, i.e. when the highest score is below the background mask threshold or does not meet the boundary constraints, the semantic association between the region and the new category is extremely low. Its original background label is retained as a reliable negative sample to participate in training normally, ensuring that the real background region continues to provide effective negative supervision.

[0035] The overall effect is that the background supervision signal undergoes a classification and purification process of "spurious label positive sample mining - fuzzy sample interference shielding - reliable negative sample retention", which systematically alleviates the negative bias problem caused by unlabeled foreground targets without introducing additional network branches or complex penalty terms.

[0036] In summary, this technical solution, through a progressive logic of "semantic confidence assessment - boundary orientation verification - classification decision," achieves adaptive purification and positive mining of background supervision signals without increasing model complexity.

[0037] In some embodiments, before projecting the visual features of the region of interest onto a semantic space to obtain semantic embedding features, and performing multiple visual-semantic alignment of the semantic embedding features with the category prototypes and attribute representations in the attribute semantic manifold, the method further includes: The visual features of the region of interest are subjected to gradient decoupling processing, so that the gradient from the attribute semantic manifold loss is back propagated to the target detection backbone network with a preset attenuation coefficient during backpropagation.

[0038] The above technical solution aims to address the problem of training instability caused by mutual interference of multi-objective optimization gradients when the semantic alignment branch and the visual detection branch share backbone features in a dual-branch few-sample object detection network. The solution introduces gradient decoupling processing before the visual features enter the semantic projection and alignment stages, performing attenuation control on the gradient from the attribute semantic manifold loss before backpropagating it to the backbone network.

[0039] The key feature of this scheme lies in setting up a gradient decoupling layer along the path from visual features to the semantic alignment branch. This layer performs an identity mapping during forward propagation, ensuring that visual features are transferred to subsequent projection and alignment modules without loss. During backpropagation, the gradient generated by the attribute semantic manifold loss is multiplied by a preset attenuation coefficient before being backpropagated to the backbone network. This design is fundamentally different from existing dual-branch joint optimization methods: existing methods either directly superimpose the gradients of multiple loss paths to update the backbone parameters, or completely block the backpropagation of gradients from the semantic branch to the backbone. The former is prone to gradient conflicts during the fine-tuning stage due to the inconsistency between the optimization directions of the semantic alignment target and the detection target, resulting in oscillations in detection accuracy or degradation of semantic representation; the latter prevents the backbone features from benefiting from semantic supervision, severing the possibility of synergy between visual and semantic learning.

[0040] This scheme establishes a continuous and controllable transition range between "complete blocking" and "direct superposition" using a gradient decay coefficient. The range of this coefficient allows for adjustable intervention of the semantic alignment target in the backbone feature extraction process: the identity mapping during forward propagation ensures that the semantic branch always receives complete visual information input without affecting alignment quality; the decay during backpropagation limits the influence of the semantic gradient on backbone parameter updates, allowing it to participate in feature learning in an auxiliary and guiding manner, rather than dominating the optimization direction. The resulting effect is that the detection loss directly determines the discriminative feature extraction direction of the backbone, while the semantic alignment loss applies a gentle semantic space constraint on this direction. The two form a synergistic relationship of "detection-dominated—semantic fine-tuning," which avoids the degradation of localization accuracy caused by semantic target interference in the main detection task, and allows backbone features to gradually acquire structured semantic space attributes.

[0041] In summary, this technical solution effectively suppresses gradient conflicts in multi-objective optimization while maintaining semantically aligned complete input.

[0042] In some embodiments, projecting the visual features of the region of interest into a semantic space specifically includes: A learnable query projection matrix and a learnable key projection matrix are introduced to map the visual features and prior information in the attribute semantic manifold to a unified alignment subspace to enhance the matching degree between vision and semantics.

[0043] The above technical solution aims to address the problem of poor direct alignment between visual features and attribute semantic manifolds due to modal differences. The solution introduces learnable query projection matrices and key projection matrices during the semantic projection stage, mapping the prior visual features and attribute manifolds to a unified alignment subspace before performing similarity calculations and loss constraints.

[0044] The key feature of this approach lies in the inclusion of a pair of learnable query-key projection matrices, both initialized as identity matrices, on the interactive interface between visual embedding features and attribute semantic manifolds. Existing methods typically calculate similarity between visual features and text embeddings directly in the same space, or use only fixed linear transformations for dimensional alignment. This rigid mapping ignores the fundamental differences between visual encoders and text encoders in terms of training data, optimization objectives, and feature distributions. Two vectors may reside in subspaces with the same dimensions but vastly different semantic structures, and directly calculating cosine similarity or inner product will introduce systematic bias.

[0045] This scheme initializes the query projection matrix and key projection matrix with an identity matrix, which are applied to the prior information in the visual embedded features and attribute manifold, respectively. The advantages are twofold: First, the necessity of the dual-matrix separation design lies in the fact that visual features originate from the layer-by-layer abstraction of pixel information by convolutional neural networks, while attribute priors originate from the semantic representation mapped by a large language model through a text encoder. The feature distributions of these two types of features have different statistical properties and geometric structures. Forcing alignment using the same projection matrix is ​​tantamount to requiring two heterogeneous features to satisfy the same linear transformation law, limiting matching flexibility. Separating the query and key matrices allows the model to learn suitable projection directions for the visual and semantic components respectively, finding the optimal cross-modal matching path while maintaining the internal structure of each modality. Second, the significance of the identity matrix initialization strategy lies in the fact that the model, at the beginning of training, is equivalent to aligning in the original semantic space, and the learning of the projection matrix is ​​a gradual fine-tuning based on this. This design ensures that the rich semantic structure contained in the pre-trained text encoder is not destroyed by random transformations in the early stages of projection. Prior information such as the relative positional relationships between attribute nodes and the semantic distance between category prototypes is fully preserved. The projection space only learns the necessary rotations and scaling required for modality adaptation based on this.

[0046] The resulting effect is that the alignment subspace inherits the prior structure of the text semantic space and integrates the statistical characteristics of the visual feature distribution. The similarity calculation of semantic embedding features and mapping category prototypes and mapping attribute nodes in this space more accurately reflects the substantial matching relationship between visual content and semantic concepts, providing a reliable measurement basis for subsequent multiple alignment loss.

[0047] In summary, this technical solution bridges the differences in cross-modal distribution with a simple parametric design, and establishes a unified and flexible metric space for visual-semantic alignment.

[0048] According to another aspect of the present invention, a few-sample target detection device based on attribute semantic alignment is provided, comprising: The feature extraction unit is used to acquire the input image and extract the visual features of the region of interest in the image using the object detection backbone network; The attribute semantic manifold construction unit is used to obtain fine-grained attribute features generated by a large language model, and construct a hypergraph structure with the attribute features as nodes and categories as hyperedges. Optimized attribute representations are generated through hypergraph propagation, and category prototypes of each category are aggregated accordingly to form an attribute semantic manifold. The visual-semantic alignment unit is used to project the visual features of the region of interest onto the semantic space to obtain semantic embedding features, and to perform multiple visual-semantic alignments of the semantic embedding features with the category prototypes and attribute representations in the attribute semantic manifold. The background correction unit is used in the model fine-tuning stage to calculate the semantic confidence between the semantic embedding features of the region of interest initially identified as background and the category prototype of the new category, and adaptively correct the label of the background region of interest to pseudo-label foreground, excluded sample or reliable background based on the semantic confidence, so as to correct the background supervision signal. The joint optimization unit is used to combine the object detection loss and the attribute semantic manifold loss to jointly optimize the network.

[0049] In order to better utilize the above method, this application proposes a small sample target detection device based on attribute semantic alignment. Each module corresponds to a step of the above method, and its specific principle has been described above and will not be repeated here.

[0050] According to another aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the above-described few-sample target detection method based on attribute semantic alignment.

[0051] In the above technical solution, to better operate and process the method, the method is stored in memory, and the processor executes the stored method. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here.

[0052] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that, when executed by a processor, the program implements the few-sample target detection method based on attribute semantic alignment as described in any one of claims 1 to 7.

[0053] In the above technical solution, to better operate and use the method, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating an embodiment of a few-sample target detection method based on attribute semantic alignment according to the present invention. Figure 2 This is a diagram of the entire network framework of an embodiment of a few-sample target detection method based on attribute semantic alignment according to the present invention. Figure 3 This is a framework diagram of the attribute semantic manifold construction module of an embodiment of a few-sample target detection device based on attribute semantic alignment according to the present invention; Figure 4 This is a framework diagram of an attribute-guided background correction module according to an embodiment of a few-sample target detection device based on attribute semantic alignment of the present invention; Figure 5 This is a framework diagram of an embodiment of a few-sample target detection device based on attribute semantic alignment according to the present invention. Detailed Implementation

[0056] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Example 1 Please see Figure 1 , Figure 2 A few-shot object detection method based on attribute semantic alignment, the method includes: S1. Acquire the input image and use the object detection backbone network to extract the visual features of the region of interest in the image; For example, an object detection dataset containing base classes and new categories is obtained, and the images are input into an object detection backbone network with a region proposal network to extract visual features of regions of interest (RoIs). In this example, S1 specifically includes: S11. Obtain the collection containing base classes. base class training set and collections containing new categories New type of training set ;in, It contains rich bounding box annotations. It contains only K labeled instances for each new category, used to perform two-stage few-shot detection training.

[0058] The specific details are as follows: (1) Dataset Selection and Category Composition: The training set used in this embodiment is derived from the training and validation sets of PASCAL VOC 2007 and VOC 2012, which covers 20 object categories in natural scenes. These categories are divided into four main categories: vehicles (e.g., airplanes, bicycles, boats, buses, cars, motorcycles, trains), animals (e.g., birds, cats, cows, dogs, horses, sheep), indoor objects (e.g., bottles, chairs, dining tables, potted plants, sofas, television monitors), and people. In the few-sample object detection task, these 20 categories are divided into a base class set containing 15 categories and a test new class set containing 5 categories.

[0059] (2) Small sample splitting scheme: To comprehensively evaluate the robustness of the model, three standard random splitting schemes (Split 1, 2, 3) were adopted. In each scheme, the five selected new classes were different (for example, in Split 1, birds, buses, cows, motorcycles, and sofas were selected as new classes), while the remaining 15 classes were used as base classes. This multi-split evaluation method aims to verify whether the model can achieve stable performance leaps when facing new class targets with different semantic features.

[0060] (3) Two-stage training paradigm and sample setup: During the base class pre-training phase: using the base class training set The training process involves using a set of objects containing rich bounding box location information (bounding box annotations) and category annotations, enabling the model to learn transferable detection representations such as general object shapes and textures. In the fine-tuning phase of the new class: on the training set of the new class The above is used to simulate extreme data-limited scenarios, where the model can only acquire a very small number of data points for each new category. This involves labeled instances. The core of this stage is to enable the model to quickly adapt to new class semantics through extremely low sample supervision.

[0061] (4) Data preprocessing and feature extraction: All input images are standardized and scaled to a uniform pixel size.

[0062] S12. Input the input image into the pre-trained deep residual network backbone for feature extraction to obtain the global feature map; S13. Generate a series of potential target candidate regions on the global feature map using a Region Proposal Network (RPN); S14. Using Region of Interest (RoI) Pooling and RoI Align operators, each candidate target region is cropped from the feature map and mapped to a fixed-dimensional vector to obtain the visual features of the region of interest. .

[0063] S2. Obtain fine-grained attribute features generated by the large language model, and construct a hypergraph structure with the attribute features as nodes and categories as hyperedges. Generate optimized attribute representations through hypergraph propagation, and aggregate them to obtain category prototypes for each category, forming an attribute semantic manifold. In this embodiment, S2 specifically includes: S21. Use a large language model to generate fine-grained text attribute descriptions for each category, and encode them into attribute embedding vectors using a text encoder. S22. Cluster the attribute embedding vectors and use the cluster centers as compact initial attribute node features; S23. Construct an association matrix based on the attribution relationship between attribute nodes and categories, and construct the hypergraph structure accordingly; S24. Using a composite propagation operator that includes a hypergraph propagation matrix, an attribute node pairwise similarity matrix, and a self-connection matrix, the attribute node features are propagated and updated hierarchically to obtain the optimized attribute representation.

[0064] In this embodiment, after obtaining the optimized attribute representation, the method further includes: S25. Introduce a residual mapping network to adaptively adjust the visual distribution of the final layer attribute features after hierarchical propagation, and generate the final optimized attribute representation. S26. Based on the association matrix, perform weighted aggregation on the final optimized attribute representation to generate the category prototypes for all categories.

[0065] For example, please refer to Figure 3 We construct an attribute semantic manifold, use a large language model and text encoder to generate fine-grained attribute features for each category, use the attribute features as nodes and categories as hyperedges to construct a hypergraph structure, and obtain optimized attribute representations and category prototypes through hypergraph propagation. In this example, S2 specifically includes: S21. Merge all foreground and background categories to form a total category set. For each category in the total category set, use a pre-set prompt template to query the large language model to generate multiple fine-grained text attribute descriptions of the category. Then, use the pre-trained CLIP text encoder to encode each text attribute description into a high-dimensional attribute embedding vector. S22. To eliminate semantic redundancy in the generated attribute embedding vectors, K-means clustering algorithm is used to cluster the normalized attribute embedding vectors based on cosine similarity as a distance metric. The cluster centers are then used as compact attribute features to construct the initial attribute matrix.

[0066] in Indicates the number of attributes. The feature dimensions of the attributes are represented; after CLIP encoding and K-means clustering of the text attribute descriptions generated using the large language model, the initial attribute matrix is ​​constructed with the cluster centers.

[0067] S23. Construct an attribute-category association matrix based on the clustering results and their corresponding category affiliations. Among them, when the first The attribute belongs to the first When describing each category, ,otherwise ,in Corresponding background category; S24. Treat each attribute as a node and each category as a hyperedge, and construct a hypergraph structure based on the association matrix H; calculate the attribute node degree matrix respectively. and category hyperedge degree matrix Both are diagonal matrices, and their diagonal elements are respectively and Based on the degree matrix and the correlation matrix, construct the hypergraph propagation matrix.

[0068] To capture higher-order semantic relationships between nodes and hyperedges; S25. Calculate the initial attribute matrix. Cosine similarity matrix between attributes This is used to encode direct pairwise semantic relationships between attributes; the propagation operator is obtained by combining the higher-order hypergraph propagation matrix, the pairwise attribute similarity matrix, and the self-connect.

[0069] in, It is the identity matrix. The scaling factor. Indicates row level Normalization processing; S26. Using the propagation operator Hierarchical message propagation and updating of attribute features, the first The update formula for the layer is:

[0070] in, For the first The attribute feature matrix of the layer, It is a multilayer perceptron consisting of two linear layers and the ReLU activation function. Indicates row level Normalization processing; S27. After multi-layer hypergraph propagation, a residual mapping network is introduced. Attributes of the final layer The formula for adaptive adjustment of visual distribution is as follows:

[0071] in, For the final layer Individual attribute features, For residual mapping networks 。 Obtain the optimized attribute representation matrix ; S28, Based on the association matrix The optimized attribute representations are weighted and aggregated to generate a category prototype matrix for all categories. The calculation formula is:

[0072] in, Optimized attribute representation matrix 。

[0073] S3. Project the visual features of the region of interest onto the semantic space to obtain semantic embedding features, and perform multiple visual-semantic alignment between the semantic embedding features and the category prototypes and attribute representations in the attribute semantic manifold. In this embodiment, the multiple visual-semantic alignment is achieved by combining at least two of the following attribute semantic manifold losses: Cosine prototype alignment loss is used to narrow the distance between the semantic embedding feature and the category prototype of its corresponding target category in the manifold space. Prototype contrastive classification loss is used to increase the distance between the semantic embedding features and the class prototypes of non-target categories, thereby enhancing global discriminability; The attribute-level alignment loss is used to constrain the response distribution of the semantic embedding features on fine-grained attribute nodes, so that it is consistent with the true attribute distribution of the target category.

[0074] In this embodiment, before S3, the following steps are also included: The visual features of the region of interest are subjected to gradient decoupling processing, so that the gradient from the attribute semantic manifold loss is back propagated to the target detection backbone network with a preset attenuation coefficient during backpropagation.

[0075] In this embodiment, projecting the visual features of the region of interest into the semantic space specifically includes: A learnable query projection matrix and a learnable key projection matrix are introduced to map the visual features and prior information in the attribute semantic manifold to a unified alignment subspace to enhance the matching degree between vision and semantics.

[0076] For example, visual-semantic alignment is performed by projecting the visual features of the region of interest into the semantic space to obtain semantic embedding features, and then aligning the semantic embedding features with the category prototypes and attribute representations in the attribute semantic manifold through a learning objective; In this example, S3 specifically includes: S31. Gradient decoupling processing, extracting visual features of the region of interest. Gradient flow control is achieved through a gradient decoupling layer (GDL); the gradient decoupling layer performs an identity mapping during forward propagation to maintain the feature flow direction unchanged, and performs a pre-set attenuation coefficient during backward propagation. Gradients from attribute branches are scaled and decayed to reduce the interference of semantic alignment process on the stability of the target detection backbone network.

[0077] S32. Preliminary semantic projection: The visual features after gradient decoupling are input into the attribute head module, utilizing a lightweight multilayer perceptron. Nonlinear mapping networks, combined with layer normalization (LayerNorm) and row-level... Normalization (Norm) projects visual features onto a dimension of 1. In the semantic space, preliminary semantic embedding features are generated. The calculation formula is as follows:

[0078] in, For the first Visual features of a region of interest It is a lightweight multilayer perceptron nonlinear mapping network.

[0079] S33. Mapping Space Construction: Introducing a Learnable Query Projection Matrix Bond projection matrix The projection matrices are all initialized as identity matrices; the preliminary semantic embedding features and attribute manifold priors are mapped to a unified alignment subspace through the projection matrices, so as to enhance the model's ability to model the matching relationship between visual components and semantic components.

[0080] S34. Alignment feature generation, using the query projection matrix. Preliminary semantic embedding features Perform linear transformation and execute Normalization yields aligned semantic embedding features. Simultaneously, utilizing the key projection matrix Each attribute node in the optimized attribute representation matrix is ​​represented separately. and the category prototypes in the category prototype matrix Perform linear transformation and Normalization process yields the mapped attribute nodes. With mapping category prototype .

[0081] S35. Calculate the cosine prototype alignment loss: For a set of region-of-interest samples with true labels or pseudo-labels generated after background correction. Calculate the semantic embedding features after alignment for each sample. Its corresponding target label Mapping category prototype The cosine distance between the samples is used to calculate the cosine prototype alignment loss over all samples. To encourage visual features of the same category to converge toward a pre-defined attribute centroid in the manifold space, the calculation formula is as follows:

[0082] in, This is a set of region-of-interest samples with real labels or pseudo-labels generated by background correction. For aligned semantic embedding features, For target label The mapping category prototype.

[0083] S36. Calculate the prototype contrastive classification loss: embed the aligned semantic features... With the matrix consisting of prototypes of all mapping categories Multiplying by the transpose, by the temperature coefficient After scaling, relative to the target label Combine calculation of cross-entropy loss The prototype-contrast classification loss is obtained by averaging all samples. To enhance the global distinguishability between different categories in the manifold space, the calculation formula is as follows:

[0084] in, This is a set of region-of-interest samples with real labels or pseudo-labels generated by background correction. For aligned semantic embedding features, The matrix consisting of prototypes of all mapping categories. For temperature coefficient, This is the cross-entropy loss function.

[0085] S37. Calculate attribute-level alignment loss: Calculate the aligned semantic embedding features. Mapping attribute nodes Inner product similarity Combined with temperature coefficient The predicted attribute distribution is obtained by normalization using the Softmax function. Based on the attribute-category association matrix during the hypergraph construction phase. Calculate the target attribute distribution corresponding to the target label The attribute-level alignment loss is obtained by calculating the KL divergence between the target attribute distribution and the predicted attribute distribution. To constrain the response distribution of visual features at the fine-grained attribute level, the calculation formula is as follows:

[0086] in, This is a set of region-of-interest samples with real labels or pseudo-labels generated by background correction. The distribution of target attributes corresponding to the target label is determined by the attribute-category association matrix; Let KL divergence be the KL divergence. To predict the distribution of attributes.

[0087] S38. Constructing the total loss of the attribute semantic manifold: Introducing hyperparameter weight coefficients. , and The cosine prototype alignment loss, prototype contrast classification loss, and attribute-level alignment loss are weighted and summed to form the attribute semantic manifold loss used to guide visual-semantic alignment. The calculation formula is:

[0088] S4. In the model fine-tuning stage, calculate the semantic confidence between the semantic embedding features of the region of interest initially identified as background and the category prototype of the new category, and based on the semantic confidence, adaptively correct the label of the background region of interest to pseudo-label foreground, excluded sample or reliable background to correct the background supervision signal. In this embodiment, the labels of the background region of interest are adaptively corrected based on the semantic confidence, and the correction decision rule specifically includes: When the highest score of the semantic confidence is greater than the preset pseudo-label threshold, and the difference between the highest score and the second highest score is greater than the preset boundary threshold, the label of the region is corrected to the corresponding new category pseudo-label. When the highest score of the semantic confidence is between the preset pseudo-label threshold and the preset mask threshold, and the difference condition is met, the region is marked as an excluded sample so that it does not participate in the calculation of negative sample loss. In other cases, the original background label of the area is retained.

[0089] For example, please refer to Figure 4 The design attribute-guided background correction module performs attribute-guided background correction. During the model fine-tuning stage, it calculates the confidence score between the semantic embedding features of the region of interest (ROI) determined to be the background and the category prototype of the new category. Based on the confidence score, the ROI is divided into pseudo-label foreground samples, excluded samples, or reliable background samples to correct the background supervision signal. In this example, S4 specifically includes: S41. For the set of regions of interest that are identified as background based on real-world annotations during the model fine-tuning stage. Calculate each region of interest in the set Aligned semantic embedding features Mapping of all new categories to category prototypes (in The cosine similarity between the two values ​​yields the attribute-guided confidence score. The calculation formula is:

[0090] in, The first one that was determined to be the background Semantic embedding features of regions of interest For the first A new category mapping category prototype.

[0091] S42. For each region of interest identified as background, obtain its highest similarity score across all new categories. Second highest similarity score The difference between the two is then calculated as the confidence boundary value.

[0092] in, The highest similarity score, This is the second highest similarity score.

[0093] Determine the confidence boundary value Is it greater than the preset boundary threshold? This filters out ambiguous regions with unclear semantic features, and only performs subsequent label correction on regions of interest that meet the boundary value constraints. S43. Extract the highest similarity score The corresponding optimal new category (Right now Based on the highest similarity score and their corresponding preset pseudo-label thresholds and background mask threshold The monitoring signal of the background region of interest is corrected using a three-state routing rule, as follows: (1) Online pseudo-label segmentation: when the highest similarity score Greater than the preset pseudo-label threshold When the boundary value constraint is satisfied, the region is determined to be a new target class that was missed by the label, and the label of the background region of interest is corrected to the optimal new class. The pseudo-labels are used as positive samples in online training; (2) Background mask exclusion: when the highest similarity score Less than or equal to the preset pseudo-label threshold And greater than the preset background mask threshold When the boundary value constraint is satisfied, the region is determined to have a strong foreground suspicion but not certain enough, and it is marked as an exclusion sample so that it can be directly masked and the erroneous negative gradient is filtered out in the subsequent negative sample supervision calculation. (3) Reliable background preservation: For regions of interest that do not meet the above two conditions (i.e., the highest similarity score) Less than or equal to the preset background mask threshold If the boundary value constraint is not met, it is determined that the semantic association with the new category is extremely low. The original background label of the region of interest is retained and used as a reliable background sample for normal training.

[0094] The rules are as follows:

[0095] in, For pseudo-label threshold, Background mask threshold, This is the boundary threshold.

[0096] S5. Combine the object detection loss and the attribute semantic manifold loss to jointly optimize the network.

[0097] For example, network joint optimization involves jointly training and updating the parameters of the object detection network based on the detection loss output by the object detection branch and the attribute semantic manifold loss calculated by combining the visual-semantic alignment and the corrected background supervision signal. In this embodiment, S5 specifically includes: S51, Feature Sharing and Gradient Decoupling Processing: The visual features of the region of interest (RoI) are shared and input into the visual detection branch and the attribute branch; to avoid fine-tuning instability caused by different optimization objectives, a gradient decoupling layer (GDL) is set before the attribute head module of the attribute branch to control the interference of attribute semantic supervision on the backbone network; S52. During the forward propagation of the network, the gradient decoupling layer performs an identity mapping operation, allowing visual features to enter the attribute branch without loss, i.e. ; S53. During the backpropagation process of the network, the gradient decoupling layer operates according to a preset attenuation coefficient. (in The gradients calculated from the attribute branches and propagated back to the backbone are scaled and decayed, and the calculation formula is as follows:

[0098] S54. Loss Calculation and Label Application: During the few-sample model fine-tuning stage, the visual detection branch maintains the standard optimization path, calculates and outputs the detection loss, which includes category classification and bounding box regression. Simultaneously, the labels (pseudo-labels, excluded labels, or original background labels) corrected by the background correction module (AGR) described in step 4 are applied to the attribute branch, and the final attribute semantic manifold loss is calculated in conjunction with the visual-semantic alignment process. ; S55, End-to-end joint parameter update will reduce the detection loss of the visual detection branch. Attribute semantic manifold loss with attribute branching Add them together to construct the final joint optimization objective. The calculation formula is:

[0099] Based on the joint optimization objective We perform end-to-end joint training and network parameter updates for the entire bi-branch object detection network model.

[0100] In this embodiment, the method further includes: S6. During the testing phase, the image containing the new category of target is input into the trained dual-branch target detection network to classify and locate the new category of target, and the average detection accuracy is calculated as the performance evaluation index. In this embodiment, S6 specifically includes: S61. During the testing phase, a pre-divided target detection test set (e.g., the PASCAL VOC test set) is obtained; under a very few-shot (K-shot) setting, images containing new target categories are extracted from the test set as test inputs, where each new category provides only one image. One labeled instance is used as a fine-tuning reference, and the remaining test images are used to verify the generalization performance of the model; S62. Input the test image into the trained dual-branch object detection network. After the backbone network extracts features, the region proposal network generates candidate regions. Combining the visual detection branch and the attribute branch, the predicted category and bounding box coordinates of each candidate region are output. S63. The average precision (AP) of the new class in a small sample is used as the evaluation index of the model detection performance when the intersection-over-union ratio (IoU) threshold is set to 0.5. The target class and bounding box coordinates predicted by the network are compared with the real annotations to calculate the classification accuracy and localization accuracy of the model for the new class of targets.

[0101] In the verification experiment of this invention, two representative partitioning schemes, Novel Class Partition 1 and Novel Class Partition 3, from the PASCAL VOC dataset were selected and evaluated respectively. , and The detection performance of the model under 1-shot, 2-shot, and 5-shot conditions is shown in Table 1. Table 1 compares the accuracy of the proposed method with other mainstream few-shot object detection methods on the PASCAL VOC test set. Experimental results show that the proposed method performs particularly well under the 1-shot setting with extremely little visual supervision, significantly improving the recognition ability of new object categories.

[0102] Table 1 Accuracy Comparison

[0103] (Note: The values ​​in the table represent nAP50 accuracy percentages; higher values ​​indicate better detection performance.) By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art: This invention is based on a dual-branch learning architecture consisting of an attribute-semantic manifold modeling module and an attribute-guided background correction module. The attribute-semantic manifold construction module transforms fine-grained attribute priors generated by a large language model into a structured semantic manifold, overcoming the limitation of traditional methods that rely on only a very small number of visual samples to construct category prototypes. This effectively alleviates the semantic drift problem caused by the shift of new category representations towards the base class. Simultaneously, leveraging the stable semantic recognition capability of the attribute manifold in the early stages of fine-tuning, a novel three-state routing rule is proposed in the attribute-guided background correction module to adaptively evaluate the confidence of the background region and correct labels. Without introducing additional complex penalty terms, this effectively eliminates negative gradient interference from unlabeled foreground targets and successfully mines potential positive sample supervision signals. Furthermore, the introduction of a gradient decoupling layer achieves efficient and stable collaborative optimization of visual detection and semantic alignment. This invention achieves significantly better detection accuracy than existing technologies under extremely small sample settings on multiple publicly available object detection datasets, greatly improving the model's generalization performance and detection robustness in complex real-world environments with limited data and incomplete annotations.

[0104] Example 2 Please see Figure 5 A few-shot target detection device based on attribute semantic alignment, comprising: The feature extraction unit is used to acquire the input image and extract the visual features of the region of interest in the image using the object detection backbone network; The attribute semantic manifold construction unit is used to obtain fine-grained attribute features generated by a large language model, and construct a hypergraph structure with the attribute features as nodes and categories as hyperedges. Optimized attribute representations are generated through hypergraph propagation, and category prototypes of each category are aggregated accordingly to form an attribute semantic manifold. The visual-semantic alignment unit is used to project the visual features of the region of interest onto the semantic space to obtain semantic embedding features, and to perform multiple visual-semantic alignments of the semantic embedding features with the category prototypes and attribute representations in the attribute semantic manifold. The background correction unit is used in the model fine-tuning stage to calculate the semantic confidence between the semantic embedding features of the region of interest initially identified as background and the category prototype of the new category, and adaptively correct the label of the background region of interest to pseudo-label foreground, excluded sample or reliable background based on the semantic confidence, so as to correct the background supervision signal. The joint optimization unit is used to combine the object detection loss and the attribute semantic manifold loss to jointly optimize the network.

[0105] In order to better utilize the method described in one of the embodiments, this application proposes a small sample target detection device based on attribute semantic alignment. Each module corresponds to a step of the above method, and its specific principle has been described above and will not be repeated here.

[0106] Example 3 An electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the few-sample target detection method based on attribute semantic alignment as described in one embodiment.

[0107] In the above technical solution, to better operate and process the method, the method is stored in memory, and the processor executes the stored method. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here.

[0108] Example 4 A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the few-sample object detection method based on attribute semantic alignment as described in one embodiment.

[0109] In the above technical solution, to better operate and use the method, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here.

[0110] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A few-sample target detection method based on attribute semantic alignment, characterized in that, The method includes: The input image is acquired, and the visual features of the region of interest in the image are extracted using an object detection backbone network. We acquire fine-grained attribute features generated by a large language model, construct a hypergraph structure with the attribute features as nodes and categories as hyperedges, generate optimized attribute representations through hypergraph propagation, and aggregate them to obtain category prototypes for each category, forming an attribute semantic manifold. The visual features of the region of interest are projected onto the semantic space to obtain semantic embedding features, and the semantic embedding features are then subjected to multiple visual-semantic alignment with the category prototypes and attribute representations in the attribute semantic manifold. During the model fine-tuning stage, the semantic embedding features of the region of interest initially identified as background are calculated and the semantic confidence between the category prototype of the new category is calculated. Based on the semantic confidence, the label of the background region of interest is adaptively corrected to a pseudo-label foreground, excluded sample, or reliable background to correct the background supervision signal. The network is jointly optimized by combining object detection loss and attribute semantic manifold loss.

2. The few-sample target detection method based on attribute semantic alignment as described in claim 1, characterized in that, The process involves acquiring fine-grained attribute features generated by a large language model, constructing a hypergraph structure with these attribute features as nodes and categories as hyperedges, generating optimized attribute representations through hypergraph propagation, and aggregating these representations to derive category prototypes for each category, thus forming an attribute semantic manifold. Specifically, this includes: We use a large language model to generate fine-grained text attribute descriptions for each category, and then encode them into attribute embedding vectors using a text encoder. Cluster the attribute embedding vectors and use the cluster centers as compact initial attribute node features; An association matrix is ​​constructed based on the attribution relationship between attribute nodes and categories, and the hypergraph structure is constructed accordingly. By using a composite propagation operator that includes a hypergraph propagation matrix, an attribute node pairwise similarity matrix, and a self-connection matrix, the attribute node features are propagated and updated hierarchically to obtain the optimized attribute representation.

3. The few-sample target detection method based on attribute semantic alignment as described in claim 2, characterized in that, After obtaining the optimized attribute representation, the process further includes: A residual mapping network is introduced to adaptively adjust the visual distribution of the final layer attribute features after hierarchical propagation, generating the final optimized attribute representation; Based on the association matrix, the final optimized attribute representation is weighted and aggregated to generate the category prototypes for all categories.

4. The few-sample target detection method based on attribute semantic alignment as described in claim 1, characterized in that, The multiple visual-semantic alignment is achieved by combining at least two of the following attribute semantic manifold losses: Cosine prototype alignment loss is used to narrow the distance between the semantic embedding feature and the category prototype of its corresponding target category in the manifold space. Prototype contrastive classification loss is used to increase the distance between the semantic embedding features and the class prototypes of non-target categories, thereby enhancing global discriminability; The attribute-level alignment loss is used to constrain the response distribution of the semantic embedding features on fine-grained attribute nodes, so that it is consistent with the true attribute distribution of the target category.

5. The few-sample target detection method based on attribute semantic alignment as described in claim 1, characterized in that, The labels of the background region of interest are adaptively corrected based on the semantic confidence, and the specific correction decision rules include: When the highest score of the semantic confidence is greater than the preset pseudo-label threshold, and the difference between the highest score and the second highest score is greater than the preset boundary threshold, the label of the region is corrected to the corresponding new category pseudo-label. When the highest score of the semantic confidence is between the preset pseudo-label threshold and the preset mask threshold, and the difference condition is met, the region is marked as an excluded sample so that it does not participate in the calculation of negative sample loss. In other cases, the original background label of the area is retained.

6. The few-sample target detection method based on attribute semantic alignment as described in claim 1, characterized in that, Before projecting the visual features of the region of interest into the semantic space to obtain semantic embedding features, and before performing multiple visual-semantic alignment of the semantic embedding features with the category prototypes and attribute representations in the attribute semantic manifold, the process further includes: The visual features of the region of interest are subjected to gradient decoupling processing, so that the gradient from the attribute semantic manifold loss is back propagated to the target detection backbone network with a preset attenuation coefficient during backpropagation.

7. The few-sample target detection method based on attribute semantic alignment as described in claim 1, characterized in that, Projecting the visual features of the region of interest into the semantic space specifically includes: A learnable query projection matrix and a learnable key projection matrix are introduced to map the visual features and prior information in the attribute semantic manifold to a unified alignment subspace to enhance the matching degree between vision and semantics.

8. A few-sample target detection device based on attribute semantic alignment, characterized in that, include: The feature extraction unit is used to acquire the input image and extract the visual features of the region of interest in the image using the object detection backbone network; The attribute semantic manifold construction unit is used to obtain fine-grained attribute features generated by a large language model, and construct a hypergraph structure with the attribute features as nodes and categories as hyperedges. Optimized attribute representations are generated through hypergraph propagation, and category prototypes of each category are aggregated accordingly to form an attribute semantic manifold. The visual-semantic alignment unit is used to project the visual features of the region of interest onto the semantic space to obtain semantic embedding features, and to perform multiple visual-semantic alignments of the semantic embedding features with the category prototypes and attribute representations in the attribute semantic manifold. The background correction unit is used in the model fine-tuning stage to calculate the semantic confidence between the semantic embedding features of the region of interest initially identified as background and the category prototype of the new category, and adaptively correct the label of the background region of interest to pseudo-label foreground, excluded sample or reliable background based on the semantic confidence, so as to correct the background supervision signal. The joint optimization unit is used to combine the object detection loss and the attribute semantic manifold loss to jointly optimize the network.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the few-sample target detection method based on attribute semantic alignment as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the few-sample object detection method based on attribute semantic alignment as described in any one of claims 1 to 7.

Images