A few-sample open set identification method and system, electronic device, and storage medium

Abstract

The application discloses a few-sample open set identification method and system, electronic equipment and storage medium, belonging to the technical field of computer vision, comprising: obtaining training images containing known category labels; extracting a plurality of semantic primitives from the auxiliary text vocabulary set through dictionary learning; setting learnable combination weights for the to-be-synthesized unknown semantic anchor, adding random disturbance, weighting and summing the semantic primitives to obtain the unknown semantic anchor; obtaining the known category text and the unknown category text prompt through coding to obtain the category text features; updating the learnable context vector and the learnable combination weight through the training target; obtaining the image features of the to-be-identified image, assigning a quality value according to the consistency measurement between the image features of the to-be-identified image and the category text features, and determining the category label according to the quality value. The technical scheme can improve the coverage diversity of unknown semantics and the reliability of boundary determination in few-sample open set identification.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, system, electronic device and storage medium for identifying few-shot open sets. Background Technology

[0002] Few-shot open-set recognition requires models trained with only a small number of labeled samples for each known category. These models must not only accurately classify samples of known categories but also identify and reject samples of unknown categories not seen during training during the inference phase. Visual-language pre-trained models, through large-scale image-text pair pre-training, establish a cross-modal embedding space shared by the image encoder and text encoder, providing a good transfer foundation for few-shot recognition tasks. Cue learning, by introducing a small number of learnable continuous vectors as context for text cues into the pre-trained model, can efficiently adapt to downstream tasks while keeping the pre-trained encoder parameters frozen, making it particularly suitable for few-shot scenarios with limited labeled data.

[0003] In existing technologies, few-shot open set recognition methods based on visual language pre-trained models typically use artificially constructed open category vocabulary sets as the source of semantic descriptions for unknown categories. Other methods utilize external generative models to generate pseudo-unknown samples as training proxies for unknown categories, or directly learn several optimizable text vectors for unknown categories in the embedding space of the text encoder. By establishing semantic reference directions for unknown categories in these ways, images can be compared with known categories and these unknown semantic directions during the inference stage to determine whether they belong to known categories.

[0004] However, artificially constructed open vocabulary sets are limited by the subjectivity of vocabulary selection and the limited coverage, making it difficult to fully represent the semantic diversity of unknown categories. External generative models require additional model training and data generation costs to generate pseudo-unknown samples, and the distribution of generated pseudo-samples deviates from the distribution of real unknown categories. Methods that directly learn unknown text vectors lack clear semantic constraints, and the multiple learned unknown text vectors tend to concentrate in local semantic regions of the embedding space, resulting in insufficient coverage of unknown directions near the boundaries of known categories, thus reducing the reliability of open set boundaries. Furthermore, the aforementioned methods typically use a single confidence score, such as the maximum softmax probability, as the criterion for determining the known category during the inference stage. This score only reflects the relative probability of the image on each known category branch. When the image is located at the boundary of the open set, if both known and unknown branches give strong responses, this single confidence score cannot distinguish between "insufficient known evidence" and "simultaneously strong known and unknown evidence," easily leading to unknown samples being mistakenly accepted as known categories or known samples being mistakenly rejected, affecting the reliability of open set recognition under conditions of few samples.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To address the aforementioned issues, this application provides a method, system, electronic device, and storage medium for identifying few-shot open sets, which can improve the coverage diversity of unknown semantics and the reliability of boundary determination in the identification of few-shot open sets.

[0007] To achieve the objectives of this application, the following technical solution is provided:

[0008] Firstly, this application provides a method for identifying few-sample open sets, including: Acquire training images containing known category labels, as well as a set of known category names, a set of auxiliary text vocabularies, and a visual language pre-trained model; the visual language pre-trained model includes an image encoder and a text encoder; The auxiliary text vocabulary set is tokenized to obtain effective token embeddings, and multiple semantic primitives are extracted from all the effective token embeddings through dictionary learning, and normalized to obtain a semantic primitive matrix. Learnable combination weights are set for multiple unknown semantic anchors to be synthesized, and random perturbations are added during the training phase. The learnable combination weights with random perturbations are weighted and summed on the semantic primitives in the semantic primitive matrix to obtain multiple unknown semantic anchors. The known category name and the unknown semantic anchor are respectively placed into a shared prompt structure containing learnable context vectors to obtain known category text prompts and unknown category text prompts, and then encoded by the text encoder to obtain category text features; the category text features include known category text features and unknown category text features; Based on the consistency measure between the image features of the training images and the category text features, a training objective is constructed that includes unknown perception calibration loss, known and unknown semantic separation loss, and unknown semantic dispersion loss. The learnable context vector and the learnable combined weights are then updated through the training objective. During the inference phase, image features of the image to be identified are acquired, evidence values ​​are generated based on the consistency measure between the image features of the image to be identified and the category text features, quality values ​​are assigned based on the evidence values, and category labels are determined based on the quality values; the evidence values ​​include known evidence and unknown evidence, and the quality values ​​include known quality, unknown quality, and unknown quality.

[0009] Secondly, this application provides a few-shot open set identification system for performing the above-described few-shot open set identification method, comprising: The sample acquisition module is used to acquire training images containing known category labels, as well as a set of known category names, a set of auxiliary text vocabulary, and a visual language pre-training model; the visual language pre-training model includes an image encoder and a text encoder; The semantic extraction module is used to tokenize the auxiliary text vocabulary set to obtain effective token embeddings, and extract multiple semantic primitives from all the effective token embeddings through dictionary learning, and normalize them to obtain a semantic primitive matrix. The semantic computation module is used to set learnable combination weights for multiple unknown semantic anchors to be synthesized, and to add random perturbation during the training phase. The learnable combination weights with random perturbation are weighted and summed on the semantic primitives in the semantic primitive matrix to obtain multiple unknown semantic anchors. The feature encoding module is used to place the known category name and the unknown semantic anchor into a shared prompt structure containing learnable context vectors, respectively, to obtain known category text prompts and unknown category text prompts, and to encode them into category text features by the text encoder; the category text features include known category text features and unknown category text features; The parameter update module is used to construct a training objective including unknown perception calibration loss, known and unknown semantic separation loss and unknown semantic dispersion loss based on the consistency measure between the image features of the training image and the category text features, and to update the learnable context vector and the learnable combined weights through the training objective; The label determination module is used to acquire image features of the image to be identified during the inference stage, generate evidence values ​​based on the consistency measure between the image features of the image to be identified and the category text features, assign quality values ​​based on the evidence values, and determine category labels based on the quality values; the evidence values ​​include known evidence and unknown evidence, and the quality values ​​include known quality, unknown quality, and unknown quality.

[0010] Thirdly, this application provides an electronic device, comprising: processor; Memory for storing the executable instructions of the processor; The processor is configured to perform the steps of the few-sample open set identification method described above by executing the executable instructions.

[0011] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned few-sample open set identification method.

[0012] The technical solution provided in this application may include the following beneficial effects: This application provides a method, system, electronic device, and storage medium for identifying few-shot open sets. It can extract semantic primitives from auxiliary text words and synthesize them into unknown semantic anchors by weighting them. This makes the unknown semantic anchors have an interpretable semantic source and does not need to rely on training images of unknown classes. It solves the problem in the prior art that the unknown semantic direction is difficult to stably cover the unknown direction near the boundary of the known class, and improves the interpretability of unknown semantic prompts and the diversity of coverage of the unknown semantic space. Furthermore, by adding random perturbations to the learnable combination weights during the training phase, the same unknown semantic anchor can undergo local semantic changes during training. This, combined with the known and unknown semantic separation loss, expands the semantic gap between known and unknown category text features. At the same time, the unknown semantic dispersion loss limits the similarity between different unknown category text features. This can prevent multiple unknown semantic anchors from concentrating in the same local region of the semantic space, improve the separability of the open set boundary and the ability to cover different unknown semantic directions, and enable the model to effectively distinguish between known and unknown categories under conditions of few samples. Meanwhile, by converting the matching score into non-negative evidence during the inference stage and assigning known quality, unknown quality, and unknown quality based on the evidence value, the three states of known support, unknown support, and insufficient evidence in the current identification result can be explicitly represented. The conflict perception score is calculated using the known quality and the unknown quality, so that the conflict perception score is suppressed when both known evidence and unknown evidence are strong at the same time. This solves the problem that a single confidence score, such as the maximum softmax probability, is difficult to distinguish between insufficient known evidence and strong known and unknown evidence at the same time. It provides a more prudent judgment basis for open set boundary samples, thus achieving both accurate identification of known categories and effective detection of unknown categories under the condition of a small number of training samples.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0014] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the embodiments of the present application to explain the application and do not constitute a limitation thereof. Obviously, the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0015] Figure 1 A flowchart illustrating a method for identifying a few open sets provided in an embodiment of this application; Figure 2 A schematic diagram of the overall structure of the training and inference phases of a few-sample open set identification method provided in this application embodiment; Figure 3 This is a schematic diagram illustrating the unknown semantic anchor synthesis process of a few-sample open set identification method provided in this application embodiment; Figure 4 A flowchart illustrating step S300 of a few-sample open set identification method provided in an embodiment of this application; Figure 5 A flowchart illustrating step S400 of a few-sample open set identification method provided in an embodiment of this application; Figure 6 This is a schematic diagram of the unknown-aware semantic anchor regularization training structure for a few-shot open set recognition method provided in this application embodiment; Figure 7 A flowchart illustrating step S500 of a few-sample open set identification method provided in an embodiment of this application; Figure 8 A flowchart illustrating step S600 of a few-sample open set identification method provided in an embodiment of this application; Figure 9 This is a schematic diagram illustrating the conflict-aware evidence reasoning process of a few-sample open set identification method provided in this application embodiment; Figure 10 A schematic diagram of the t-SNE visualization results of UTL on the C100 dataset provided in the embodiments of this application; Figure 11 A schematic diagram illustrating the t-SNE visualization results of a few-shot open set identification method provided in this application embodiment on the C100 dataset; Figure 12 This is a schematic diagram illustrating the distribution of open set decision scores of UTL on the C100 dataset, provided in an embodiment of this application. Figure 13 A schematic diagram illustrating the distribution of open set decision scores on the C100 dataset for a few-sample open set identification method provided in this application embodiment; Figure 14 This is a schematic diagram of a few-sample open set identification system provided in an embodiment of this application. Detailed Implementation

[0016] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0017] The technical solution of this application can be applied to visual recognition scenarios where sample annotation costs are high and unknown categories may appear during the testing phase. For example, in industrial defect detection, a model can be trained using only a small number of labeled known defect samples, and known defect types can be identified on the production line, while novel, unseen defects can be classified as unknown categories. In intelligent security and unmanned inspection scenarios, target recognition can be completed using a small number of known target images, and open-set rejection can be performed on abnormal targets, unfamiliar objects, or abnormal scenes that did not appear during the training phase. In scenarios such as remote sensing image analysis, medical image-assisted recognition, robot environmental perception, and autonomous driving perception, the method of this application can also be used to maintain the ability to recognize known categories under limited labeled sample conditions and improve the reliability of discrimination against unknown categories or samples with ambiguous boundaries. The above application scenarios are merely examples, and this application is not limited to a specific dataset, a specific category name, or a specific visual task.

[0018] This example implementation first provides a method for identifying open sets with few samples. (See reference...) Figure 1 As shown, the few-sample open set identification method may include the following steps: Step S100: Obtain training images containing known category labels, as well as a set of known category names, a set of auxiliary text vocabulary, and a visual language pre-trained model; the visual language pre-trained model includes an image encoder and a text encoder.

[0019] Step S200: Tokenize the auxiliary text vocabulary set to obtain effective token embeddings, and extract multiple semantic primitives from all the effective token embeddings through dictionary learning, and normalize them to obtain a semantic primitive matrix.

[0020] Step S300: Set learnable combination weights for multiple unknown semantic anchors to be synthesized, and add random perturbation during the training phase. Sum the learnable combination weights with random perturbation to the semantic primitives in the semantic primitive matrix to obtain multiple unknown semantic anchors.

[0021] Step S400: The known category name and the unknown semantic anchor are respectively placed into a shared prompt structure containing learnable context vectors to obtain known category text prompts and unknown category text prompts, and the category text features are encoded by the text encoder; the category text features include known category text features and unknown category text features.

[0022] Step S500: Based on the consistency measure between the image features of the training image and the category text features, construct a training objective that includes unknown perception calibration loss, known and unknown semantic separation loss, and unknown semantic dispersion loss, and update the learnable context vector and the learnable combined weights through the training objective.

[0023] Step S600: In the inference stage, image features of the image to be identified are obtained, evidence values ​​are generated based on the consistency measure between the image features of the image to be identified and the category text features, quality values ​​are assigned based on the evidence values, and category labels are determined based on the quality values; the evidence values ​​include known evidence and unknown evidence, and the quality values ​​include known quality, unknown quality, and unknown quality.

[0024] It should be noted that this embodiment uses an image encoder. and text encoder A visual language pre-trained model. The known set of categories is represented as... .image Image features are obtained through an image encoder. Text prompt Text features are obtained through a text encoder. Both image and text features are normalized, and their matching scores can be calculated using temperature-scaled cosine similarity.

[0025] At the same time, such as Figure 2 As shown, the few-shot open set recognition method provided in this application includes a training phase and an inference phase. In the training phase, the learnable context vector and the learnable combined weights are optimized using the few-shot training images and the auxiliary text vocabulary set. In the inference phase, the random perturbation is turned off, and the optimized learnable context vector and the learnable combined weights are used to determine whether the image to be recognized belongs to a known or unknown category based on the conflict perception score.

[0026] Below, we will refer to Figures 3 to 9 This example embodiment will describe in more detail each step of the few-shot open set identification method described above.

[0027] In step S100, training images containing known category labels, a set of known category names, a set of auxiliary text vocabulary, and a visual language pre-trained model are obtained.

[0028] It should be noted that the training images are few-sample training images, with each known category containing only a small number of labeled samples, and all training images belong to the known category set, excluding images of unknown categories. The known category name set consists of the category names corresponding to all known category labels. The auxiliary text vocabulary set is independent of the known category name set, consisting of publicly available category name vocabulary, and is not used as the labeled categories for the training images, but only as the text source for subsequently constructing semantic primitives. The visual language pre-trained model is a multimodal model pre-trained with large-scale image-text pairs. Its image encoder is used to encode the input image into an image feature vector, and the text encoder is used to encode the input text prompt into a text feature vector. The image features and text features reside in the same shared embedding space.

[0029] Specifically, the training images are a small sample of training images containing known category labels, excluding images with unknown categories. The auxiliary text vocabulary serves only as a source of text for constructing semantic primitives and is not considered as the true unknown category label. In one evaluation example, the auxiliary text vocabulary consists of 1000 category names selected from ImageNet-21K based on WordNet; these selected names are semantically unrelated to the closed-set categories. For a target unknown category given in advance during evaluation, all related words are excluded based on the WordNet thesaurus. The same fixed source of auxiliary vocabulary is used for all evaluation tasks.

[0030] In step S200, the auxiliary text vocabulary set is tokenized to obtain effective token embeddings, and multiple semantic primitives are extracted from all the effective token embeddings through dictionary learning and normalized to obtain a semantic primitive matrix.

[0031] It should be noted that tokenization is the process of converting each word into a sequence of tokens. The effective token embedding is the embedding vector corresponding to each token remaining after removing the start and end tokens. Sparse dictionary learning learns a set of overcomplete semantic primitives from the token embedding matrix. Each semantic primitive represents a basic direction or atomic component in the semantic space. The token embeddings of different words can be approximately reconstructed by a sparse linear combination of these semantic primitives. Modulus normalization ensures that each semantic primitive has a unit length, facilitating subsequent consistency measurement with image features within the cosine similarity framework.

[0032] Specifically, the auxiliary text vocabulary set consists of publicly known category names and does not contain any words that overlap with the known category names; in the evaluation embodiment where a target unknown category is predefined, words related to the target unknown category are also excluded based on the synonym set identifier; each name in the auxiliary text vocabulary is input into a word segmenter, and after removing the start token and end token, valid tokens are collected and embedded to form:

[0033] in, For effective token embedding, The total number of valid tokens. For token embedding dimension.

[0034] Solving using sparse dictionary learning:

[0035] in, For inclusion A dictionary of semantic primitives It is a sparse coefficient matrix. For the first Each token embeds a corresponding sparse coefficient. is the sparse regularization coefficient; the th The semantic primitives are normalized as follows: Perform operations on each semantic primitive in the dictionary. Normalization yields In the numerical implementation, the normalized denominator is set with a positive lower bound corresponding to the calculation precision to avoid division by zero caused by a zero vector.

[0036] In step S300, learnable combination weights are set for multiple unknown semantic anchors to be synthesized, and random perturbations are added during the training phase. The learnable combination weights with random perturbations are weighted and summed on the semantic primitives in the semantic primitive matrix to obtain multiple unknown semantic anchors.

[0037] It should be noted that the learnable combined weights are trainable parameters with randomly set initial values, optimized through backpropagation during training. Each unknown semantic anchor is composed of a weighted combination of multiple semantic primitives, and different unknown semantic anchors correspond to different learnable combined weight vectors, thus expressing different unknown semantic directions. The random perturbation is added only during the forward computation of the training phase to induce local semantic changes in the same unknown semantic anchor during training, simulating the uncertainty of the distribution of unknown categories in the semantic space. No random perturbation is added during the inference phase to ensure the repeatability of the judgment results for the same image to be recognized.

[0038] Specifically, for An unknown hint allows for the setting of a learnable combined weight matrix. For the first An unknown hint is sampled in the computation before each training iteration:

[0039] And order Then calculate: , ; in, For the first Learnable combined weights with unknown hints Let the variance be the Gaussian perturbation. A matrix composed of normalized semantic primitives. For the first An unknown semantic anchor.

[0040] During training, random perturbations are used to generate continuously varying unknown semantic directions around each combined weight. During inference, random perturbations are disabled, and the learned combined weights are used to determine the unknown semantic anchors, thus ensuring repeatable inference results for the same input. Figure 3 As shown, the unknown semantic anchor synthesis process can include auxiliary text vocabulary input, token embedding collection, sparse dictionary learning, semantic primitive normalization, combined weight perturbation, and unknown semantic anchor output. This process ensures that the unknown cue is not a sourceless free vector, but rather formed by the combination of interpretable semantic primitives.

[0041] In one possible implementation, such as Figure 4 As shown, step S300 may further include the following sub-steps: In step S310, a preset number of unknown semantic anchors to be synthesized is determined, and a learnable combination weight is initialized for each unknown semantic anchor; the dimension of each learnable combination weight is equal to the number of semantic primitives in the semantic primitive matrix.

[0042] It should be noted that the preset quantity is a hyperparameter, set by the user according to the actual task, representing the number of unknown semantic directions to be modeled. The learnable combined weight vector corresponding to each unknown semantic anchor has a dimension of K, where K is equal to the total number of semantic primitives in the semantic primitive matrix, ensuring that each combined weight can be assigned a weight coefficient to each semantic primitive.

[0043] In step S320, during each forward computation in the training phase, a random perturbation is independently sampled for each learnable combined weight, and the random perturbation is added to the learnable combined weight to obtain the perturbed combined weight.

[0044] It should be noted that the random perturbation can be independently sampled from a Gaussian distribution with a mean of zero and a preset variance. Each learnable combined weight is independently perturbed in each forward computation, causing a slight change in the semantic direction of the same unknown semantic anchor in different forward propagations within the same training batch.

[0045] In step S330, the semantic primitives in the semantic primitive matrix are weighted and summed using the perturbed combined weights to obtain the initial vectors corresponding to each unknown semantic anchor.

[0046] It should be noted that the weighted summation involves multiplying the f-th component of the perturbed combined weight vector with the f-th semantic primitive in the semantic primitive matrix, and then summing all the products to obtain an initial vector. This initial vector, located in the embedding space of the text encoder, is a weighted linear combination of the semantic primitives.

[0047] In step S340, the initial vectors are normalized by their magnitudes to obtain multiple unknown semantic anchors.

[0048] It should be noted that modulus normalization divides the initial vector by its own modulus to obtain an unknown semantic anchor of unit length, so that it is in the same normalized embedding space as the known category name embedding vector, which facilitates text feature encoding under a unified shared prompt structure.

[0049] In step S400, the known category name and the unknown semantic anchor are respectively placed into a shared prompt structure containing learnable context vectors to obtain known category text prompts and unknown category text prompts, and the category text features are obtained by encoding through the text encoder.

[0050] It should be noted that the shared prompt structure is a template framework used by all categories of text prompts. The learnable context vectors within it are uniformly optimized during training, enabling the model to learn the context representation best suited to the current task. Known category names and unknown semantic anchors are both used as category placeholders within the same shared prompt structure, ensuring consistency in prompt format between known and unknown categories.

[0051] In one possible implementation, such as Figure 5 As shown, step S400 may further include the following sub-steps: In step S410, a shared cue structure is obtained; the shared cue structure contains multiple learnable context vectors.

[0052] It should be noted that the learnable context vectors in the shared prompt structure are a preset number of trainable embedding vectors, the number of which is set by the user. All known category text prompts and unknown category text prompts share the same set of learnable context vectors, enabling the model to share prompt context information between different categories and efficiently learn parameters in scenarios with few samples.

[0053] In step S420, for each known category name, the start token, the learnable context vector, the embedding vector of the known category name, and the end token are concatenated in sequence to obtain the known category text prompt corresponding to the known category.

[0054] It should be noted that for each known category, the name of the known category is first input into the word segmenter of the text encoder to obtain the embedding vector corresponding to the name. Then, the start token, all learnable context vectors, the name embedding vector, and the end token are concatenated in sequence to form a complete text prompt sequence for the known category. The start token and end token are standard special tokens predefined by the text encoder, used to mark the start and end positions of the text sequence.

[0055] Specifically, the shared cue structure contains learnable context vectors. . No. The known category hints and the first The unknown category hints are as follows:

[0056]

[0057] in, For the first Embedding of a known category name, and These are the start token and the end token, respectively.

[0058] Both use the same context structure, the difference being that the category position is occupied by an embedding of a known category name and a synthesized unknown semantic anchor, respectively. The text encoder outputs text features of known categories. and unknown category text features For training image features Calculate the known matching scores respectively. Matching score with unknown .

[0059] In step S430, for each unknown semantic anchor, the start token, the learnable context vector, the unknown semantic anchor, and the end token are concatenated in sequence to obtain the unknown category text prompt corresponding to the unknown category.

[0060] It should be noted that the text hints for unknown categories use the exact same concatenation order and structure as those for known categories. The only difference is that the category placeholder is placed using the unknown semantic anchor synthesized in step S300, rather than the embedding vector of the known category name. This allows the unknown category to have an equivalent representation in the text encoder as the known category.

[0061] In step S440, all known category text prompts and all unknown category text prompts are input into the text encoder for encoding, thereby obtaining known category text features and unknown category text features.

[0062] It should be noted that all known category text prompts and all unknown category text prompts are input into the text encoder, and the text encoder outputs the text feature vector corresponding to each prompt. The text features of all known category texts constitute the known category text feature set, and the text features of all unknown category texts constitute the unknown category text feature set. The two types of features are located in the same embedding space and can be directly compared with image features for similarity calculation.

[0063] In step S500, a training objective is constructed based on the consistency measure between the image features of the training image and the category text features. This objective includes unknown perception calibration loss, known and unknown semantic separation loss, and unknown semantic dispersion loss. The learnable context vector and the learnable combined weights are then updated using the training objective.

[0064] It should be noted that the parameters of the image encoder and the text encoder are frozen during the training phase. That is, all parameters of the pre-trained model are not updated during training; only the learnable context vector and the learnable combined weights are optimized. This parameter-efficient training method is suitable for scenarios with few samples and can avoid overfitting problems caused by insufficient labeled data.

[0065] Specifically, such as Figure 6 As shown, the training structure takes known cue features, unknown cue features, and training image features as inputs, and simultaneously forms three types of constraints: unknown perception calibration, separation of known and unknown semantics, and dispersion of unknown semantics. These three constraints work together on the shared context vector and the combined weights of unknown cue features.

[0066] In one possible implementation, such as Figure 7 As shown, step S500 may further include the following sub-steps: In step S510, the training image is input into the image encoder to obtain the image features.

[0067] It should be noted that after the training image is input into the image encoder, the image encoder outputs the corresponding image feature vector. This image feature vector and the text feature vector are located in the same shared embedding space, and both have undergone modulus normalization, which facilitates the use of cosine similarity for consistency measurement.

[0068] In step S520, the similarity between the image features and each of the known category text features is calculated to obtain a known matching score set, and the similarity between the image features and each of the unknown category text features is calculated to obtain an unknown matching score set.

[0069] It should be noted that the similarity can be achieved using temperature-scaled cosine similarity. This involves first calculating the cosine similarity between the image feature vector and the text feature vector, then dividing by the temperature parameter for scaling. The temperature parameter adjusts the smoothness of the similarity distribution, affecting the sharpness of the predicted probability.

[0070] In step S530, an unknown perception calibration loss is constructed based on the known matching score set and the unknown matching score set; the unknown perception calibration loss is used to retain the response of each unknown category branch while supervising the classification of known categories.

[0071] It should be noted that the unknown perception calibration loss calculates the predicted probabilities of both known and unknown category branches under a unified denominator. This ensures that unknown category branches also receive supervision signals during training, thus encouraging an increase in the predicted probabilities of each unknown category branch. This allows unknown category text features to obtain semantic responses during the training phase, avoiding the problem of insufficient unknown branch responses that occurs when unknown categories are only introduced during the inference phase.

[0072] Furthermore, the unknown perception calibration loss is:

[0073] in, , , ; Calibration loss for unknown perception Given the class probabilities, For truly known categories, This represents the trade-off coefficient between known category discrimination and unknown perception calibration. For known class identification - unknown branch joint activation loss, For unknown information, auxiliary supervision weights are provided. The probability of this is unknown. For image samples With the Matching scores for known category text features. For temperature parameters, For image samples With the Matching scores for known category text features. For image samples With the Matching scores for text features of unknown categories.

[0074] In step S540, a known-unknown semantic separation loss is constructed based on the known category text features and the unknown category text features; the known-unknown semantic separation loss is used to reduce the similarity between any known category text features and any unknown category text features.

[0075] It should be noted that the known and unknown semantic separation loss adopts a hinge loss form, which only generates a non-zero loss when the similarity between any known category text feature and any unknown category text feature exceeds a preset boundary value; the loss is zero when the similarity is below the boundary value. This promotes the separation of known category text features and unknown category text features in the embedding space, forming a clear open set decision boundary.

[0076] Furthermore, the known-unknown semantic separation loss is:

[0077] in, For the separation loss of known and unknown semantics, The number of prompts is unknown. Given the number of categories, For the first Unknown category text features For the first Known category text features, This represents the similarity boundary between known and unknown semantics.

[0078] In step S550, an unknown semantic dispersion loss is constructed based on the pairwise similarity between each of the unknown category text features; the unknown semantic dispersion loss is used to reduce the similarity between different unknown category text features.

[0079] It should be noted that the unknown semantic dispersion loss can be in the form of the square of the hinge loss, and a non-zero loss only occurs when the similarity between text features of different unknown categories exceeds a preset boundary value. This allows the text features corresponding to each unknown semantic anchor to be dispersed in the embedding space, avoiding multiple unknown semantic anchors from concentrating in the same local region of the semantic space, thereby enhancing the ability to cover the semantic diversity of unknown categories.

[0080] Furthermore, the unknown semantic dispersion loss is:

[0081] in, For unknown semantic dispersion loss, This represents the similarity boundary between different unknown semantics.

[0082] In step S560, the unknown perception calibration loss, the known and unknown semantic separation loss, and the unknown semantic dispersion loss are weighted and summed to obtain the total training objective, and the learnable context vector and the learnable combination weights are updated according to the total training objective.

[0083] It should be noted that after obtaining the overall training objective by weighted summation of the three types of losses, the gradients of the learnable context vector and the learnable combined weights relative to the overall training objective are calculated through backpropagation, and the optimizer updates these learnable parameters based on the gradients. The training process is iterative until the overall training objective converges or reaches the preset number of training epochs.

[0084] Furthermore, the overall training objective is:

[0085] in, For the overall training objective, This is the regularization weight.

[0086] In step S600, during the inference phase, image features of the image to be identified are obtained, an evidence value is generated based on the consistency measure between the image features of the image to be identified and the category text features, a quality value is assigned based on the evidence value, and a category label is determined based on the quality value.

[0087] It should be noted that the difference between the inference phase and the training phase is that the inference phase does not add random perturbations to the learnable combined weights, nor does it update any learnable parameters. It only uses the learnable context vector and learnable combined weights optimized in the training phase to perform forward computation on the image to be recognized and output the judgment result.

[0088] In one possible implementation, such as Figure 8 As shown, step S600 may further include the following sub-steps: In step S610, during the inference stage, the image to be identified is input into the image encoder for encoding to obtain the image features of the image to be identified.

[0089] It should be noted that the image to be identified can be of a known category or an unknown category; the model does not know its category before inference. After inputting the image to be identified into the image encoder, an image feature vector is obtained that lies in the same feature space as the training image.

[0090] In step S620, the similarity between the image features and each of the known category text features is calculated to obtain a set of known matching scores for the image to be identified, and the similarity between the image features and each of the unknown category text features is calculated to obtain a set of unknown matching scores for the image to be identified.

[0091] It should be noted that this step calculates the similarity between the features of the image to be identified and the features of all known categories and all unknown categories of text, resulting in two sets of matching scores. The known matching score set reflects the tendency of the image to be identified to belong to each known category, while the unknown matching score set reflects the tendency of the image to be identified to belong to each unknown semantic direction.

[0092] In step S630, each score in the known matching score set is converted into a non-negative known evidence value, and each score in the unknown matching score set is converted into a non-negative unknown evidence value.

[0093] It should be noted that the nonlinear mapping uses the Softplus function, which maps the input matching score to positive evidence and is a monotonically increasing function, so that a higher matching score corresponds to a larger evidence value and a lower matching score corresponds to a smaller evidence value, but all evidence values ​​are non-negative, which meets the basic requirement of evidence non-negativity in evidence theory.

[0094] In step S640, the maximum value is selected from all known evidence values ​​as the strongest known evidence, and the maximum value is selected from all unknown evidence values ​​as the strongest unknown evidence.

[0095] It should be noted that the strongest known evidence is selected from all known evidence values, and the strongest unknown evidence is selected from all unknown evidence values, rather than summing or averaging all evidence values. In open set identification, a sample may have some similarity to multiple known categories or multiple unknown semantic directions. Taking the maximum value can highlight the most discriminative evidence and avoid the dilution of evidence due to multi-category distribution.

[0096] In step S650, known quality, unknown quality, and unknown quality are allocated according to the strongest known evidence, the strongest unknown evidence, and the preset prior quality; wherein the sum of the known quality, the unknown quality, and the unknown quality is 1.

[0097] It should be noted that the preset prior quality can be a positive hyperparameter, representing the initial confidence level of the identification framework in the absence of any evidence. The total quality is the sum of the strongest known evidence, the strongest unknown evidence, and the preset prior quality. The known quality is the proportion of the strongest known evidence in the total quality, the unknown quality is the proportion of the strongest unknown evidence in the total quality, and the ignorance quality is the proportion of the preset prior quality in the total quality. The sum of these three is 1, constituting a complete quality allocation.

[0098] Specifically, for the image to be recognized The known and unknown matching scores are mapped to nonnegative evidence using the Softplus function:

[0099]

[0100] Retain the strongest evidence from both the known and unknown branches:

[0101]

[0102] In the identification framework Next, we introduce prior quality. ,calculate: .

[0103] Furthermore, the known mass is:

[0104] in, , , ; Known quality, that is, quality that supports the known category hypothesis. The strongest known evidence, For the amount of evidence used in the final decision, Given the known evidence value, For image samples With the Matching scores for known category text features. Preset prior quality; The unknown mass is:

[0105] in, , ; For unknown quality, that is, quality that supports the hypothesis of an unknown category. The strongest piece of unknown evidence, Value of unknown evidence. For image samples With the Matching scores for text features of unknown categories; The quality of ignorance is:

[0106] in, The quality of ignorance is the quality of ignorance to which the current evidence cannot be assigned to either hypothesis.

[0107] It should be noted that, In this embodiment, quality allocation and conflict-aware scoring are performed, but Dempster combination among multiple independent sources of evidence is not performed.

[0108] In step S660, a conflict perception score is calculated based on the known quality and the unknown quality; the conflict perception score has an inhibitory effect on the known quality when the unknown quality increases.

[0109] It should be noted that the conflict-aware score is equal to the difference between the known quality and the unknown quality. That is, when the unknown quality is large, even if the known quality is also large, the conflict-aware score will still be significantly suppressed. This approach can address the problem of boundary samples where both known and unknown evidence are strong. When the model favors both a known category and an unknown semantic direction, the conflict-aware score helps the model remain cautious and avoids blindly outputting known category labels.

[0110] Specifically, the formula for calculating the conflict perception score is as follows:

[0111] when When the image to be recognized is determined to be a known category image, the output is... The corresponding known category is output; otherwise, the unknown category determination is output. To preset the open set rejection threshold, and the aforementioned temperature parameters different.

[0112] like Figure 9 As shown, the conflict-aware evidence reasoning process first maps the known matching score and the unknown matching score to non-negative evidence, then retains the strongest evidence in the two branches and assigns known quality, unknown quality and unknown quality, and finally completes the known category output or unknown category determination through the conflict-aware score.

[0113] In step S670, when the conflict perception score is greater than the preset rejection threshold, the image to be identified is determined to be of a known category, and the known category label corresponding to the maximum score in the known matching score set is output; otherwise, the image to be identified is determined to be of an unknown category, and the unknown category determination is output.

[0114] It should be noted that the preset rejection threshold is a judgment threshold pre-set by the user according to task requirements. Adjusting this threshold can balance the accuracy of known class recognition and the detection rate of unknown classes. A higher threshold makes the model more inclined to output an unknown class judgment, increasing the unknown class detection rate but decreasing the known class recognition accuracy; a lower threshold makes the model more inclined to output a known class label, increasing the known class recognition accuracy but decreasing the unknown class detection rate. When a known class is determined, the output specific class label is the class corresponding to the highest score in the known matching score set.

[0115] Furthermore, this application provides a specific example implementation of a few-shot open set identification method, taking the CIFAR-10 few-shot open set identification task as an example. This embodiment is used to illustrate the specific execution process, parameter substitution method, and formula calculation logic of steps S100 to S600. The listed categories, terms, thresholds, and intermediate values ​​are all examples for ease of understanding and are not intended to limit the scope of protection of this application.

[0116] Step 1: Obtain training images, category sets, auxiliary text vocabulary sets, and a visual language pre-trained model. Six categories from the CIFAR-10 dataset are selected as known categories, with the known category names set as {airplane, automobile, bird, cat, deer, dog}, and their corresponding known category labels as {0, 1, 2, 3, 4, 5}. Sixteen labeled training images are selected for each known category, resulting in a total of 96 few-shot training images. During the testing phase, {frog, horse, ship, truck} can be used as unknown category samples, but images of unknown categories and their category names are not included in the training.

[0117] The auxiliary text vocabulary set is used to construct a composable foundation for the unknown semantic space, and may include, for example, {accordion, ambulance, banana, bridge, castle, guitar, hammer, kettle, lighthouse, microscope, mountain, piano, radio, sofa, television, tractor, violin, volatil, waterfall, backpack}. In actual simulations, the auxiliary text vocabulary set can be expanded to 1000 category names from ImageNet-21K or WordNet, with words that overlap with known categories and target unknown categories or directly correspond semantically to them removed. The visual language pre-trained model uses CLIPViT-B / 16, whose image encoder and text encoder parameters are frozen during training.

[0118] Step 2: Construct a semantic primitive dictionary based on the auxiliary text vocabulary set: Input the auxiliary text vocabulary into the CLIP word segmenter and word embedding layer. After removing the start tag SOS and end tag EOS, the effective token embedding set Z is obtained. If the example obtains N=2000 effective token embeddings, and each token embedding dimension d=512, then the size of Z is 2000×512. K=100 semantic primitives are obtained using sparse dictionary learning. The semantic primitive dictionary D has a size of 100×512, and the sparse coefficient matrix A has a size of 2000×100.

[0119]

[0120] in, =1.0, the maximum number of iterations for sparse dictionary learning is set to 1000, and minimum angle regression is used for the solution. After training, each dictionary atom is normalized to ensure consistent scaling in subsequent cosine similarity calculations, specifically:

[0121] Among them, if the first semantic primitive The L2 norm is =2.5, then the normalized semantic primitives are = This leads to the normalized semantic primitive dictionary. .

[0122] Step 3: Generate unknown semantic anchors and construct unknown hints based on the semantic primitive dictionary: Let the number of unknown hints be... The combined weight matrix W has a size of 50×100 and is initialized according to a Gaussian distribution with a mean of 0 and a standard deviation of 0.02. To avoid confusion with the unknown label smoothing factor ε, this embodiment denotes the training-period combined weight perturbation as ε0. and set Follows a mean of 0 and a standard deviation of σ. p A Gaussian distribution with a value of 0.1.

[0123]

[0124]

[0125] For example, if the weights of the first three combinations of the first unknown hint are The first three perturbations obtained from sampling during a forward propagation are The combined weights after perturbation are The unknown semantic anchor point is obtained by linearly combining the perturbed combined weights with the normalized semantic primitive dictionary.

[0126]

[0127]

[0128] Taking the first unknown prompt as an example, when only the first three semantic primitives are shown, there is After normalization, we get Repeat the above calculation for 50 unknown prompts to obtain the set of unknown semantic anchors. Simultaneously, set the number of context tokens L=16, and construct known category hints and unknown category hints; for example, the hint for the known category "airplane" is... The first unknown prompt is .

[0129] Step 4: Perform few-shot training based on known and unknown cues: For a training image x labeled "airplane", normalized image features are obtained using the CLIP image encoder. The CLIP text encoder is used to obtain text features of known categories. and unknown prompt text features The cosine similarity between the image and the text prompt is calculated. Here, the superscript K represents the known category branch, and the superscript U represents the unknown prompt branch.

[0130] ,

[0131] For example, if the image has a similarity of 6 known category hints. The similarity with the first three of the 50 unknown clues is Then, the class probability can be obtained by normalizing the joint set of known and unknown prompts. For ease of display, use... Indicates the first Given the similarity of the prompts, use Indicates the first The similarity of the unknown prompts.

[0132] ; ; If the true class of the training image is "airplane", and the normalized result is the true class probability... The probabilities of the first three unknown prompts are respectively , , The unknown perception assistance loss is calculated using the following formula, and the remaining unknown prompt probabilities are substituted in the same way.

[0133]

[0134]

[0135] in, For the unknown label smoothing factor, this embodiment takes... ; The combined weights of the closed-set recognition loss and the unknown perception auxiliary loss are taken in this embodiment. During training, a stochastic gradient descent optimizer was used with a batch size of 32 and an initial learning rate of 0.002. Cosine annealing was used to decay the learning rate over 50 training epochs.

[0136] Step 5 performs boundary regularization on known and unknown semantics: In order to keep the synthesized unknown semantic anchor points away from the known category semantic regions and to avoid multiple unknown prompts collapsing into the same local region, this embodiment introduces known-unknown separation loss and unknown-unknown discrete loss.

[0137]

[0138]

[0139]

[0140]

[0141] In this embodiment, m= =0.2, =0.5. For example, if an unknown prompt has similarities of 0.10, 0.18, and 0.27 with known categories such as airplane, automobile, and bird, respectively, then a similarity of 0.27 exceeds the interval of 0.2, resulting in a separation loss term max(0, 0.27). 0.2) = 0.07. If the similarity between two unknown prompts is 0.31, then a discrete loss term max(0, 0.31) is generated. 0.2) = 0.11. When only the two terms violating the margin constraint are illustrated, the semantic anchor regularization contribution can be expressed as 0.5 × (0.07 + 0.11) = 0.09; in actual training, and Calculate the average of all known and unknown hints.

[0142] Step 6: In the reasoning stage, perform conflict-aware evidence reasoning and output the recognition result: For the image to be recognized x, first calculate its similarity with known category hints and unknown hints, and convert it into a non-negative evidence value through the Softplus activation function.

[0143]

[0144]

[0145]

[0146]

[0147] Under the Dempster-Shafer evidence theory, let's establish an identification framework. Where ID represents a known class hypothesis, OOD represents an unknown class hypothesis, and prior quality is introduced. This indicates uncertainty when evidence is insufficient. This embodiment takes... .

[0148]

[0149]

[0150]

[0151]

[0152]

[0153]

[0154]

[0155] For example, if the maximum known evidence for a test image =5.0, Maximum Unknown Evidence =2.0, and Therefore, S = 5.0 + 2.0 + 10 = 17.0. =5.0 / 17.0≈0.294, =2.0 / 17.0≈0.118, =10 / 17.0≈0.588, =0.294×(1 0.118)≈0.259. If the open set rejection threshold τ=0.2 is set, then if Score>τ, the image is determined to be of a known category, and the category with the highest similarity among the known branches in {airplane, automobile, bird, cat, deer, dog} is further selected as the output.

[0156] For example, if the strongest known evidence for another test image... =1.5, maximum unknown evidence =6.0, then S = 1.5 + 6.0 + 10 = 17.5. ≈0.086, ≈0.343, ≈0.571, Score=0.086×(1 0.343)≈0.057. Since Score≤τ, the image is rejected as an unknown category. Therefore, this embodiment can maintain the ability to identify known categories while improving the reliability of rejecting unknown categories and samples with blurred boundaries, using only a small number of samples labeled with known categories.

[0157] It should be noted that the above-mentioned category division, auxiliary vocabulary, probability, intermediate evidence value and threshold are all exemplary simulation values; in different datasets, different numbers of known categories, different distributions of unknown categories or different deployment scenarios, the category set, auxiliary vocabulary size, number of unknown prompts, number of semantic primitives, regularization weight and rejection threshold can be adjusted according to task requirements.

[0158] Furthermore, simulation experiments were conducted on the few-sample open set recognition of this application: The experimental datasets used were C10, C10+10, C10+50, C100, T200, T200-H, M100, and IN200. See Table 1 for details. Table 1 Experimental Data Sets and Category Classification

[0159] The visual language pre-trained model uses CLIP ViT-B / 16; number of context tokens Number of semantic primitives Unknown number of prompts The combined weight matrix is ​​initialized with a Gaussian distribution with a mean of 0 and a standard deviation of 0.02. , , , , Sparse dictionary learning adopts... The maximum number of iterations is 1000, and the solution uses minimum angular regression. The standard deviation of the combined weight perturbation during training is set to 0.1. The optimizer employs stochastic gradient descent with a batch size of 32, an initial learning rate of 0.002, and uses cosine annealing to decay over 50 training epochs. The above values ​​are only example simulation parameters and are not intended to limit the protection range.

[0160] In the 16-shot small sample setting, the method of this application is compared with the representativeness same setting method (UTL), as shown in Table 2: Table 2 Comparison results between this application and UTL under 16-shot settings.

[0161] With a 16-shot few-sample setting, the area under the open set detection curve (AUROC) of the method in this application is 87.85%, 90.03%, 85.62%, 96.27%, and 97.79% on the C100, T200, T200-H, M100, and IN200 benchmarks, respectively; and the known class recognition accuracy (ACC) on the C10, C10+50, C100, T200-H, M100, and IN200 benchmarks is 97.92%, 98.44%, 88.42%, 85.77%, 95.51%, and 82.34%, respectively.

[0162] To further verify the effectiveness of the proposed method under different training data scales, experiments were conducted to compare the 16-shot and all-data versions with traditional OSR methods and CLIP-based OSR methods. As shown in Table 3: Table 3 Comparison of core results between this application and UTL (all-data)

[0163] Under the all-data setting, the proposed method achieves the best AUROC on C10, C10+50, T200, and IN200, and only slightly lags behind ARPL+CS on C10+10; it also achieves the best ACC on all five listed benchmarks. Even with a significantly reduced training sample size, the 16-shot version still meets or exceeds the UTLall-data results in AUROC for C10, ACC for C10+50, and AUROC / ACC for IN200, indicating that the performance improvement is not solely due to the increased training data size.

[0164] Visual experiments were conducted to verify the method of this application and UTL on the C100 dataset: like Figures 10-11 As shown, the unknown hints generated by UTL are mainly concentrated in local regions far from the main data manifold, with limited coverage of inter-category transition regions and complex boundaries. The open hints generated by the method in this application are more widely distributed in the shared visual language embedding space, especially covering the areas around known category clusters and boundary regions. This demonstrates that semantic anchor synthesis and regularization strategies can construct a more structured unknown semantic space.

[0165] like Figures 12-13 As shown, the distribution of open set decision scores further indicates that there is still significant overlap between the scores of known and unknown samples in UTL. After adopting conflict-aware evidence scores in the method of this application, the overlapping area is reduced, and the separation of known / unknown samples is clearer. This result supports the claim that conflict-aware evidence reasoning in this application can suppress overconfidence on boundary samples and improve the reliability of open set decision.

[0166] Furthermore, this application also provides a few-shot open set identification system for performing the above-described few-shot open set identification method, with reference to... Figure 14 As shown, the system may include: The data acquisition module is used to acquire training images containing known category labels, as well as a set of known category names, a set of auxiliary text vocabularies, and a visual language pre-trained model; the visual language pre-trained model includes an image encoder and a text encoder. The semantic extraction module is used to tokenize the auxiliary text vocabulary set to obtain effective token embeddings, and extract multiple semantic primitives from all the effective token embeddings through dictionary learning, and normalize them to obtain a semantic primitive matrix. The semantic computation module is used to set learnable combination weights for multiple unknown semantic anchors to be synthesized, and to add random perturbation during the training phase. The learnable combination weights with random perturbation are weighted and summed on the semantic primitives in the semantic primitive matrix to obtain multiple unknown semantic anchors. The feature encoding module is used to place the known category name and the unknown semantic anchor into a shared prompt structure containing learnable context vectors, respectively, to obtain known category text prompts and unknown category text prompts, and to encode them into category text features by the text encoder; the category text features include known category text features and unknown category text features; The parameter update module is used to construct a training objective including unknown perception calibration loss, known and unknown semantic separation loss and unknown semantic dispersion loss based on the consistency measure between the image features of the training image and the category text features, and to update the learnable context vector and the learnable combined weights through the training objective; The label determination module is used to acquire image features of the image to be identified during the inference stage, generate evidence values ​​based on the consistency measure between the image features of the image to be identified and the category text features, assign quality values ​​based on the evidence values, and determine category labels based on the quality values; the evidence values ​​include known evidence and unknown evidence, and the quality values ​​include known quality, unknown quality, and unknown quality.

[0167] Furthermore, this application also provides an electronic device, including: processor; Memory for storing the executable instructions of the processor; The processor is configured to perform the steps of the few-sample open set identification method described above by executing the executable instructions.

[0168] Furthermore, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0169] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0170] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. This application is not limited to the exact structures described above and illustrated in the accompanying drawings, and it should not be considered that the specific implementation of this application is limited to these descriptions. For those skilled in the art, various changes and modifications made without departing from the concept of this application should be considered to fall within the protection scope of this application.

Claims

1. A method for identifying open sets with few samples, characterized in that, include: Acquire training images containing known category labels, as well as a set of known category names, a set of auxiliary text vocabularies, and a visual language pre-trained model; the visual language pre-trained model includes an image encoder and a text encoder; The auxiliary text vocabulary set is tokenized to obtain effective token embeddings, and multiple semantic primitives are extracted from all the effective token embeddings through dictionary learning, and normalized to obtain a semantic primitive matrix. Learnable combination weights are set for multiple unknown semantic anchors to be synthesized, and random perturbations are added during the training phase. The learnable combination weights with random perturbations are weighted and summed on the semantic primitives in the semantic primitive matrix to obtain multiple unknown semantic anchors. The known category name and the unknown semantic anchor are respectively placed into a shared prompt structure containing learnable context vectors to obtain known category text prompts and unknown category text prompts, and then encoded by the text encoder to obtain category text features; the category text features include known category text features and unknown category text features; Based on the consistency measure between the image features of the training images and the category text features, a training objective is constructed that includes unknown perception calibration loss, known and unknown semantic separation loss, and unknown semantic dispersion loss. The learnable context vector and the learnable combined weights are then updated through the training objective. During the inference phase, image features of the image to be identified are acquired, evidence values ​​are generated based on the consistency measure between the image features of the image to be identified and the category text features, quality values ​​are assigned based on the evidence values, and category labels are determined based on the quality values; the evidence values ​​include known evidence and unknown evidence, and the quality values ​​include known quality, unknown quality, and unknown quality.

2. The method for identifying few-sample open sets according to claim 1, characterized in that, The step of setting learnable combination weights for multiple unknown semantic anchors to be synthesized, adding random perturbations during the training phase, and then weighting and summing the learnable combination weights with random perturbations on the semantic primitives in the semantic primitive matrix to obtain multiple unknown semantic anchors includes: Determine the preset number of unknown semantic anchors to be synthesized, and initialize a learnable combination weight for each unknown semantic anchor; the dimension of each learnable combination weight is equal to the number of semantic primitives in the semantic primitive matrix; In each forward computation during the training phase, a random perturbation is independently sampled for each learnable combined weight, and the random perturbation is added to the learnable combined weight to obtain the perturbed combined weight. The semantic primitives in the semantic primitive matrix are weighted and summed using the perturbed combined weights to obtain the initial vectors corresponding to each unknown semantic anchor. The initial vectors are normalized by their magnitudes to obtain multiple unknown semantic anchors.

3. The method for identifying few-sample open sets according to claim 1, characterized in that, The step of placing the known category name and the unknown semantic anchor into a shared prompt structure containing learnable context vectors to obtain known category text prompts and unknown category text prompts, and then encoding the category text features through the text encoder, includes: Obtain the shared cue structure; the shared cue structure contains multiple learnable context vectors; For each known category name, the start token, the learnable context vector, the embedding vector of the known category name, and the end token are concatenated in sequence to obtain the known category text prompt corresponding to the known category; For each unknown semantic anchor, the start token, the learnable context vector, the unknown semantic anchor, and the end token are concatenated in sequence to obtain the unknown category text prompt corresponding to the unknown category; All known category text prompts and all unknown category text prompts are input into the text encoder for encoding, thereby obtaining the known category text features and the unknown category text features.

4. The method for identifying few-sample open sets according to claim 1, characterized in that, The step of constructing a training objective, comprising unknown perception calibration loss, known-unknown semantic separation loss, and unknown semantic dispersion loss, based on the consistency measure between the image features of the training images and the category text features, and updating the learnable context vector and the learnable combined weights through the training objective, includes: The training image is input into the image encoder to obtain the image features; The similarity between the image features and each of the known category text features is calculated to obtain a known matching score set; the similarity between the image features and each of the unknown category text features is calculated to obtain an unknown matching score set. An unknown perception calibration loss is constructed based on the known matching score set and the unknown matching score set; the unknown perception calibration loss is used to preserve the response of each unknown category branch while supervising the classification of known categories; A known-unknown semantic separation loss is constructed based on the known category text features and the unknown category text features; the known-unknown semantic separation loss is used to reduce the similarity between any known category text features and any unknown category text features; An unknown semantic dispersion loss is constructed based on the pairwise similarity between each of the unknown category text features; the unknown semantic dispersion loss is used to reduce the similarity between different unknown category text features. The unknown perception calibration loss, the known and unknown semantic separation loss, and the unknown semantic dispersion loss are weighted and summed to obtain the total training objective. The learnable context vector and the learnable combination weights are then updated based on the total training objective.

5. The method for identifying few-sample open sets according to claim 4, characterized in that, The known and unknown semantic separation loss is: in, For the separation loss of known and unknown semantics, The number of prompts is unknown. Given the number of categories, For the first Unknown category text features For the first Known category text features, This represents the similarity boundary between known and unknown semantics. The unknown semantic dispersion loss is: in, For unknown semantic dispersion loss, The similarity boundary between different unknown semantics; The unknown perception calibration loss is: in, , ， ； Calibration loss for unknown perception Given the class probabilities, For truly known categories, This represents the trade-off coefficient between known category discrimination and unknown perception calibration. For known class identification - unknown branch joint activation loss, For unknown information, auxiliary supervision weights are provided. The probability of this is unknown. For image samples With the Matching scores for known category text features. For temperature parameters, For image samples With the Matching scores for known category text features. For image samples With the Matching scores for text features of unknown categories; The overall training objective is: in, For the overall training objective, This is the regularization weight.

6. The method for identifying few-shot open sets according to claim 1, characterized in that, The steps of acquiring image features of the image to be identified during the inference phase, generating evidence values ​​based on the consistency measure between the image features of the image to be identified and the category text features, assigning quality values ​​based on the evidence values, and determining category labels based on the quality values ​​include: During the inference phase, the image to be identified is input into the image encoder for encoding to obtain the image features of the image to be identified; The similarity between the image features and each of the known category text features is calculated to obtain a set of known matching scores for the image to be identified; and the similarity between the image features and each of the unknown category text features is calculated to obtain a set of unknown matching scores for the image to be identified. Each score in the known matching score set is converted into a non-negative known evidence value, and each score in the unknown matching score set is converted into a non-negative unknown evidence value. The maximum value among all known evidence values ​​is selected as the strongest known evidence, and the maximum value among all unknown evidence values ​​is selected as the strongest unknown evidence. Based on the strongest known evidence, the strongest unknown evidence, and the preset prior quality, a known quality, an unknown quality, and an unknown quality are assigned; wherein, the sum of the known quality, the unknown quality, and the unknown quality is 1; A conflict perception score is calculated based on the known quality and the unknown quality; the conflict perception score has an inhibitory effect on the known quality when the unknown quality increases. When the conflict perception score is greater than the preset rejection threshold, the image to be identified is determined to be of a known category, and the known category label corresponding to the maximum score in the known matching score set is output; otherwise, the image to be identified is determined to be of an unknown category, and the unknown category determination is output.

7. The method for identifying few-sample open sets according to claim 6, characterized in that, The known mass is: in, , , ; Given the mass, The strongest known evidence, For the amount of evidence, Given the known evidence value, For image samples With the Matching scores for known category text features. Preset prior quality; The unknown mass is: in, , , For unknown mass, The strongest piece of unknown evidence, Value of unknown evidence. For image samples With the Matching scores for text features of unknown categories; The quality of ignorance is: in, Quality of ignorance.

8. A few-sample open set identification system, characterized in that, A method for performing the few-sample open set identification method according to any one of claims 1-7 includes: The sample acquisition module is used to acquire training images containing known category labels, as well as a set of known category names, a set of auxiliary text vocabulary, and a visual language pre-training model; the visual language pre-training model includes an image encoder and a text encoder; The semantic extraction module is used to tokenize the auxiliary text vocabulary set to obtain effective token embeddings, and extract multiple semantic primitives from all the effective token embeddings through dictionary learning, and normalize them to obtain a semantic primitive matrix. The semantic computation module is used to set learnable combination weights for multiple unknown semantic anchors to be synthesized, and to add random perturbation during the training phase. The learnable combination weights with random perturbation are weighted and summed on the semantic primitives in the semantic primitive matrix to obtain multiple unknown semantic anchors. The feature encoding module is used to place the known category name and the unknown semantic anchor into a shared prompt structure containing learnable context vectors, respectively, to obtain known category text prompts and unknown category text prompts, and to encode them into category text features by the text encoder; the category text features include known category text features and unknown category text features; The parameter update module is used to construct a training objective including unknown perception calibration loss, known and unknown semantic separation loss and unknown semantic dispersion loss based on the consistency measure between the image features of the training image and the category text features, and to update the learnable context vector and the learnable combined weights through the training objective; The label determination module is used to acquire image features of the image to be identified during the inference stage, generate evidence values ​​based on the consistency measure between the image features of the image to be identified and the category text features, assign quality values ​​based on the evidence values, and determine category labels based on the quality values; the evidence values ​​include known evidence and unknown evidence, and the quality values ​​include known quality, unknown quality, and unknown quality.

9. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to perform the steps of the few-sample open set identification method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the few-sample open set identification method according to any one of claims 1-7.

Images