Zero-shot object detection method based on semantic perception and adaptive contrast learning

Abstract

The application belongs to the technical field of machine learning, and specifically relates to a zero-shot target detection method based on semantic perception and adaptive contrast learning. The application adopts DETR based on a transformer network as a target detection model framework, and introduces a semantic perception module based on attention to alleviate the bias of a mapping type zero-shot target detection model to seen classes; and adopts an adaptive contrast loss based on metric learning, so that the model learns better feature representation while maintaining the semantic structure relationship between classes, and further improves the detection performance. Experimental results show that, compared with existing methods, the application has better performance and generalization ability in the zero-shot target detection task, and has a wide application prospect.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, specifically relating to a zero-shot object detection method based on semantic awareness and adaptive contrastive learning. Background Technology

[0002] Over the past decade, deep learning-based computer vision has made tremendous progress on many tasks. However, most of the proposed methods are fully supervised, relying on large amounts of labeled data and only able to infer objects that appear in the training set. In contrast, the human visual system can easily infer objects it has never seen before based on linguistic descriptions of new objects. For example, a human can easily identify a zebra, which they have never seen before, based on the description "an animal with black and white fur like a panda, stripes like a tiger, and a body shape like a horse." However, this human visual ability poses a significant challenge to deep learning models. Therefore, zero-shot learning—aiming to infer new targets not seen during training—has received widespread attention from the machine learning and computer vision communities in recent years.

[0003] Most existing zero-shot learning work focuses on zero-shot recognition (ZSR), which classifies only salient objects in an image. However, this does not reflect real-world scenarios. In reality, an image may contain multiple objects of different categories. Furthermore, in addition to the object's category, we may also be interested in the object's location within the image. Therefore, this invention addresses a more realistic problem: zero-shot object detection (ZSD), which aims to simultaneously identify and locate novel objects not present in the training data.

[0004] Zero-shot object detection is defined as follows: In zero-shot object detection, the dataset is divided into visible and unseen classes. Visible classes are used for training, and unseen classes are used for testing and validation. Each category in both visible and unseen classes provides a semantic vector describing that category. Most existing methods use Region Feature Proposal Networks (RPNs) to generate candidate boxes, and then use cosine similarity to calculate the similarity between the candidate box features and the category features, thus completing the candidate box classification. Depending on the scenario, the testing phase of zero-shot object detection can generally be divided into two settings: narrow zero-shot object detection (ZSD) and generalized zero-shot object detection (GZSD). Narrow zero-shot object detection is defined as testing images containing only objects from unseen categories; generalized zero-shot object detection is defined as testing images containing both seen and unseen categories. It can be seen that generalized zero-shot object detection is closer to real-world scenarios and is more challenging.

[0005] Previous zero-shot object detection methods can be broadly categorized into two types based on how semantic vectors are used: mapping-based zero-shot object detection methods and generative zero-shot object detection methods. Mapping-based methods directly map visual features to the semantic space and use the category corresponding to the nearest semantic vector as the classification result; generative methods synthesize visual features of unseen categories based on semantic vectors, thereby training a fully connected classifier.

[0006] Because of the lack of data on unseen categories during training, mapping-based zero-shot object detection methods tend to map the visual features of unseen categories to the vicinity of the semantic vectors of seen categories, thus classifying unseen categories as seen categories. In other words, mapping-based methods exhibit bias towards seen categories. In contrast, generative methods inject the semantic features of unseen categories into the classifier through generative models, thereby mitigating this bias. However, generative methods rely on complex and time-consuming multi-step training processes, limiting their application. This invention aims to provide a mapping-based zero-shot object detection method that can alleviate the bias problem.

[0007] Furthermore, while previous methods have attempted to leverage contrastive learning to help detection models learn discriminative feature representations to improve their detection performance, they have neglected the semantic structural relationships between categories, resulting in only minor performance gains. Another contribution of this invention is the proposal of an adaptive contrastive learning loss to help the model learn features that are both discriminative and preserve semantic structural relationships.

[0008] Finally, previous zero-shot object detection methods relied on detectors built from convolutional neural networks, requiring non-maximum suppression (NMS) as a subsequent processing operation, thus failing to achieve fully end-to-end object detection. However, with the rise of the transformer architecture (a neural network structure utilizing attention mechanisms), object detection algorithms based on the transformer architecture, such as DETR, have successfully applied the transformer to object detection without relying on NMS, thereby achieving end-to-end object detection. Currently, the best-performing object detection algorithms are all modified from DETR, such as DINO and Deformable DETR. Therefore, the zero-shot object detection method proposed in this invention chooses DETR as its detection framework. Summary of the Invention

[0009] The purpose of this invention is to provide a zero-shot object detection method based on semantic awareness and adaptive contrastive learning, so as to alleviate the bias problem of existing mapping-based methods for known categories, and maintain the semantic structural relationship between different categories in the feature space, thereby improving detection performance.

[0010] The zero-shot object detection method based on semantic awareness and adaptive contrastive learning provided by this invention employs several novel techniques, including constructing a DETR-based zero-shot object detection model; constructing an attention-based semantic awareness module; and using an adaptive contrastive loss based on metric learning. The specific steps are as follows:

[0011] (1) Definition of the zero-shot target detection problem;

[0012] Suppose there are two non-overlapping sets of categories: the seen category... and unseen categories Where S is the number of seen classes and U is the number of unseen classes; That is, the seen categories and the unseen categories belong to two disjoint sets; during the training phase, the training set consists of a large number of samples from the seen categories.

[0013]

[0014] Where, x i It is the input image; Representing the image x i The category label for each known category object j; Representing the image x i The bounding box coordinates of each seen category object j in N; tr It is the training set The number of samples in the sample;

[0015] Since there are no unseen class samples available for training in the zero-shot learning setting, the semantic description W = [W s W u ], that is, word embedding vectors learned from large-scale language sources or manually designed attributes are used to bridge seen and unseen categories, where W s ∈R S×d W u ∈R U×d Let S represent the semantic vectors of seen and unseen categories, respectively, where d is the embedding dimension of the semantic vectors, and S is the number of seen categories and U is the number of unseen categories.

[0016] (2) Constructing a zero-shot target detection model

[0017] This invention uses DETR based on transformer networks as the target detection model framework and integrates the corresponding modules proposed in this invention. The structure of the DETR target detection model can be divided into two parts: feature extraction and prediction of the detection set.

[0018] Feature extraction; Given an image x, DETR uses a convolutional neural network (CNN) to extract local features f. l :

[0019] f l =CNN(x). (2)

[0020] Next, the model uses the encoder Enco from the transformer network. d er further extracts global features fg:

[0021] f g = E ncoder(f l (3)

[0022] Ensemble prediction; DETR treats object detection as an ensemble prediction problem and directly infers a fixed-size prediction ensemble based on a learnable query vector q. First, a self-attention layer (MSA) is used to compute the embedding representation e of the target query vector q. q :

[0023] h q =MSA(q, q, q), (4)

[0024] e q =LN(h q +q), (5)

[0025] Where LN represents layer normalization, h q This represents the intermediate calculation result; next, the embedding vector e is calculated. q and global features f g The cross-attention layer MCA between the two yields the fused features e. f :

[0026] h f =MCA(e q f g f g (6)

[0027] e f =LN(h f +e q (7)

[0028] Among them, h f This is an intermediate calculation result; then, feature e f The input is fed into the feedforward network FFN (composed of fully connected networks) module to obtain the final embedding representation v of the target query vector q:

[0029] h = FFN(e f(8)

[0030] v=LN(h+e f (9)

[0031] Where h represents the intermediate calculation result; in order to apply DETR to zero-shot object detection tasks, this invention employs a visual semantic alignment layer FC. VSL (Achieved via a fully connected layer) Replace the last classification layer in DETR to map visual features to a semantic feature space, resulting in embedded features s in the semantic space:

[0032] s=FC VSL (v), (10)

[0033] Consistent with previous mapping-based zero-shot object detection methods, this invention uses cosine similarity to measure the embedded features s and semantic vector W. * The distance between them is calculated, and the sigmoid function is applied to obtain the final classification score.

[0034]

[0035] Furthermore, consistent with the original DETR, this invention employs a regression network FC composed of a fully connected network. reg Obtain the target's position coordinates

[0036]

[0037] Thus, the final prediction set is:

[0038] (3) Furthermore, it also includes the training of the DETR object detection model;

[0039] To achieve the prediction set To achieve this between the baseline truth set y = (b, p), this invention employs the Hungarian matching algorithm to find a matching method that minimizes the matching loss.

[0040]

[0041] in, For all possible matching methods; matching loss Classification loss and regression loss Together they constitute:

[0042]

[0043] In the formula, λ cls , λbbox , λ iou , λ giou The proportional coefficients for each type of loss; To compare the losses, For generalized intersection and comparison of losses;

[0044] The model loss is calculated based on the matching results, and the model weights are optimized using the backpropagation algorithm.

[0045]

[0046] (4) Further, construct an attention-based semantic perception module.

[0047] However, simply using the fused features e between visual features and query vectors f This can lead to bias in the model towards known categories. To overcome this problem, and inspired by generative zero-shot object detection methods, this invention proposes a novel semantic awareness module, which is inserted into the ensemble prediction module of the object detection model, so that the model can incorporate features e. f During the mapping to the semantic space, semantic information of the category is injected into the original features through the semantic awareness module, thereby helping the model perceive possible unseen categories in the test image and alleviating the bias of zero-shot object detection methods based on the mapping framework towards seen categories. Specifically, this invention uses a cross-dot product attention network to implement the semantic awareness module, in order to maintain model simplicity and better integrate it into the DETR framework; considering the original embedded features e f As the query vector, the semantic vector W * As key and value vectors in the key-value attention mechanism, they are input into the semantic awareness module proposed in this invention to obtain enhanced feature embeddings e. w :

[0048] h w =SA(e f W * W * )h w =W * softmax(e f W * (16)

[0049] e w =LN(h w +e f ), (17)

[0050] Where SA represents the semantic awareness module implemented through the dot product attention mechanism; W * W s / W u / [Ws W u ], corresponding to the semantic vectors of the classes seen during training / the semantic vectors of the classes not seen during testing in narrow zero-shot object detection / the semantic vectors of the classes seen and not seen during testing in generalized zero-shot object detection; softmax represents the normalization exponential function; h w This is an intermediate calculation result; through the semantic awareness module, the model obtains the enhanced embedded feature e. w This allows for the perception of potentially unseen categories when mapping visual features to the semantic space, thereby alleviating the bias problem of previous mapping-based zero-shot object detection methods towards seen categories;

[0051] Then, based on the enhanced feature e w Perform ensemble prediction instead of the original feature e f The computational process and model building are kept consistent; that is, the semantically enhanced features e are used in the same way. w The input is fed into the feedforward network FFN module to obtain the final embedding representation v of the target query vector q:

[0052] h = FFN(e w (18)

[0053] v=LN(h+e w (19)

[0054] Then, the visual features are mapped to the semantic feature space through a visual-semantic alignment layer to obtain the embedded features s in the semantic space:

[0055] s=FC VSL (v), (20)

[0056] Next, cosine similarity is used to measure the embedding features s and semantic vector W. * The distance between them is calculated, and the sigmoid function is applied to obtain the final classification score:

[0057]

[0058] The target's position coordinates are obtained using a fully connected network.

[0059]

[0060] Thus, the final prediction set is obtained.

[0061] (5) Furthermore, it also includes the design of adaptive contrastive loss based on metric learning.

[0062] To help the proposed detection model learn good feature representations and further improve detection performance, this invention also proposes an adaptive contrastive loss based on metric learning to help the model better transfer from seen categories to unseen categories. The key idea behind standard contrastive learning is to make embedded feature pairs from the same category as close as possible and to make embedded feature pairs from different categories as far apart as possible. Given the i-th embedded feature in category c... and embedded feature sets with the same category labels and an embedded feature set with different category labels (including background category). The standard contrastive loss function is shown in the following equation:

[0063]

[0064] Where τ is the temperature parameter, which can be set to 0.1 by default; exp represents the exponential function. Therefore, the overall contrastive loss of M embedded features... It can be represented as:

[0065]

[0066] Where c(i) represents the embedded feature s i The category.

[0067] However, this invention argues that standard contrastive loss functions cannot accurately reflect the semantic structural relationships between categories, and are therefore unsuitable for zero-shot object detection tasks. Specifically, standard contrastive loss functions encourage equal distances between all category pairs, regardless of the inherent semantic distance between categories. However, in semantic space, the distance between some category pairs may be closer or farther than that between others. For example, the distance between "dog" and "wolf" in the category semantic space may be closer than the distance between "dog" and "cat". Therefore, equating the mapped distance between "dog" and "cat" with the mapped distance between "dog" and "wolf" is unreasonable. Figure 3 As shown. To solve the above problems, this invention introduces an adaptive weight μ. c-c′ The labeled contrastive loss function is then extended to the following form:

[0068]

[0069] Where, μ c-c′ It is an adaptive weight between categories c and c′, specifically depending on the similarity between their semantic vectors. This invention introduces an adaptive weight μ. c-c′ The subsequent contrastive loss function is the adaptive contrastive loss function. When μ c-c′ When μ ≡ 1, the adaptive contrastive loss function degenerates into the standard contrastive loss function. c-c′The specific form of the adaptive weight μ can theoretically be any function that monotonically decreases with respect to similarity, but for the sake of simplicity, Ennio invented an exponential form to represent the adaptive weight μ. c-c′ :

[0070] μ c-c′ =exp(-cos(w) c w c′ (26)

[0071] Among them, w c and w c′ are the semantic vectors of category c and category c′, respectively; cos represents the cosine function. Therefore, the final overall loss of the model is as follows:

[0072]

[0073] After calculating the total loss in each iteration, the model weights are optimized using the backpropagation algorithm.

[0074] (6) Inference of the target detection model:

[0075] During inference, directly use the model prediction set The top k highest-scoring targets are used as the output, i.e., for the predicted target (p) i b i Sort the data so that:

[0076] p1≥p2≥p3≥…>p N ,

[0077] The final output prediction set is Where k is a manually set parameter, typically set to 300. Because the zero-shot target detection method of this invention is based on the DETR model, non-maximum suppression is not required as a subsequent processing operation, thus achieving end-to-end zero-shot target detection.

[0078] The present invention has at least the following beneficial effects:

[0079] (1) Reduce the bias of the mapping zero-shot target detection method on the seen categories.

[0080] In the ablation experiment (as shown in Table 2), the semantic perception module proposed in this invention improved the detection performance of unseen categories compared with the baseline model, while the performance of seen categories remained basically unchanged, thereby narrowing the performance difference between unseen and seen categories, i.e. reducing the bias of the detection model on seen categories.

[0081] (2) It helps the detection model learn discriminative feature representations while maintaining the semantic structure relationship between categories.

[0082] like Figure 4This invention visualizes the feature representations of different categories learned by the model, showing that after introducing contrastive learning ( Figure 4 b) Compared to the model without contrastive learning (4.a), the model can learn more discriminative feature representations (features between different categories are separated); while the adaptive contrastive learning (4.c) proposed in this invention, on the basis of learning discriminative features, also makes the distance between features of different categories and their semantic distance positively correlated, that is, it maintains the semantic structural relationship between categories.

[0083] (3) A mapping-based zero-shot target detection method based on semantic perception and adaptive contrastive learning is simple and efficient;

[0084] Compared to previous generative zero-shot object detection methods that required complex and time-consuming multi-step training processes (training of the localizer, offline feature extraction, generator training, and classifier training), this invention can directly complete model training in one step. Furthermore, this invention's method is based on DETR and does not require subsequent processing steps such as non-maximum suppression, thus making it a completely end-to-end zero-shot object detection method. Attached Figure Description

[0085] Figure 1 This is a framework diagram of the zero-shot target detection method based on semantic awareness and adaptive contrastive learning of this invention.

[0086] Figure 2 The diagram shows the structure of the semantic awareness module in this invention. (a) is a schematic diagram of a previous mapping-based zero-shot target detection framework, (b) is a diagram of a generative zero-shot target detection framework, and (c) is a diagram of the mapping-based zero-shot target detection framework based on the semantic awareness module proposed in this invention.

[0087] Figure 3 The diagram illustrates the adaptive contrastive learning method in this invention. (a) is a diagram of standard contrastive learning, (b) is a diagram of the adaptive contrastive learning method proposed in this invention, and (c) is a diagram of the category semantic space.

[0088] Figure 4 The diagrams are visualizations of features learned through adaptive contrastive loss. (a) shows a visualization of features without any contrastive loss, (b) shows a visualization of features with standard contrastive loss, and (c) shows a visualization of features with adaptive contrastive loss.

[0089] Figure 5 This is a visual illustration of the detection results of the present invention on a benchmark dataset. Detailed Implementation

[0090] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0091] This invention uses the COCO and VOC datasets as average metrics for model performance. Following the common practice of zero-shot object detection, the COCO dataset was divided into two data partitions: 48 / 17 and 65 / 15 for seen / unseen categories; the VOC dataset was divided into one data partition: 16 / 4 for seen / unseen categories.

[0092] This invention uses the average detection accuracy (AP) at an intersection-to-union ratio (IoU) of 0.5 as the average index of the model's detection performance, and tests the average detection accuracy of unseen categories under the narrow zero-sample target detection setting, as well as the average detection accuracy of seen and unseen categories under the generalized zero-sample target detection setting.

[0093] This invention employs ResNet50 as the feature extractor, constructing multi-scale feature maps from the outputs of its last three layers, and uses DINO (a variant of the DETR detector) as our detection framework. For model optimization, we use the AdamW optimizer, setting its learning rate and weight decay coefficient to 0.0001. The proposed model was trained for 20,000 and 30,000 epochs on the COCO dataset with two class splits (48 / 17 and 65 / 15), respectively; and for 20,000 epochs on the VOC dataset. The adaptive contrastive loss coefficients were set to 0.02 / 0.03 / 0.1, respectively. cls , λ iou , λ giou Set them to 1.0, 5.0 and 2.0 respectively.

[0094] This invention tested the performance of two test scenarios, narrow zero-shot object detection and generalized zero-shot object detection, on the COCO and VOC datasets. The experimental results are shown in Table 1.

[0095] Table 1 shows the experimental results of this invention. All indicators refer to the average detection accuracy (AP) at IoU = 0.5, S represents the seen category, U represents the unseen category, and HM represents the harmonic mean of the seen and unseen categories.

[0096] Table 1

[0097]

[0098] To intuitively understand the impact of the two components proposed in this invention on detection performance, we conducted ablation experiments under the COCO 48 / 17 category classification, and the results are shown in Table 2:

[0099] Table 2 Ablation experiments. All indicators refer to the mean detection precision (AP) at IoU = 0.5, S represents the seen class, U represents the unseen class, and HM represents the harmonic mean of the seen and unseen classes. Ablation experiments were performed on the COCO 48 / 17 class classification.

[0100] Table 2

[0101]

[0102]

[0103] Comparing the first and second models in the ablation experiment results table, it can be seen that the semantic perception module proposed in this invention can significantly improve the detection performance of unseen categories under the generalized zero-sample target detection setting (the average detection accuracy is improved by 1 percentage point), without significantly impairing the detection performance of seen categories, thus narrowing the gap between the two and alleviating the model's bias towards seen categories.

[0104] Furthermore, comparing the third and fourth models in the table reveals that while the standard contrastive learning loss improves model performance, the improvement is not significant because it ignores the semantic structural relationships between categories. In contrast, the adaptive contrastive loss proposed in this invention considers the semantic constraints between categories, thus helping the detection model achieve a more significant performance improvement.

[0105] In addition, such as Figure 4 As shown, we visualized the categorical features learned by the model. 4.a is the feature map without contrastive loss, 4.b is the feature map with standard contrastive loss, and 4.c is the feature map with the adaptive contrastive loss proposed in this invention. It can be seen that although the standard contrastive loss learns discriminative features, it ignores the semantic relationships between categories; while the adaptive contrastive loss proposed in this invention not only learns discriminative features but also maintains the semantic structural relationships between categories (such as...). Figure 4 Features of "airplane" and "bus" in .c.

[0106] Finally, we also provide some visualization examples of the detection results, such as Figure 5 As shown, the effectiveness of the zero-shot target detection algorithm proposed in this invention can be clearly seen.

[0107] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A zero-shot target detection method based on semantic awareness and adaptive contrastive learning, characterized in that, The specific steps are as follows: (1) Definition of zero-shot target detection problem Suppose there are two non-overlapping sets of categories: the seen category... and unseen categories ,in, It is the number of classes that have already been seen. It is the number of unseen classes; That is, the seen categories and the unseen categories belong to two disjoint sets; during the training phase, the training set consists of a large number of samples from the seen categories. : （1） in, It is the input image; Representing an image Each seen category object Category labels; , representing an image Each seen category object The bounding box coordinates; It is the training set The number of samples in the sample; Since there are no unseen class samples available for training in the zero-shot learning setting, semantic description... That is, word embedding vectors learned from large-scale language sources or manually designed attributes are used to bridge seen and unseen categories, where These represent the semantic vectors of seen and unseen categories, respectively. It is the embedding dimension of the semantic vector; (2) Constructing a zero-shot target detection model We adopted DETR, a transformer-based network, as the object detection model framework and integrated the corresponding modules into it. The DETR object detection model has two parts: feature extraction and prediction of the detection set. (a) Feature extraction; given an image DETR uses a convolutional neural network (CNN) to extract local features. : ． （2） Next, the encoder in the transformer network is used to further extract global features. : ， （3） (b) Ensemble prediction; The DETR object detection model treats the object detection problem as an ensemble prediction problem and, based on the learnable query vector... Directly deduce a fixed size prediction set First, a self-attention layer (MSA) is used to calculate the target query vector. Embedded representation : （4） （5） Where LN represents layer normalization, This represents the intermediate calculation result; then the embedding vector is calculated. and global features The cross-attention layer MCA between the two obtains the fused features. : （6） （7） in, These are intermediate calculation results; The DETR object detection model is trained as follows: To achieve the prediction set With the benchmark set of truth For matching between pairs, the Hungarian matching algorithm is used to find the matching method that minimizes the matching loss. : （13） in, For all possible matching methods; matching loss Classification loss and regression loss Together they constitute: （14） In the formula, , , , The proportional coefficients for each type of loss; To compare the losses, For generalized intersection and comparison of losses; The model loss is calculated based on the matching results, and the model weights are optimized using the backpropagation algorithm. （15）； Construct an attention-based semantic awareness module and insert it into the ensemble prediction module of the object detection model so that the model can incorporate features. During the mapping to the semantic space, a semantic awareness module injects category semantic information into the original features, helping the model perceive potential unseen categories in the test image. This alleviates the bias of zero-shot object detection methods based on the mapping framework towards seen categories. Specifically, a cross-dot product attention network is used to implement the semantic awareness module to maintain model simplicity and better integrate it into the DETR framework. This takes into account the original embedded features. As a query vector, semantic vector As key and value vectors in the key-value attention mechanism, they are input into the semantic awareness module to obtain enhanced feature embeddings. : （16） (17) in, This represents a semantic awareness module implemented using a dot product attention mechanism. express , respectively correspond to the semantic vectors of the categories seen during training / the semantic vectors of the categories not seen during testing in narrow zero-shot object detection / the semantic vectors of the categories seen and not seen during testing in generalized zero-shot object detection; softmax represents the normalization exponential function; This is an intermediate calculation result; the enhanced embedded features are obtained through the semantic awareness module. This allows for the perception of potentially unseen categories when mapping visual features to the semantic space, thereby alleviating the bias problem of existing mapping-based zero-shot object detection methods towards seen categories; Features enhanced with semantic awareness The input is fed into the feedforward network FFN module to obtain the target query vector. The final embedding representation : （18） ， （19） Then, a visual-semantic alignment layer is used to map the visual features to the semantic feature space, resulting in embedded features in the semantic space. : ， （20） Next, cosine similarity is used to measure the embedded features. and semantic vectors The distance between them is calculated, and the sigmoid function is applied to obtain the final classification score: ， （21） The target's position coordinates are obtained using a fully connected network. （22） Thus, the final prediction set is obtained. .

2. The zero-sample target detection method according to claim 1, characterized in that, To enable the object detection model to learn good feature representations and further improve detection performance, an adaptive contrastive loss based on metric learning is adopted to help the model better transfer from seen categories to unseen categories. The idea behind standard contrastive learning is to bring feature embeddings from the same class as close as possible and to keep feature embeddings from different classes as far apart as possible; given a class The first in Embedded features and embedded feature sets with the same category labels. and embedding feature sets with different category labels The standard contrastive loss function is shown in the following equation: ， （23） in, It is a temperature parameter; exp represents an exponential function, therefore... Overall contrastive loss of embedded features Represented as: （24） in, Representing embedded features Category; Further introduce adaptive weights The labeled contrastive loss function is then extended to the following form: （25） in, It is a category and The adaptive weights between them depend specifically on the similarity between their semantic vectors; this is called introducing adaptive weights. The subsequent contrastive loss function is the adaptive contrastive loss function; when At that time, the adaptive contrastive loss function degenerates into the standard contrastive loss function; for Specifically, it adopts an exponential form: （26） in, and They are categories and categories The semantic vector; cos represents the cosine function; therefore, the final overall loss of the model is as follows: (s)， （27） After calculating the total loss in each iteration, the model weights are optimized using the backpropagation algorithm.

3. The zero-sample target detection method according to claim 2, characterized in that, The inference of the object detection model directly uses the model's prediction set. Center front The target with the highest score is used as the output, i.e., the predicted target. Sort the data so that: ； The final output prediction set is ,in Parameters set by humans.

Images