An efficient method for unsupervised domain adaptation object detection
By designing a domain query module and a category prototype alignment module, the UDA-DETR model was optimized, which solved the problem of decreased detection performance caused by domain offset, achieved more efficient cross-domain target detection, and improved the detector's recognition and feature representation capabilities in different domains.
Patent Information
- Application Number
- CN202411953308.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2044-12-27
AI Technical Summary
Traditional object detectors suffer from performance degradation when faced with domain shifts, especially when there are significant feature differences between training and test data. Existing unsupervised domain adaptation methods ignore the differences in feature distributions of different categories in different domains when aligning global features, leading to classification and localization errors.
The design incorporates a domain query module and a category prototype alignment module. Through adversarial loss and contrastive learning, the UDA-DETR model is optimized, enabling it to extract domain-invariant and category features in the encoder and decoder parts, and gradually generate global category prototype features, thus achieving more accurate domain-adaptive detection.
It significantly improves the model's detection performance in different domains, enhances its ability to distinguish between different categories of objects and acquire contextual information of global features, and achieves more efficient cross-domain object detection.
Smart Images
Figure CN119992044B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, specifically to an efficient method for unsupervised adaptive target detection. Background Technology
[0002] Object detection has always been a hot area of research in computer vision, aiming to predict the bounding box coordinates and class labels of objects of interest in images, which is crucial for everyday applications such as security monitoring and autonomous driving. Initially, most object detection methods employed two-stage detectors, involving the design of anchor boxes and prior boxes, extraction of regions of interest, and tedious post-processing steps such as non-maximum suppression (NMS). With advancements in research, one-stage detectors based on Transformers were proposed and quickly gained attention. These methods eliminate the need for manually designed prior boxes and cumbersome post-processing steps, thus greatly simplifying the detection process.
[0003] While state-of-the-art detection methods excel in streamlining processes and improving performance, they typically rely on large amounts of labeled data for training. Furthermore, the performance of most detectors degrades significantly when there are substantial feature differences between the training data (source domain) and the test data (target domain). This phenomenon is known as domain shift. The performance of most detectors typically suffers a noticeable decline in the face of domain shift. Manually relabeling the data is a direct approach to addressing domain shift, but this is often impractical due to its high cost and complexity. Therefore, utilizing unlabeled target domain data has become a more realistic option, driving the development of Unsupervised Domain Adaptive Object Detection (UDAOD) methods. In UDAOD, models are typically trained using labeled source domain data and unlabeled target domain data, with labeled data only available in the source domain. By minimizing domain differences through adversarial learning, supervised detection models trained on the source domain can generalize better, extending to the target domain for knowledge transfer and representation.
[0004] Some studies have shown that the DEtection Transformer (DETR) exhibits superior performance in addressing domain offset phenomena, typically outperforming pure convolutional detectors. CNNs and Transformers differ fundamentally: CNNs capture visual features on feature maps through convolutional blocks, while Transformers model features from data images using attention mechanisms. DETR innovatively integrates CNNs and Transformers, with the system consisting of three parts: a CNN-based backbone network and Transformer-based encoders and decoders. This combination significantly improves feature extraction capabilities and eliminates cumbersome post-processing steps such as anchor point design and non-maximum suppression (NMS), greatly simplifying the training process.
[0005] Currently, many domain adaptation detectors based on DETR primarily focus on aligning image-level features. These methods employ adversarial feature alignment in the backbone network, improving the detector's ability to extract domain-invariant features by obfuscating the sources of domain features. Additionally, some methods attempt to align instance-level features in the decoder, typically using image-level alignment mechanisms. Recent research indicates that applying domain feature alignment solely to the CNN backbone of DETR for domain adaptation improvement has very limited effectiveness.
[0006] Therefore, a new solution is needed to address the above problems. Summary of the Invention
[0007] The purpose of this invention is to provide an efficient method for unsupervised domain adaptive target detection, aiming to solve the performance degradation problem of traditional target detectors caused by domain shift.
[0008] To achieve the above objectives, the present invention provides the following technical solution: an efficient method for unsupervised adaptive target detection, comprising at least the following steps:
[0009] S1: Design a domain query module that, under the constraint of adversarial loss, enables the model to focus on image features of a specific domain, allowing the detector to extract more domain-invariant features and achieve domain-adaptive target detection through adversarial training.
[0010] S2: Design a category prototype alignment module, which focuses on aligning instance features of objects of the same category from different domains. In each model iteration, mini-batch category prototype features are continuously updated and stored to gradually generate category prototype features representing the entire dataset. At the same time, through contrastive learning, the feature representations of the same category become more compact, while the feature representations of different categories become more distant, effectively improving the model's ability to distinguish between objects in different domains.
[0011] S3: Design a dataset-level category prototype alignment strategy to make full use of the information of the entire dataset, so as to improve the global representation ability of features and the discriminative power between categories, and enhance the model's ability to obtain contextual information at the global feature level.
[0012] S4: Build the UDA-DATR model based on the domain query module, the category prototype alignment module, and the dataset-level category prototype alignment strategy;
[0013] S5: The total loss function of the UDA-DETR model is optimized by integrating supervised detection loss, adversarial loss and contrastive loss. The purpose of optimization is to improve the detection performance in the target domain and thus alleviate the domain offset problem. The adversarial loss includes the backbone network, encoder and decoder.
[0014] S6: Target detection is performed using the optimized UDA-DETR model.
[0015] Furthermore, the design domain query module in S1 includes at least the following steps:
[0016] Token-wise multi-scale feature alignment was performed inside the model's encoder;
[0017] This module consists of a domain query and a cross-attention layer. The domain query is similar to the object queries in the decoder part. It is a set of trainable vectors used to identify the domain-specific features of objects.
[0018] Inside the encoder, feature tokens provide key-value pairs, while the domain query is used as the query vector input to the cross-attention layer to update the domain query.
[0019] The domain query will tend to focus on feature tokens that are discriminative for domain classification, that is, those domain-specific features;
[0020] use This represents the domain query updated by the cross-attention layer in the i-th encoder layer. This indicates the domain query before the update; This represents the feature token in the i-th layer encoder;
[0021] The cross-attention layer receives two inputs: a query and key-value pairs.
[0022] In the attention mechanism, the query focuses on specific keys in key-value pairs according to the attention weights and maps itself to a linear combination of the corresponding values.
[0023] Linear is a simple linear mapping layer used to perform this mapping;
[0024] The optimization method for domain queries is shown in formula (1):
[0025]
[0026] The entire domain query module is optimized using binary cross-entropy loss, as shown in formula (2):
[0027]
[0028] represents the total loss of the domain query module, used to train the domain classifier; d represents the domain label, where a value of 0 indicates the source domain image and a value of 1 indicates the target domain image.
[0029] This represents the domain prediction of the encoder's domain classifier for the final domain query. The predicted value is in the range [0,1] and represents the probability that the image belongs to the target domain.
[0030] By minimizing the loss function The domain query is trained to highlight the features most critical for distinguishing the source and target domains; this setup forces the detector to extract more domain-invariant features to minimize adversarial loss. This enables domain-specific adaptation.
[0031] Furthermore, the design category prototype alignment module in S2 includes at least the following steps:
[0032] The decoder receives the output object queries, and each object query aggregates the corresponding semantic information through the decoder's cross-attention mechanism;
[0033] For each object query, first determine its predicted category;
[0034] Then, the features of the queries for objects of the same category are merged, and the center of the merged aggregated features is calculated to obtain the center feature representation of each category. This process is shown in formula (3):
[0035]
[0036] Q n This represents the decoded object representation of the N initial object queries output after the decoder; C n represents the predicted category for the nth object query; c represents the category index;
[0037] When C n When =c, [C n The value of =c] is 1, otherwise it is 0;
[0038] In this way, I obtain the feature prototype representation F for each category. c ;
[0039] Subsequently, the category prototype feature F cThe input is fed into an adversarial domain discriminator to distinguish whether the feature center comes from the source domain or the target domain;
[0040] The domain discriminator is trained by minimizing the binary cross-entropy loss, while simultaneously making the class feature prototype F c Alignment is achieved in the two domains to deceive the discriminator. The optimization method is shown in formula (4):
[0041]
[0042] Where D d (F c ) represents the output of the domain discriminator, d represents the domain label, a value of 0 indicates the source domain, and a value of 1 indicates the target domain.
[0043] Furthermore, the dataset-level category prototype alignment strategy includes at least the following steps:
[0044] Store the mini-batch of class prototype features extracted in each training iteration, and model them as dataset-level class prototype features by calculating the average value of the stored local class prototype features, as shown in formula (5):
[0045]
[0046] Represents the dataset-level class prototype feature representation of class c, n c This represents the number of object queries for class c in the current training iteration, while This indicates the cumulative number of object queries for category c;
[0047] During the training process of the model, the prototype features of each category are continuously updated so that they can reflect the overall features of the category, including the source domain and the target domain.
[0048] In this way, the model can learn more comprehensive and representative feature representations, thereby improving the performance of cross-domain object detection;
[0049] To further enhance the class discriminability of the model, contrastive learning is introduced to bring features of the same class closer together and exclude features of different classes. The calculation method is shown in formula (6):
[0050]
[0051] F Si and F Ti These represent the central features in the source and target domains, respectively. Represents the global category prototype feature, where C represents the total number of categories;
[0052] For the central feature F of each category Si and F Ti They are calculated by dot product with the global central feature F. Ti The similarity between them;
[0053] Then, the softmax function is used to calculate the relative probability of similarity between local prototype features and global prototype features for each category. For each category's local prototype feature, the log probability of its similarity to the global prototype feature is calculated, and the log probabilities of all categories are summed. Finally, the log probabilities of all categories are averaged to obtain the contrastive learning loss L. con .
[0054] Furthermore, S5 includes at least the following steps:
[0055] Set the source domain dataset with annotation information as And target domain datasets without annotation information
[0056] Where x represents the image itself, y = (b, c) represents the annotation information corresponding to the image, t indicates that the image comes from the target domain, s indicates that the image comes from the source domain, i represents the current image, and N s N represents the number of source domain images. t The annotation information indicates the number of target domain images, and includes the bounding box position b and the category label c;
[0057] The proposed UDA-DATR model is trained in the source domain and its performance is evaluated in the target domain. This setup aims to improve the model's detection performance in the target domain by effectively utilizing the labeled data in the source domain, thereby mitigating the impact of domain shift.
[0058] Deformable-DETR is selected as the basic detector of the model, and the Deformable-DETR includes at least a backbone network, an encoder, and a decoder.
[0059] In supervised learning, the source domain image is processed by calculating the supervised loss L. sup To optimize the model, the loss is calculated in the source domain, as shown in Equation (7):
[0060]
[0061] x s Represents the source domain image, y s This represents the label value corresponding to the source domain image. This represents the image bounding box regression loss. Represents the Giou loss of the source domain image. Represents the category loss of the source domain image;
[0062] The overall objective of UDA-DETR is further summarized, which consists of three adversarial losses. A comparative loss L con And a supervised detection loss composition;
[0063] Combining the above factors, we can construct the overall loss function L. UDA-DETR As shown in formula (8):
[0064]
[0065] Compared with the prior art, the beneficial effects of the present invention are:
[0066] This invention designs a domain query module that performs adversarial alignment on the features output by the encoder, enabling the detector to extract more domain-invariant features. In addition, a category prototype alignment module is proposed, which can extract the category prototype features of each category from the decoder to achieve category-aware cross-domain feature alignment. In each training iteration, local category prototypes gradually generate global category prototypes. The detector is guided to achieve global feature alignment through contrastive learning and adversarial loss. Extensive experimental results show that the proposed model achieves excellent detection performance on multiple domain-adapted benchmark datasets. Attached Figure Description
[0067] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0068] Figure 1 This is a schematic diagram of the model architecture of the present invention;
[0069] Figure 2 This is a visual comparison of the detection performance of the present invention with other advanced domain adaptive methods;
[0070] Figure 3 The diagram shows the detection results achieved when using different modules in this invention.
[0071] Figure 4 This is a heat map of the model used in this invention for detection in adverse weather conditions. Detailed Implementation
[0072] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0073] When there is a domain gap between the training data (source domain data) and the data to be detected in the real-world application (target domain data), the performance of many object detectors degrades significantly. To address this issue, many unsupervised domain adaptation detectors try to narrow the gap by minimizing the domain difference in the class space and aligning global features between the two domains. While these techniques have achieved some success, they often employ a class-agnostic approach to global feature alignment, ignoring the differences in the distribution of features across different classes in different domains, leading to classification and localization errors.
[0074] The method proposed in this invention is essentially an efficient unsupervised domain adaptive detection architecture (UDA-DETR), which pays more attention to global image features and local instance features from different domains to achieve more accurate domain adaptive detection.
[0075] Please see Figure 1 An efficient method for adaptive object detection in unsupervised domains includes at least the following steps:
[0076] S1: Design a domain query module that, under the constraint of adversarial loss, enables the model to focus on image features of a specific domain, allowing the detector to extract more domain-invariant features and achieve domain-adaptive target detection through adversarial training.
[0077] S2: Design a category prototype alignment module, which focuses on aligning instance features of objects of the same category from different domains. In each model iteration, mini-batch category prototype features are continuously updated and stored to gradually generate category prototype features representing the entire dataset. At the same time, through contrastive learning, the feature representations of the same category become more compact, while the feature representations of different categories become more distant, effectively improving the model's ability to distinguish between objects in different domains.
[0078] S3: Design a dataset-level category prototype alignment strategy to make full use of the information of the entire dataset, so as to improve the global representation ability of features and the discriminative power between categories, and enhance the model's ability to obtain contextual information at the global feature level.
[0079] S4: Build the UDA-DATR model based on the domain query module, the category prototype alignment module, and the dataset-level category prototype alignment strategy;
[0080] S5: The total loss function of the UDA-DETR model is optimized by integrating supervised detection loss, adversarial loss and contrastive loss. The purpose of optimization is to improve the detection performance in the target domain and thus alleviate the domain offset problem. The adversarial loss includes the backbone network, encoder and decoder.
[0081] S6: Target detection is performed using the optimized UDA-DETR model.
[0082] The UDA-DATR model framework structure is as follows: Figure 1 As shown, for the input image data x, it is first processed by the backbone network to obtain the corresponding feature map z = R. C×H×W C represents the number of channels, H represents the height, and W represents the width. The feature map is then flattened into a z-shape. P =R C×L L = H × W represents the number of tokens, which is input into the encoder to obtain higher-dimensional semantic information. Next, the decoder performs object detection using a set of N object queries (N = 300), ultimately obtaining the detection loss. Furthermore, we performed category-independent image-level feature alignment in the backbone network, resulting in a loss function. The loss function is obtained by embedding a domain query model into the encoder part and performing token-wise multi-scale feature alignment. In the decoder section, category prototype feature alignment was designed and combined with contrastive learning to obtain... and L con .
[0083] The design domain query module in S1 includes at least the following steps:
[0084] Token-wise multi-scale feature alignment was performed inside the model's encoder;
[0085] This module consists of a domain query and a cross-attention layer. The domain query is similar to the object queries in the decoder part. It is a set of trainable vectors used to identify the domain-specific features of objects.
[0086] Inside the encoder, feature tokens provide key-value pairs, while the domain query is used as the query vector input to the cross-attention layer to update the domain query.
[0087] Domain queries tend to focus on feature tokens that are discriminative for domain classification, i.e., those domain-specific features;
[0088] use This represents the domain query updated by the cross-attention layer in the i-th encoder layer. This indicates the domain query before the update; This represents the feature token in the i-th layer encoder;
[0089] The cross-attention layer receives two inputs: a query and key-value pairs.
[0090] In the attention mechanism, the query focuses on specific keys in key-value pairs according to the attention weights and maps itself to a linear combination of the corresponding values.
[0091] Linear is a simple linear mapping layer used to perform this mapping;
[0092] The optimization method for domain queries is shown in formula (1):
[0093]
[0094] The entire domain query module is optimized using binary cross-entropy loss, as shown in formula (2):
[0095]
[0096] represents the total loss of the domain query module, used to train the domain classifier; d represents the domain label, where a value of 0 indicates the source domain image and a value of 1 indicates the target domain image.
[0097] This represents the domain prediction of the encoder's domain classifier for the final domain query. The predicted value is in the range [0,1] and represents the probability that the image belongs to the target domain.
[0098] By minimizing the loss function The domain query is trained to highlight the features most critical for distinguishing the source and target domains; this setup forces the detector to extract more domain-invariant features to minimize adversarial loss. This enables domain-specific adaptation.
[0099] The design category prototype alignment module in S2 includes at least the following steps:
[0100] The decoder receives the output object queries, and each object query aggregates the corresponding semantic information through the decoder's cross-attention mechanism;
[0101] For each object query, first determine its predicted category;
[0102] Then, the features of the queries for objects of the same category are merged, and the center of the merged aggregated features is calculated to obtain the center feature representation of each category. This process is shown in formula (3):
[0103]
[0104] Q n This represents the decoded object representation of the N initial object queries output after the decoder; C n represents the predicted category for the nth object query; c represents the category index;
[0105] When C n When =c, [C n The value of =c] is 1, otherwise it is 0;
[0106] In this way, I obtain the feature prototype representation F for each category. c ;
[0107] Subsequently, the category prototype feature F c The input is fed into an adversarial domain discriminator to distinguish whether the feature center comes from the source domain or the target domain;
[0108] The domain discriminator is trained by minimizing the binary cross-entropy loss, while simultaneously making the class feature prototype F c Alignment is achieved in the two domains to deceive the discriminator. The optimization method is shown in formula (4):
[0109]
[0110] Where D d (F c ) represents the output of the domain discriminator, d represents the domain label, a value of 0 indicates the source domain, and a value of 1 indicates the target domain.
[0111] Dataset-level category prototype alignment strategies include at least the following steps:
[0112] Store the mini-batch of class prototype features extracted in each training iteration, and model them as dataset-level class prototype features by calculating the average value of the stored local class prototype features, as shown in formula (5):
[0113]
[0114] Represents the dataset-level class prototype feature representation of class c, n cThis represents the number of object queries for class c in the current training iteration, while This indicates the cumulative number of object queries for category c;
[0115] During the training process of the model, the prototype features of each category are continuously updated so that they can reflect the overall features of the category, including the source domain and the target domain.
[0116] In this way, the model can learn more comprehensive and representative feature representations, thereby improving the performance of cross-domain object detection;
[0117] To further enhance the class discriminability of the model, contrastive learning is introduced to bring features of the same class closer together and exclude features of different classes. The calculation method is shown in formula (6):
[0118]
[0119] F Si and F Ti These represent the central features in the source and target domains, respectively. Represents the global category prototype feature, where C represents the total number of categories;
[0120] For the central feature F of each category Si and F Ti They are calculated by dot product with the global central feature F. Ti The similarity between them;
[0121] Then, the softmax function is used to calculate the relative probability of similarity between local prototype features and global prototype features for each category. For each category's local prototype feature, the log probability of its similarity to the global prototype feature is calculated, and the log probabilities of all categories are summed. Finally, the log probabilities of all categories are averaged to obtain the contrastive learning loss L. con .
[0122] S5 includes at least the following steps:
[0123] Set the source domain dataset with annotation information as And target domain datasets without annotation information
[0124] Where x represents the image itself, y = (b, c) represents the annotation information corresponding to the image, t indicates that the image comes from the target domain, s indicates that the image comes from the source domain, i represents the current image, and N s N represents the number of source domain images. t The number of target domain images is indicated by the annotation information, which includes the bounding box position b and the category label c.
[0125] The proposed UDA-DATR model is trained in the source domain and its performance is evaluated in the target domain. This setup aims to improve the model's detection performance in the target domain by effectively utilizing the labeled data in the source domain, thereby mitigating the impact of domain shift.
[0126] Deformable-DETR was chosen as the base detector for the model. Deformable-DETR includes at least a backbone network, an encoder, and a decoder.
[0127] In supervised learning, the source domain image is processed by calculating the supervised loss L. sup To optimize the model, the loss is calculated in the source domain, as shown in Equation (7):
[0128]
[0129] x s Represents the source domain image, y s This represents the label value corresponding to the source domain image. This represents the image bounding box regression loss. Represents the Giou loss of the source domain image. Represents the category loss of the source domain image;
[0130] The overall objective of UDA-DETR is further summarized, which consists of three adversarial losses. A comparative loss L con And a supervised detection loss composition;
[0131] Combining the above factors, we can construct the overall loss function L. UDA-DETR As shown in formula (8):
[0132]
[0133] To demonstrate the superior performance of the proposed UDA-DETR, comparative experiments were conducted on three domain adaptation benchmark datasets against many advanced unsupervised domain adaptation methods. The experimental results show that the proposed method significantly outperforms all previous advanced methods.
[0134] This invention first investigated the domain-adaptive detection performance under different weather scenarios, using the normal weather scenario Cityscapes as the source domain and the foggy weather scenario Foggy Cityscapes as the target domain. The experimental results are shown in Table 1. The UDA-DETR of this invention achieved the best AP of 46.0 mAP. For some difficult-to-detect categories (such as riders), UDA-DETR still achieved the best AP of 50.3.
[0135] Secondly, this invention investigated domain adaptation from Cityscapes to BDD 100k-daytime, and the experimental results are shown in Table 2. Due to the large difference in feature distribution between the source and target domains, most unsupervised domain adaptation methods perform poorly, but the UDA-DETR method of this invention still achieved the best mAP of 30.2.
[0136] Finally, this invention investigated domain adaptation from Sim10k to Cityscapes (Car), and the experimental results are shown in Table 3. The UDA-DETR method proposed in this invention significantly outperforms previous state-of-the-art (SOTA) techniques, achieving a best mAP of 55.1.
[0137] And can be consulted Figure 2 , 3 A comparison with 4 is shown below, where Figure 4 The darker the color in a region, the more attention the model pays to that region.
[0138] The experimental results above demonstrate that the UDA-DETR method proposed in this invention exhibits superior performance in various domain adaptation scenarios.
[0139] Table 1. Results of the adaptation experiment across different weather scenarios: Cityscapes to Foggy Cityscapes
[0140]
[0141] Table 2. Results of cross-camera scene adaptation experiment: Cityscapes to BDD100k-daytime
[0142]
[0143]
[0144] Table 3. Domain adaptation experiment results from synthetic scene to real scene: Sim10kto Cityscapes results
[0145]
[0146] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. An efficient method for adaptive target detection in unsupervised domains, characterized in that: At least the following steps are included: S1: Design a domain query module that, under the constraint of adversarial loss, enables the model to focus on image features of a specific domain, allowing the detector to extract more domain-invariant features and achieve domain-adaptive target detection through adversarial training; S2: Design a category prototype alignment module. This module focuses on aligning instance features of objects of the same category from different domains. In each model iteration, a small batch of category prototype features is continuously updated. These local category prototype features are stored to gradually generate category prototype features that represent the entire dataset. At the same time, through contrastive learning, the feature representations of the same category become more compact, while the feature representations of different categories become more distant, effectively improving the model's ability to distinguish between classes of objects in different domains. S3: Design a dataset-level category prototype alignment strategy to make full use of the information of the entire dataset, so as to improve the global representation ability of features and the discriminative power between categories, and enhance the model's ability to obtain contextual information at the global feature level. S4: Build the UDA-DATR model based on the domain query module, the category prototype alignment module, and the dataset-level category prototype alignment strategy; In the framework of the UDA-DATR model, for the input image data First, the feature maps are obtained through processing by the backbone network. , Represents the number of channels. Represents height, The width is represented, and then the feature map is flattened. , This represents the number of tokens, which is then input into the encoder to obtain higher-dimensional semantic information. Next, the decoder is configured with... Object detection is performed using object queries, and the supervised detection loss is ultimately obtained. Furthermore, image-level feature alignment independent of category was performed in the backbone network to obtain an adversarial loss. An adversarial loss is obtained by embedding a domain query module in the encoder part and performing token-wise multi-scale feature alignment. In the decoder part, we designed class prototype feature alignment and combined it with contrastive learning to obtain adversarial loss. And comparative loss ; S5: The total loss function of the UDA-DETR model is optimized by combining one supervised detection loss, three adversarial losses, and one contrastive loss. The purpose of the optimization is to improve the detection performance in the target domain and thus alleviate the domain shift problem. S6: Target detection is performed using the optimized UDA-DETR model.
2. The efficient method for unsupervised adaptive target detection according to claim 1, characterized in that: The design domain query module in S1 includes at least the following steps: Token-wise multi-scale feature alignment was performed within the model's encoder. This module consists of a domain query and a cross-attention layer. The domain query is similar to the object query in the decoder part. It is a set of trainable vectors used to identify the domain-specific features of objects. Inside the encoder, feature tokens provide key-value pairs, while the domain query is used as the query vector input to the cross-attention layer, thereby updating the domain query; The domain query will tend to focus on feature tokens that are discriminative for domain classification, that is, those domain-specific features; use Indicates the first In the layer encoder, the domain query is updated after passing through the cross-attention layer; This indicates the domain query before the update; Indicates the first Feature tokens in the layer encoder; The cross-attention layer receives two inputs and queries AND key-value pairs. In the attention mechanism, the query focuses on a specific key in the key-value pair according to the attention weight, and maps itself to a linear combination of the corresponding values; Linear is a simple linear mapping layer used to perform this mapping; The optimization method for domain queries is shown in formula (1): The entire domain query module is optimized using binary cross-entropy loss, as shown in formula (2): This represents the total loss of the domain query module, used to train the domain classifier; This represents the domain label, where a value of 0 indicates the source domain image and a value of 1 indicates the target domain image. This represents the domain prediction of the encoder's domain classifier for the final domain query. The predicted value is in the range [0,1] and represents the probability that the image belongs to the target domain. By minimizing the loss function The domain query is trained to highlight the features most critical for distinguishing the source and target domains; this setup forces the detector to extract more domain-invariant features to minimize adversarial loss. This enables domain-specific adaptation.
3. The efficient method for unsupervised adaptive target detection according to claim 2, characterized in that: The design category prototype alignment module in S2 includes at least the following steps: The decoder receives the output object queries, and each object query aggregates the corresponding semantic information through the decoder's cross-attention mechanism; For each object query, first determine its predicted category; Then, the features of the queries for objects of the same category are merged, and the center of the merged aggregated features is calculated to obtain the center feature representation of each category. This process is shown in formula (3): in This represents the decoded object representation of the N initial object queries output after the decoder; This indicates the predicted category for the nth object query; Indicates a category index; when hour, The value is 1 if it is 1, otherwise it is 0. In this way, the prototype feature representation of each category is obtained. ; Subsequently, the category prototype features The input is fed into an adversarial domain discriminator to distinguish whether the feature center comes from the source domain or the target domain; The domain discriminator is trained by minimizing the binary cross-entropy loss, while simultaneously incorporating class prototype features. Alignment is achieved in the two domains to deceive the discriminator. The optimization method is shown in formula (4): in This represents the output of the neighborhood discriminator. This represents the domain label; a value of 0 indicates the source domain, and a value of 1 indicates the target domain.
4. The efficient method for unsupervised adaptive target detection according to claim 3, characterized in that: The dataset-level category prototype alignment strategy includes at least the following steps: Store the mini-batch of class prototype features extracted in each training iteration, and model them as dataset-level class prototype features by calculating the average value of the stored local class prototype features, as shown in formula (5): Indicate category Data set-level category prototype feature representation, Indicates the category in the current training iteration The number of object queries, This indicates the category. The cumulative number of object queries; During the training process of the model, the prototype features of each category are continuously updated so that they can reflect the overall features of the category, including the source domain and the target domain. In this way, the model can learn more comprehensive and representative feature representations, thereby improving the performance of cross-domain object detection; To further enhance the class discriminability of the model, contrastive learning is introduced to bring features of the same class closer together and exclude features of different classes. The calculation method is shown in formula (6): and These represent the central features in the source and target domains, respectively. Represents global category prototype features. Indicates the total number of categories; For the central feature of each category and They are calculated by dot product with the global central features. The similarity between them; Then, the softmax function is used to calculate the relative probability of similarity between local prototype features and global prototype features for each category. For each category's local prototype feature, the log probability of its similarity to the global prototype feature is calculated, and the log probabilities of all categories are summed. Finally, the log probabilities of all categories are averaged to obtain the contrastive loss. .
5. The efficient method for unsupervised adaptive target detection according to claim 4, characterized in that: The S5 includes at least the following steps: Set the source domain dataset with annotation information as And target domain datasets without annotation information ; in, Represents the image itself. This represents the annotation information corresponding to the image. This indicates that the image comes from the target domain. This indicates that the image comes from the source domain. Indicates the current image. Indicates the number of source domain images. The annotation information indicates the number of target domain images, including the bounding box positions. and category labels ; The proposed UDA-DATR model is trained in the source domain and its performance is evaluated in the target domain. This setup aims to improve the model's detection performance in the target domain by effectively utilizing the labeled data in the source domain, thereby mitigating the impact of domain shift. Deformable-DETR is selected as the basic detector of the model, and the Deformable-DETR includes at least a backbone network, an encoder, and a decoder. In supervised learning, the source domain image is processed by calculating the supervised detection loss. To optimize the model, the loss is calculated in the source domain, as shown in Equation (7): Represents the source domain image. This represents the label value corresponding to the source domain image. This represents the image bounding box regression loss. Representing the source domain image loss, Represents the category loss of the source domain image; The overall objective of UDA-DETR is further summarized, which consists of three adversarial losses and one contrastive loss. and a monitoring and detection loss composition; Combining the above factors, construct the overall loss function. As shown in formula (8): .