A domain adaptation object detection method based on contrastive learning and diffusion model

Abstract

The application provides a domain adaptive target detection method based on contrast learning and diffusion model, belongs to the image classification technical field of transfer learning, and comprises the following steps: a diffusion model is used for target domain data, features extracted by a feature extractor are denoised in a latent space to generate feature pairs with consistent semantics and domain differences, a contrast loss is calculated based on the feature pairs, and features invariant to domains are effectively strengthened; based on a self-adaptive teacher-student framework, the branch is added to a student model, the student model is trained to update a teacher model, and finally, the target domain data is subjected to target detection through the teacher model. According to the application, positive samples are obtained by diffusion enhancement on the target domain data, other samples are used as negative samples, the model is forced to learn features invariant to domains by minimizing the contrast loss, the stability of model training is ensured by fusing an adversarial alignment loss and the like, and the accuracy of target detection of the model is ensured through a supervision loss of a source domain.

Description

Technical Field

[0001] This invention belongs to the field of image classification technology of transfer learning, and particularly relates to a domain adaptive target detection method based on contrastive learning and diffusion model. Background Technology

[0002] Object detection is one of the core tasks of computer vision, playing a crucial role in numerous applications such as autonomous driving and security monitoring. However, deep learning models face a significant challenge in practical applications: their performance degrades significantly when deployed to new scenes (i.e., the target domain) not present in the training set. The root cause of this performance degradation lies in "domain shift." Deep learning models assume that the distribution of training data (source domain) and test data (target domain) is consistent. However, in the real world, the target domain (such as a rainy day or a rural scene) differs greatly from the source domain (such as a sunny day or an urban scene) in image style, lighting conditions, weather conditions, or object distribution. This leads to a severe decrease in the generalization ability of models trained on labeled source domains.

[0003] To alleviate the domain shift problem, unsupervised domain adaptive object detection (UDA-OD) has become a core research direction. This task aims to improve the detection performance of models in completely unlabeled target domains by utilizing richly labeled source domain data. Although existing methods have made some progress, key limitations still exist: Adversarial feature alignment-based methods: Early DAOD methods primarily relied on adversarial feature alignment, such as DA-Faster. These methods attempted to confuse the model's distinction between the source and target domains at the feature level (e.g., image-level, instance-level) by introducing a domain discriminator. However, these methods performed poorly in terms of granular alignment control and training stability, making it difficult to effectively approximate complex domain distributions.

[0004] Self-training methods based on teacher-student frameworks: In recent years, teacher-student frameworks have significantly improved DAOD performance through pseudo-label self-training. For example, AT (Adaptive Teacher) and UMT use the "mean teacher paradigm" to generate high-quality pseudo-labels to guide student models. These methods face a common, core challenge: the quality of pseudo-labels is severely constrained by domain shift. Specifically: (1) Insufficient supervision: To ensure the accuracy of pseudo-labels, a high confidence threshold is usually set. However, this will result in a small number of "high-quality" pseudo-labels generated by the model when the domain differences are large, resulting in insufficient supervision of the student model.

[0005] (2) Guiding noise: If the threshold is relaxed, a large number of false labels will be introduced. These "noise labels" will interfere with the training of student models, leading to unstable model training and even performance degradation.

[0006] Masked Autoencoder (MAE) based methods: Some works have attempted to use MAE for domain adaptation tasks. For example, the MRT framework uses MAE as a parallel branch to help the model capture target domain features from a limited set of pseudo-labels. However, such methods rely excessively on the student model's own domain understanding ability. When the domain offset between the source and target domains is very large, the model itself struggles to accurately understand the target domain, resulting in limited MAE reconstruction performance.

[0007] Domain generalization methods based on diffusion models: Due to their powerful generative capabilities and inherent domain generalization ability gained after pre-training on diverse datasets, diffusion models have begun to be used in DAOD tasks. For example, DDT (Diffusion Domain Teacher Model) proposes using a diffusion model with frozen weights as a feature extractor. However, the core problem with DDT is that it still uses the traditional mean-teacher framework and does not solve the inherent "pseudo-label progressive optimization" problem in the teacher-student framework. That is, it lacks a mechanism to handle the challenges of dynamic changes in pseudo-label quality and noise interference during self-training.

[0008] In summary, existing unsupervised domain adaptive object detection (UDA-OD) methods face two main challenges in practical applications: The quality and noise interference of pseudo-labels: Self-trained teacher-student frameworks are currently the mainstream, but their performance is highly dependent on the pseudo-labels generated by the teacher model. Domain offset inevitably leads to noise in pseudo-labels, while high-threshold filtering results in insufficient supervision, making the model prone to getting stuck in suboptimal solutions.

[0009] The learning efficiency of domain-invariant features: Both adversarial alignment and MAE reconstruction attempt to learn a kind of "domain-invariant feature", that is, a feature that can maintain a consistent semantic expression in both the source and target domains. However, existing methods lack robustness in learning features when there are large domain differences, and are easily affected by domain noise such as illumination and style.

[0010] Therefore, designing a framework that can efficiently learn domain-invariant features and overcome self-training noise interference for target detection is a technical challenge that urgently needs to be solved. Summary of the Invention

[0011] To address the aforementioned technical issues of domain shift, pseudo-label noise interference, and low efficiency in learning domain-invariant features, this invention proposes a dual-branch diffusion contrastive teacher-student (DDC-TS) framework based on a Deformable DETR backbone network and designs a multi-stage training paradigm. The core innovation of this method lies in constructing a dual-branch diffusion contrastive learning (DBD-CL) module: utilizing a diffusion model with frozen weights, it generates "semantically consistent but domain-distinct feature pairs" for the target domain image, and then forces the model to learn domain-invariant features through contrastive loss. Furthermore, the DDC-TS framework integrates a teacher model based on exponential moving average (EMA) (for generating high-quality dynamic pseudo-labels) with an adversarial feature alignment mechanism; the multi-stage training process (source domain pre-training → cross-domain feature adaptation → teacher-student self-training) steadily guides the model to adapt to the target domain.

[0012] To achieve the above objectives, this invention provides a domain-adaptive target detection method based on contrastive learning and a diffusion model, comprising: Construct a two-branch diffusion comparison framework for teacher-student interaction that includes both student and teacher models; The data paths of the dual-branch diffusion comparison teacher-student framework are defined, including the source domain student path, the target domain teacher path, and the target domain student path, and the parameters of the teacher model are updated using the parameters of the student model based on the exponential moving average method. In the target domain student path of the student model, a two-branch diffusion contrast learning module is designed. Based on the two-branch diffusion contrast learning module, the target domain features are subjected to two independent diffusion processes using a frozen diffusion model to generate the first denoised feature and the second denoised feature. Construct an optimization objective that integrates multiple losses, wherein the optimization objective includes at least source domain supervised loss, target domain unsupervised loss, adversarial alignment loss, and bi-branch contrastive loss based on the first denoising feature and the second denoising feature; A multi-stage training strategy is adopted to train the student model based on the optimization objective, and the teacher model is updated using the trained student model through an exponential moving average method. The updated teacher model is then used to perform target detection on the target domain data to obtain the detection results.

[0013] Preferably, the dual-branch diffusion comparison teacher-student framework uses the adaptive teacher-student framework as the baseline, and both the student model and the teacher model use deformable detection transformers as basic detectors and residual networks as backbone networks.

[0014] Preferably, the process of defining the data path of the dual-branch diffusion comparison teacher-student framework includes: The enhanced source domain image is input into the student model, and the supervised loss of the source domain is calculated based on the true labels of the source domain. The weakly enhanced target domain image is input into the teacher model to generate target domain pseudo-labels; The enhanced target domain image is input into the student model. The unsupervised loss of the target domain is calculated based on the pseudo-labels generated by the teacher model. The two-branch contrastive loss is calculated through the two-branch diffusion contrastive learning module. The parameters of the teacher model are updated using an exponential moving average based on the parameters of the student model.

[0015] Preferably, the process of designing the dual-branch diffusion contrastive learning module includes: Multi-scale features of strongly enhanced target domain images are extracted using the backbone network of the student model. The multi-scale features are subjected to two independent diffusion processes using a pre-trained diffusion model with frozen weights. Each diffusion process includes noise addition and denoising operations to generate a first denoised feature and a second denoised feature. The first denoised feature and the second denoised feature are used to form a feature pair for contrastive learning.

[0016] Preferably, the two independent diffusion processes include: In the first diffusion process, the first noise and the first diffusion time step are randomly sampled, and the multi-scale features are denoised according to the diffusion formula. The denoised features and the first diffusion time step are then input into the diffusion model to obtain the first denoised features. In the second diffusion process, the second noise and the second diffusion time step are randomly sampled, where the second noise is different from the first noise. The multi-scale features are denoised according to the diffusion formula. The denoised features and the second diffusion time step are input into the diffusion model to obtain the second denoised features. In this process, the two diffusion processes share the backbone network weights of the student model and the diffusion model weights, and the feature differences are created only through the differences in noise and diffusion time steps.

[0017] Preferably, the weights of the diffusion model are kept frozen during training; The two independent diffusion processes are achieved through independent noise sampling and independent diffusion time step sampling, wherein the diffusion time step is randomly selected within a preset interval to ensure a balance between semantic preservation and noise difference.

[0018] Preferably, the process of constructing the optimization objective of multi-loss fusion includes: Calculate the source domain supervised loss based on the true source domain labels; Calculate the unsupervised loss for the target domain based on the pseudo-labels generated by the teacher model; A domain discriminator is added after the backbone network, encoder, and decoder of the student model to calculate the adversarial alignment loss based on the source domain features and the target domain features. The bi-branch contrastive loss is calculated based on the feature pairs used for contrastive learning, which consist of the first and second denoised features.

[0019] Preferably, the process of calculating the adversarial alignment loss includes: Global features are extracted from the backbone network output of the student model, token-level multi-scale features are extracted from the encoder, and instance features are extracted from the decoder. The global features, token-level multi-scale features, and instance features are respectively input into the corresponding domain discriminators; The binary classification cross-entropy loss is calculated based on the output of the domain discriminator and the domain label, and the adversarial alignment loss is obtained.

[0020] Preferably, the process of calculating the two-branch contrast loss includes: Cosine similarity calculation is performed on the first and second denoised features of images in the same target domain. Cosine similarity calculation is performed on the denoising features of different images in the current batch; The calculated similarity value is input into the information noise contrast estimation loss function to obtain the two-branch contrast loss.

[0021] Preferably, the multi-stage training strategy includes: In the first stage, the student model is trained using only source domain supervised loss to obtain baseline detection capability; In the second stage, based on the supervised loss in the source domain, a bi-branch contrastive loss and an adversarial alignment loss are added to train the student model to learn domain-invariant features. In the third stage, a complete multi-loss fusion optimization objective is adopted. The teacher model is updated by exponential moving average, which enables the teacher model to generate pseudo-labels. The student model is then self-trained and optimized under the supervision of all losses.

[0022] Compared with the prior art, the present invention has the following advantages and technical effects: (1) Enhancing the robustness of domain-invariant features: The DBD-CL module designed in this invention uses a frozen diffusion model to simulate cross-domain differences in style, lighting, etc. (domain noise), generating feature pairs that are "semantically consistent but domain-different". By contrastive learning, the student model is forced to bring these feature pairs closer together, making the feature representation learned by the model insensitive to domain noise, thus significantly improving the domain invariance and robustness of the features.

[0023] (2) Optimize pseudo-label utilization and suppress noise: Traditional teacher-student frames rely entirely on pseudo-labels. This invention introduces... As an auxiliary self-supervised signal, even in cases of sparse pseudo-labels (high threshold) or high noise (low threshold), It still provides stable gradient guidance, optimizes the utilization efficiency of pseudo-labels, and suppresses the interference of noisy labels on training.

[0024] (3) Addressing the limitations of diffusion models in DAOD: Existing methods such as DDT only use diffusion models as feature extractors and do not address the optimization problem of pseudo-labels. This invention combines diffusion models with contrastive learning and teacher-student frameworks to construct a more comprehensive adaptive framework, solving the problem of progressive optimization of pseudo-labels during training.

[0025] (4) Stable training and superior performance: Through a multi-stage training strategy, the model first acquires basic capabilities in the source domain, then gradually adapts to the target domain, and finally performs self-training, effectively avoiding catastrophic forgetting in the early stages of training. On several mainstream cross-domain object detection benchmarks (such as Sim10k→Cityscapes), the detection performance of the framework of this invention is significantly better than the existing state-of-the-art (SOTA) methods. Attached Figure Description

[0026] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention. Detailed Implementation

[0027] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0028] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0029] Object detection plays a central role in numerous computer vision applications, including autonomous driving and security monitoring. However, when models face labeled source domains (such as sunny urban scenes) and unlabeled target domains (such as rainy rural scenes), differences in image style, lighting conditions, or target distribution can trigger a "domain shift" problem, severely impairing the model's generalization ability and leading to a significant drop in detection performance. Therefore, Domain Adaptive Object Detection (DAOD) has become a core research direction for mitigating this problem.

[0030] While existing DAOD methods have made some progress, key limitations remain. Early methods relied on adversarial feature alignment, but performed poorly in terms of alignment granularity and stability. The teacher-student framework improved performance through pseudo-label self-training, but the quality of its pseudo-labels is constrained by domain offset—too high a threshold leads to insufficient supervision, while too low a threshold introduces guiding noise. Masked autoencoders (MAEs) have been used for domain adaptation tasks, but they rely too heavily on the model's own domain understanding ability, resulting in limited effectiveness when the domain offset is large. Although diffusion models possess inherent domain generalization capabilities, they have not yet solved the problem of progressive optimization of pseudo-labels in DAOD tasks.

[0031] In view of the problems in the prior art described above, this embodiment discloses a domain-adaptive target detection adaptive teacher-student framework based on bi-branch diffusion contrastive learning. This framework gradually guides the model to learn domain-invariant features and ultimately optimizes it on the target domain through self-training. Now combined with Figure 1 This paper describes the publicly disclosed domain-adaptive object detection adaptive teacher-student framework, the implementation of which includes the following steps: Step 1: Formulate a definition for the problem of Unsupervised Domain Adaptation Detection (UDAD); Step 2: Based on the formulaic definition, the model is forced to learn domain-invariant features by using the newly added dual-branch diffusion contrastive learning module. This method is called dual-branch diffusion contrastive learning (DBD-CL). Step 3: Implement the application of the bi-branch diffusion structure in the adaptive teacher-student framework—generate features that are insensitive to domain noise through the diffusion structure for the unlabeled target domain, thereby improving cross-domain generalization ability.

[0032] The implementation idea of ​​bi-branch diffraction contrastive learning is: to strongly enhance images of the same target domain. First, extract multi-scale features from the student's backbone. ={ , , , }( (corresponding to the four stages of ResNet output), where the top-level features Including the richest semantic information, for Two independent diffusion processes were performed.

[0033] In this embodiment, the specific implementation steps are as follows: First diffusion noise addition-denoising (generation) Randomly sampled noise sequence and time step Noisy features are generated according to the diffusion formula. (Add noise) and Input diffusion model, output denoised features (Noise reduction); Second diffusion noise addition-denoising (generation) ): Using random noise sequences ( ) and different time steps ( [5,10], Can be with (Same but with different noise levels), and obtain the output by following a similar step to S1. .

[0034] It is important to note that the two diffusion processes share all weights of the student backbone and the diffusion model, and only through... / , / Differences in manufacturing characteristics ensure and It is a "domain difference view of the same image".

[0035] "Targeting the enhanced target domain image" The "domain difference view" feature of the target domain image is generated by two independent denoising operations using a diffusion model and then trained on the student model. This is defined as the Dual-Branch Diffusion-Contrastive Module. This dual-branch module is embedded into the student model. Combined with contrastive loss, it forces the student to learn domain-invariant features that are "insensitive to domain noise and preserve core semantics", thereby improving the cross-domain generalization ability of the entire model. The entire framework is defined as the Dual-Branch Diffusion-Contrastive Teacher-Student Framework DDC-TS.

[0036] like Figure 1 As shown, this embodiment provides a domain-adaptive target detection method based on contrastive learning and a diffusion model, including: Construct a two-branch diffusion contrastive teacher-student (DDC-TS) framework that includes student and teacher models; Define the data paths of the dual-branch diffusion comparison teacher-student framework, including the source domain student path for learning basic skills, the target domain teacher path for generating pseudo-labels, and the target domain student path for adapting to the target domain. The parameters of the teacher model are updated using the parameters of the student model based on the exponential moving average (EMA) method. In the target domain student path of the student model, a dual-branch diffusion contrastive learning (DBD-CL) module is designed. Based on the dual-branch diffusion contrastive learning module, the frozen diffusion model is used to perform two independent diffusion processes on the target domain features to generate feature pairs that are "semantically consistent and domain-different", and obtain the first denoised feature and the second denoised feature. Construct an optimization objective that integrates multiple losses. The optimization objective includes at least a source domain supervised loss for basic capability learning, a target domain unsupervised loss for adaptation, an adversarial alignment loss, and a bi-branch contrastive loss based on the first and second denoised features. A multi-stage training strategy is adopted. The student model is trained based on the optimization objective, and the teacher model is updated by exponential moving average using the trained student model. The updated teacher model is then used to perform target detection on the target domain data to obtain the detection results.

[0037] Furthermore, the dual-branch diffusion contrastive teacher-student framework uses the adaptive teacher-student (AT) framework as a baseline. Both the student and teacher models employ deformable detection transformers as basic detectors and ResNet as the backbone network.

[0038] Furthermore, the construction of the Adaptive Teacher-student framework includes: Infrastructure selection: Both the teacher and student models utilize a Deformable DETR detector, which comprises four core modules: Backbone, Deformable Transformer Encoder, Deformable Transformer Decoder, and a detection head. The Backbone uses ResNet50, the default backbone network for Deformable DETR, adaptable to multi-scale feature requirements. Both the Encoder and Decoder are 6-layer Deformable Transformer structures, effectively reducing computational cost through a sparse attention mechanism. The detection head consists of a classification head and a bounding box regression head. The classification head uses the cross-entropy loss function, while the bounding box regression head uses a combination of L1 and GIoU loss functions. Architecture data flow analysis: Source domain data path: Strongly augmented source domain images are input into the student model, and supervised loss is calculated using ground truth labels; Teacher path for target domain: The weakly augmented target domain image is input into the teacher model to generate pseudo-labels (filtered by dynamic threshold). Student path in target domain: The enhanced target domain image is input into the student model, and features are generated through bi-branch diffusion noise addition and denoising. At the same time, the unsupervised loss is calculated using teacher pseudo-labels and the contrastive loss is calculated using bi-branch features. Pseudo-label generation and EMA update for the teacher model: The teacher model and student model have completely identical structures. The teacher model updates weights only from the student model using EMA (Exponential Moving Average). Its core function is to provide high-quality pseudo-labels for students. The update formula is: ; in, and These represent the parameters of the teacher model and the student model, respectively. Hyperparameters used to control the smoothness of weights.

[0039] Data augmentation strategy analysis: Weak Augmentation: Used for image input teacher models in the target domain, it includes only two operations: random cropping and random horizontal flipping.

[0040] Strong Augmentation: Used for inputting images from the source and target domains into a student model, including two types of operations: Color transformation: color space conversion (such as RGB to HSV), contrast adjustment, histogram equalization, sharpening enhancement, and posterization. Spatial transformations: image rotation, shearing, and translation.

[0041] Furthermore, the process of defining the data path in the bi-branch diffusion comparison framework for teachers and students includes: The source domain image, after strong augmentation, is input into the student model, and the supervised loss of the source domain is calculated based on the true labels of the source domain. ; The weakly augmented target domain image is input into the teacher model to generate target domain pseudo-labels. ; The target domain image after strong augmentation is input into the student model, and the unsupervised loss of the target domain is calculated based on the pseudo-labels generated by the teacher model. The two-branch contrastive loss is calculated using the two-branch diffusion contrastive learning module. ; The parameters of the teacher model are updated using an exponential moving average through the parameters of the student model; Specifically, the parameters of the teacher model Instead of updating via backpropagation, it updates via the parameters of the student model. Perform Exponential Moving Average (EMA) updates: .

[0042] in, and These represent the parameters of the teacher model and the student model, respectively. Hyperparameters used to control the smoothness of weights.

[0043] Furthermore, the process of designing the Two-Branch Diffusion Contrast Learning (DBD-CL) module includes: Multi-scale features of strongly enhanced target domain images are extracted using the backbone network of the student model. A pre-trained diffusion model with frozen weights is used to perform two independent diffusion processes on multi-scale features. Each diffusion process includes noise addition and denoising operations to generate the first denoised feature and the second denoised feature. The first and second denoised features are used to form a feature pair for contrastive learning.

[0044] Furthermore, the two independent diffusion processes included: In the first diffusion process, the first noise and the first diffusion time step are randomly sampled. The multi-scale features are denoised according to the diffusion formula. The denoised features and the first diffusion time step are input into the diffusion model to obtain the first denoised features. In the second diffusion process, the second noise and the second diffusion time step are randomly sampled. The second noise is different from the first noise. The multi-scale features are denoised according to the diffusion formula. The denoised features and the second diffusion time step are input into the diffusion model to obtain the second denoised features. In this process, the two diffusion processes share the backbone network weights of the student model and the diffusion model weights, and the feature differences are created only through the differences in noise and diffusion time steps.

[0045] Furthermore, the weights of the diffusion model remain frozen during training; The two independent diffusion processes are achieved through independent noise sampling and independent diffusion time step sampling, wherein the diffusion time step is randomly selected within a preset range to ensure a balance between semantic preservation and noise difference.

[0046] Furthermore, the design of the Two-Branch Diffusion-Contrast Learning (DBD-CL) module includes: (1) Feature extraction: Input the strongly enhanced target domain image into the backbone of the student model to extract multi-scale features. ={ , , , }( (This corresponds to the output of the four stages of ResNet).

[0047] (2) Frozen Diffusion: A pre-trained diffusion model (such as the U-Net architecture) with frozen weights is used. The weights are frozen to preserve their strong domain generalization prior knowledge.

[0048] (3) Two-branch processing: for features ={ , , , Perform two independent diffusion processes, each including "noise addition" and "noise removal": Branch 1: Random Sampling Noise and time step ( [5,10], balancing noise differences and semantic preservation), noise selection: Gaussian noise (0, I), right ={ , , , Add noise; then input it into the diffusion model to generate noisy features. Then and Input diffusion model, output denoised features .

[0049] Branch 2: Randomly sampled independent noise ( ) and different time steps ( [5,10], Can be with (The noise levels must be different even if they are the same), repeat the noise addition and denoising steps to generate denoised features. .

[0050] (4) Feature pair generation: and This forms a feature pair that is semantically consistent but domain-differentiated, which is then used for subsequent comparative learning.

[0051] Furthermore, the two independent diffusion-denoising-denoising processes specifically include: First diffusion noise addition-denoising (generation) ): Random sampled noise sequence and time step ( [5,10], balancing noise differences and semantic preservation), noise selection: Gaussian noise (0, I) represents noise randomly sampled from a standard normal distribution; Adding noise: Generating noisy features according to the diffusion formula , Predefined noise scheduling; ; Noise reduction: and Input diffusion model, output denoised features (dimensions and) Consistent, C = 2048) Second diffusion noise addition-denoising (generation) ): Random sampled noise sequence ( ) and different time steps ( [5,10], Can be with (They are the same, but the noise levels must be different). Repeat the noise addition and denoising steps to generate denoised features. ; Weight sharing: The two diffusion processes share all weights of the student backbone and the diffusion model, only through... / , / Differences in manufacturing characteristics ensure and It is a "domain difference view of the same image".

[0052] Furthermore, the process of constructing the optimization objective of multi-loss fusion includes: Calculate the source domain supervised loss based on the true source domain labels; Calculate the unsupervised loss for the target domain based on the pseudo-labels generated by the teacher model; A domain discriminator is added after the backbone network, encoder, and decoder of the student model to calculate the adversarial alignment loss based on the source domain features and the target domain features. The bi-branch contrastive loss is calculated based on the feature pairs used for contrastive learning, which consist of the first and second denoised features.

[0053] Furthermore, the process of calculating the adversarial alignment loss includes: Global features are extracted from the backbone network output of the student model, token-level multi-scale features are extracted from the encoder, and instance features are extracted from the decoder. Input the global features, token-level multi-scale features, and instance features into the corresponding domain discriminators; The binary classification cross-entropy loss is calculated based on the output of the domain discriminator and the domain label, and the adversarial alignment loss is obtained.

[0054] Furthermore, the process of calculating the two-branch contrastive loss includes: Cosine similarity calculation is performed on the first and second denoised features of images in the same target domain. Cosine similarity calculation is performed on the denoising features of different images in the current batch; The calculated similarity value is input into the information noise contrast estimation loss function to obtain the two-branch contrast loss.

[0055] Furthermore, the optimization objectives of the multi-loss fusion involved in this embodiment include: The overall training loss of the student model needs to integrate three major objectives: "source domain basic ability", "target domain adaptation" and "domain difference robustness". It consists of four parts: source domain supervised loss, target domain unsupervised loss, adversarial alignment loss (to reduce the feature distribution gap between the source and target domains) and bi-branch diffusion contrast loss (to enhance the robustness of features to domain differences). The influence of each loss is balanced by weight coefficients.

[0056] Specifically, the overall training loss of the student model consists of four weighted components: in: Default is 1.0. To compare the loss weights (hyperparameter range 0.1~0.5, default 0.3, tuned with the validation set to balance the detection task and the objective of contrastive learning).

[0057] (1) (Supervised loss in source domain): Calculate the standard detection loss (classification, regression, GIoU loss) of Deformable DETR using ground truth labels in the source domain. (2) (Unsupervised loss in the target domain): Pseudo-labels generated by the teacher model and filtered by a dynamic threshold. Calculate the classification loss of the target domain image; (3) (Adversarial alignment loss): A domain discriminator is added after the Backbone, Encoder, and Decoder to bring the feature distributions of the source and target domains closer together; (4) (Two-branch contrastive loss): For the generated feature pairs and Calculate the InfoNCE contrastive loss. The goal of this loss is to "bring positive sample pairs closer together" (i.e., ...). and The "pushing away of negative sample pairs" (i.e., the diffusion features of other images within the mini-batch) forces the model to learn domain-invariant features that are insensitive to domain noise.

[0058] Furthermore, the loss function for the student model is defined as follows: Source domain has supervised loss ( Definition and calculation of ) Strongly enhanced source domain image With the corresponding truth label ( For the bounding box coordinates ([ , , , ]), (Target category label). The supervised loss is calculated using the standard detection loss of Deformable DETR, which includes three parts: classification loss, bounding box regression loss, and GIoU loss.

[0059] Classification loss ( Cross-entropy loss is used to constrain the consistency between the model's predicted probability of the target class and the true value. ; Where K is the total number of categories. Let be the predicted probability of the student model for the k-th class. One-hot encoding of the true value (1 for the correct class, 0 for the rest).

[0060] Bounding box regression loss ( Using L1 loss, the coordinate errors between the predicted bounding box and the ground truth box are constrained: ; in The bounding box coordinates predicted for the student model.

[0061] GIoU loss ( Based on L1 loss, further constraints are placed on the intersection-over-union (IoU) ratio of the bounding boxes to improve positioning accuracy. ; in (A is the area of ​​the smallest bounding rectangle, and U is the area of ​​the union of the two frames).

[0062] The total source domain supervised loss is: ; Unsupervised loss in target domain ( Definition and calculation of ) Strongly enhanced target domain image (Consistent with the source domain enhancement strategy, ensuring consistent domain alignment) and the filtered pseudo-labels generated by the teacher model. ( To predict the bounding box, To predict the category. False labels need to be filtered using a dynamic threshold, retaining only samples with confidence scores higher than the adaptive category threshold. Since the target domain has no ground truth labels, the unsupervised loss is only the classification loss, as shown in the following formula: ; in The calculation method is consistent with the source domain classification loss, only the ground truth label is included. Replace with teacher pseudo-tags .

[0063] Two-branch contrast loss ( Definition and calculation of ) Based on the dual-branch diffusion characteristics and The calculation aims to "bring together features from two branches of the same image and widen features from different images": ; Where N: mini-batch size; Temperature coefficient (default is 0.05, controls the similarity distinction and avoids gradient vanishing); Cosine similarity (calculated after standardizing features to avoid the influence of feature scale, formula: ).

[0064] Positive sample pairs are bi-branch features of the same target domain image ( , The negative samples are the bi-branch features of other target domain images within the mini-batch. .when and The higher the similarity, the better. The lower the similarity, the smaller the loss.

[0065] Adversity alignment loss ( Definition and calculation of ) These are connected to the outputs of the student's Backbone, Encoder, and Decoder, respectively, for domain alignment of "Global Features," "Token-Level Multi-Scale Features," and "Instance Features." The formula is as follows: in, For the features of the k-th module, For the domain label (source domain=0, target domain=1). This is the cross-entropy loss for binary classification.

[0066] Furthermore, multi-stage training strategies include: In the first stage, the student model is trained using only source domain supervised loss to obtain baseline detection capability; In the second stage, based on the supervised loss in the source domain, a bi-branch contrastive loss and an adversarial alignment loss are added to train the student model to learn domain-invariant features. In the third stage, a complete multi-loss fusion optimization objective is adopted. The teacher model is updated by exponential moving average, which enables the teacher model to generate pseudo-labels. The student model is then self-trained and optimized under the supervision of all losses.

[0067] Specifically, this embodiment adopts a multi-stage training process: Phase 1: Source Domain Pre-training. Only using... Train the Deformable DETR on the source domain to obtain the baseline model.

[0068] Phase Two: Cross-Domain Feature Adaptation. This involves combining source domain supervision. and the target domain and This allows the model to begin learning domain-invariant features.

[0069] Phase Three: Teacher-Student Self-Training. Utilizing the complete... The teacher model begins to update and generate pseudo-labels through EMA, while the student model self-trains under the supervision of all losses to further adapt to the target domain.

[0070] The method in this embodiment introduces the DBD-CL module, which effectively utilizes the diffusion model to generate high-quality domain-discrepancy feature pairs. It also significantly enhances the model's domain-invariant feature learning ability through contrastive learning. At the same time, the DDC-TS framework integrates multiple losses to optimize the use of pseudo-labels and solves the noise interference problem in the traditional teacher-student framework. As a result, it achieves superior performance in unsupervised domain adaptive object detection tasks.

[0071] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A domain-adaptive target detection method based on contrastive learning and diffusion models, characterized in that, include: Construct a two-branch diffusion comparison framework for teacher-student interaction that includes both student and teacher models; The data paths of the dual-branch diffusion comparison teacher-student framework are defined, including the source domain student path, the target domain teacher path, and the target domain student path, and the parameters of the teacher model are updated using the parameters of the student model based on the exponential moving average method. In the target domain student path of the student model, a two-branch diffusion contrast learning module is designed. Based on the two-branch diffusion contrast learning module, the target domain features are subjected to two independent diffusion processes using a frozen diffusion model to generate the first denoised feature and the second denoised feature. Construct an optimization objective that integrates multiple losses, wherein the optimization objective includes at least source domain supervised loss, target domain unsupervised loss, adversarial alignment loss, and bi-branch contrastive loss based on the first denoising feature and the second denoising feature; A multi-stage training strategy is adopted to train the student model based on the optimization objective, and the teacher model is updated using the trained student model through an exponential moving average method. The updated teacher model is then used to perform target detection on the target domain data to obtain the detection results. The process of defining the data path of the dual-branch diffusion comparison teacher-student framework includes: The enhanced source domain image is input into the student model, and the supervised loss of the source domain is calculated based on the true labels of the source domain. The weakly enhanced target domain image is input into the teacher model to generate target domain pseudo-labels; The enhanced target domain image is input into the student model. The unsupervised loss of the target domain is calculated based on the pseudo-labels generated by the teacher model. The two-branch contrastive loss is calculated through the two-branch diffusion contrastive learning module. The parameters of the teacher model are updated using an exponential moving average based on the parameters of the student model. The process of designing a dual-branch diffusion contrastive learning module includes: Multi-scale features of strongly enhanced target domain images are extracted using the backbone network of the student model. The multi-scale features are subjected to two independent diffusion processes using a pre-trained diffusion model with frozen weights. Each diffusion process includes noise addition and denoising operations to generate a first denoised feature and a second denoised feature. The first denoised feature and the second denoised feature are used to form a feature pair for contrastive learning.

2. The method according to claim 1, characterized in that, The dual-branch diffusion contrastive teacher-student framework uses the adaptive teacher-student framework as the baseline. Both the student model and the teacher model use deformable detection transformers as basic detectors and residual networks as backbone networks.

3. The method according to claim 1, characterized in that, The two independent diffusion processes include: In the first diffusion process, the first noise and the first diffusion time step are randomly sampled, and the multi-scale features are denoised according to the diffusion formula. The denoised features and the first diffusion time step are then input into the diffusion model to obtain the first denoised features. In the second diffusion process, the second noise and the second diffusion time step are randomly sampled, where the second noise is different from the first noise. The multi-scale features are denoised according to the diffusion formula. The denoised features and the second diffusion time step are input into the diffusion model to obtain the second denoised features. In this process, the two diffusion processes share the backbone network weights of the student model and the diffusion model weights, and the feature differences are created only through the differences in noise and diffusion time steps.

4. The method according to claim 1, characterized in that, The weights of the diffusion model remain frozen during training; The two independent diffusion processes are achieved through independent noise sampling and independent diffusion time step sampling, wherein the diffusion time step is randomly selected within a preset interval to ensure a balance between semantic preservation and noise difference.

5. The method according to claim 1, characterized in that, The process of constructing an optimization objective through multi-loss fusion includes: Calculate the source domain supervised loss based on the true source domain labels; Calculate the unsupervised loss for the target domain based on the pseudo-labels generated by the teacher model; A domain discriminator is added after the backbone network, encoder, and decoder of the student model to calculate the adversarial alignment loss based on the source domain features and the target domain features. The bi-branch contrastive loss is calculated based on the feature pairs used for contrastive learning, which consist of the first and second denoised features.

6. The method according to claim 5, characterized in that, The process of calculating adversarial alignment loss includes: Global features are extracted from the backbone network output of the student model, token-level multi-scale features are extracted from the encoder, and instance features are extracted from the decoder. The global features, token-level multi-scale features, and instance features are respectively input into the corresponding domain discriminators; The binary classification cross-entropy loss is calculated based on the output of the domain discriminator and the domain label, and the adversarial alignment loss is obtained.

7. The method according to claim 5, characterized in that, The process of calculating the two-branch contrastive loss includes: Cosine similarity calculation is performed on the first and second denoised features of images in the same target domain. Cosine similarity calculation is performed on the denoising features of different images in the current batch; The calculated similarity value is input into the information noise contrast estimation loss function to obtain the two-branch contrast loss.

8. The method according to claim 1, characterized in that, The multi-stage training strategy includes: In the first stage, the student model is trained using only source domain supervised loss to obtain baseline detection capability; In the second stage, based on the supervised loss in the source domain, a bi-branch contrastive loss and an adversarial alignment loss are added to train the student model to learn domain-invariant features. In the third stage, a complete multi-loss fusion optimization objective is adopted. The teacher model is updated by exponential moving average, which enables the teacher model to generate pseudo-labels. The student model is then self-trained and optimized under the supervision of all losses.

Images