Domain adaptive object detection method and system based on complementary mask and cyclic training

Abstract

This invention provides a domain-adaptive object detection method and system based on complementary masks and cyclic training, belonging to the field of object detection technology. The invention first acquires labeled source domain images and unlabeled target domain images, generates weak / strong enhancement samples, and constructs teacher and student networks. In the pre-training phase: complementary and non-overlapping binary masks are generated from the target domain images and applied to the teacher and student networks respectively; the student network is optimized by combining source domain supervision and Hungarian matching; the teacher network parameters are updated using exponential moving average (EMA), and the optimal weights are used to initialize the next stage of the network. In the self-training phase: adversarial feature alignment and hybrid pseudo-label training based on target domain sample caching are alternately executed. Multiple feature alignments and single pseudo-label trainings are organized in small-cycle cyclical manner to improve pseudo-label quality and suppress error accumulation, thereby enhancing the accuracy and robustness of target domain detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a domain-adaptive target detection method and system based on complementary masks and cyclic training. Background Technology

[0002] General object detection aims to locate and classify objects in images. As a fundamental task in computer vision, it has various applications, such as defect detection, object tracking, and autonomous driving. However, in real-world scenarios, the assumption that "training (source domain) data and test (target domain) data follow the same distribution" does not always hold true. Research shows that distribution shifts can lead to a decrease in object detection accuracy across different domains. Therefore, Unsupervised Domain Adaptive Object Detection (UDAOD) has emerged as a key solution for achieving cross-domain generalization and mitigating the accuracy degradation caused by distribution shifts.

[0003] Unsupervised Domain Adaptive Object Detection (UDAOD) methods can generalize object detection models trained on labeled source domains to unlabeled target domains without requiring additional annotation. Early UDAOD methods reduced differences in appearance and feature distribution between different domains through adversarial feature alignment and image style transfer techniques. However, the lack of target domain annotations limited the performance of these methods.

[0004] In recent years, teacher-student frameworks have dominated unsupervised domain adaptive object detection (UDAOD) due to their ability to generate pseudo-labels for the target domain. Existing methods typically employ a two-stage training process: pre-training and self-training. First, the student network is pre-trained on labeled source domain data. Second, the teacher network is initialized from the pre-trained student network parameters, then generates pseudo-labels for the unlabeled target domain data. These pseudo-labels are combined with the source domain labels to supervise the training of the student network. Furthermore, the student model incorporates an adversarial alignment technique based on gradient inversion layers (GRL) to bridge inter-domain gaps and suppress false positives.

[0005] However, due to the lack of understanding of the target domain during the pre-training phase, the teacher network, initialized by the pre-trained student network, often generates sparse and inaccurate pseudo-labels. These low-quality pseudo-labels may cause the student network to learn incorrect semantic features during the self-training phase, thus affecting the object detection capability. Although some noisy pseudo-labels can be filtered out through simple thresholding and GRL-based adversarial feature alignment (AFA), residual noise will still propagate back to the teacher network through exponential moving average (EMA) parameter updates, causing the student network to gradually overfit to the noisy signal, ultimately resulting in performance degradation.

[0006] In summary, there is currently a lack of methods to address the training shortcomings of teacher-student frameworks in domain-adaptive object detection, thus meeting the high robustness requirements of intelligent driving perception systems in complex dynamic environments. This includes addressing the performance degradation issues caused by domain differences in intelligent driving, specifically performance degradation in scenarios such as severe weather, different urban street environments, and insufficient generalization ability when transferring synthetic training data to real-world road data. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, the purpose of this invention is to provide a domain adaptive target detection method and system based on complementary masking and cyclic training. Through complementary masking pre-training and small-cycle cyclic self-training, the quality of target domain pseudo-labels is effectively improved and the cumulative effect of erroneous pseudo-labels is significantly suppressed.

[0008] To achieve the above objectives, the present invention provides the following solution: A domain-adaptive object detection method based on complementary masks and cyclic training includes: Acquire labeled source domain images and unlabeled target domain images, and generate weakly enhanced images and strongly enhanced images based on the target domain images; Build teacher and student networks; During the pre-training phase, a pair of complementary and non-overlapping binary masks are generated based on the target domain image and applied to the teacher network and the student network respectively; both the teacher network and the student network are randomly initialized; the teacher network generates pseudo-labels based on the weakly enhanced image and obtains a pseudo-label set by filtering according to a threshold, and uses Hungarian matching to pair the pseudo-label set with the output of the student network, while the student network is trained under supervision based on the source domain image labels; During the self-training phase, adversarial feature alignment and hybrid pseudo-label training are performed alternately. Both the teacher network and the student network are initialized with the best-performing model from the pre-training phase. The adversarial feature alignment involves adding a domain discriminator and setting a gradient inversion layer after the student network. The hybrid pseudo-label training involves setting a target domain sample cache. The teacher network generates pseudo-labels based on the weakly enhanced image and combines them with the corresponding strongly enhanced image. Cache samples are obtained from the target domain sample cache, and hybrid data augmentation and mosaic data augmentation are performed respectively before training the student network. The target domain sample cache is then updated with the strongly enhanced image with pseudo-labels. The self-training phase is divided into multiple small cycles, each of which includes several adversarial feature alignments and a single hybrid pseudo-label training. After completing all the small cycles, the student network for target domain object detection is output.

[0009] Preferably, generating a weakly enhanced image and a strongly enhanced image based on the target domain image includes: Perform a first enhancement operation on the target domain image to generate the weakly enhanced image; A second enhancement operation is performed on the target domain image to generate the strongly enhanced image; the second enhancement operation includes more enhancement types than the first enhancement operation.

[0010] Preferably, during the pre-training phase, a pair of complementary and non-overlapping binary masks are generated based on the target domain image and applied to the teacher network and the student network respectively, including: Randomly select several mask positions in the spatial location of the target domain image to generate a first binary mask; A second binary mask is generated based on the first binary mask, such that the first binary mask and the second binary mask have opposite values ​​at the same spatial location, thereby forming a complementary and non-overlapping binary mask pair; The first binary mask and the second binary mask are respectively input into the teacher network and the student network as mask constraints for the corresponding networks.

[0011] Preferably, during the pre-training phase, a pair of complementary and non-overlapping binary masks are generated based on the target domain image and applied to the teacher network and the student network respectively, including: Feature maps of the target domain image are extracted from the backbone networks of the teacher network and the student network, respectively. Map the binary mask to a size consistent with the feature map; The feature values ​​at the masked locations are set to zero based on the binary mask, so that the zeroed features do not participate in subsequent feature fusion and prediction calculations.

[0012] Preferably, the teacher network generates pseudo-labels based on the weakly enhanced image and filters them according to a threshold to obtain a pseudo-label set, including: The teacher network outputs candidate detection results and corresponding feature embeddings for the weakly enhanced image; the candidate detection results include category prediction results and bounding box prediction results. A confidence threshold is set based on the candidate detection results, and the pseudo-label set is obtained by filtering according to the confidence threshold; Each pseudo-label in the pseudo-label set is associated with its corresponding category information, bounding box information, and corresponding feature embedding to form a pseudo-label representation for training.

[0013] Preferably, Hungarian matching is used to pair the pseudo-label set with the student network output, including: Construct a matching cost matrix; the matching cost matrix includes a classification cost term, a bounding box regression cost term, and an intersection-union ratio (IU / UCR) cost term. Hungarian matching is performed based on the matching cost matrix to obtain pairing relationships, and the paired student network output is determined based on the pairing relationships; The unsupervised training loss is calculated based on the paired student network output and the pseudo-label set, and is used together with the supervised training loss calculated based on the source domain image labels to train the student network; the teacher network updates its parameters using an exponential moving average mechanism.

[0014] Preferably, the adversarial feature alignment involves adding a domain discriminator after the student network and setting a gradient inversion layer, including: Set domain labels for the source domain image and the target domain image respectively; The features output by the student network are input into the domain discriminator to output the domain discrimination result, and the domain discrimination loss is calculated based on the domain discrimination result and the domain label; The gradient reversal layer is set between the student network and the domain discriminator to reverse the gradient direction and update the parameters of the student network during backpropagation, thereby enabling the student network to learn domain-invariant features; the teacher network updates its parameters through an exponential moving average mechanism.

[0015] Preferably, the hybrid pseudo-label training involves setting up a target domain sample cache, whereby the teacher network generates pseudo-labels based on the weakly enhanced image and combines them with the corresponding strongly enhanced image, samples cached samples from the target domain sample cache, performs hybrid data augmentation and mosaic data augmentation on each sample, trains the student network, and updates the target domain sample cache with the strongly enhanced image containing pseudo-labels, including: The target domain samples and their corresponding pseudo-labels are stored in the target domain sample cache, and the cached samples are output from the target domain sample cache based on a preset sampling rule; Perform hybrid data augmentation on the cached samples and the strongly enhanced images. The hybrid data augmentation includes weighting the two images according to the hybrid coefficient and merging the pseudo-labels corresponding to the two images to generate hybrid training samples. Mosaic data augmentation is performed on the cached samples. The mosaic data augmentation includes sampling four cached samples and stitching the four cached samples into a synthetic image, while merging the pseudo-labels corresponding to the four cached samples to generate synthetic training samples. The mixed training samples and the synthetic training samples are input into the student network to complete the training, and the target domain sample cache is updated based on the latest strongly enhanced image with pseudo-labels; the teacher network updates its parameters through an exponential moving average mechanism.

[0016] Preferably, the self-training phase is divided into multiple small cycles, each cycle containing several adversarial feature alignments and a single hybrid pseudo-label training iteration. After completing all the small cycles, the student network for target domain object detection is output, including: Within each of the aforementioned small cycles, the adversarial feature alignment is first performed a preset number of times to update the student network; Then perform a single training iteration of the hybrid pseudo-label to update the student network and update the target domain sample cache; After completing the current small cycle, the next small cycle is entered, and in the next small cycle, the processes of "continuously performing the adversarial feature alignment a preset number of times to update the student network" and "performing a single hybrid pseudo-label training to update the student network and update the target domain sample cache" are repeated until all the small cycles are completed, and then the student network for target domain object detection is output.

[0017] A domain-adaptive object detection system based on complementary masks and cyclic training includes: The source domain and target domain data acquisition and enhancement unit is used to acquire labeled source domain images and unlabeled target domain images, and generate weakly enhanced images and strongly enhanced images based on the target domain images. The teacher network and student network construction unit is used to build teacher networks and student networks; A complementary mask pre-training unit is used to generate a pair of complementary and non-overlapping binary masks based on the target domain image during the pre-training phase and apply them to the teacher network and the student network respectively; both the teacher network and the student network are randomly initialized; the teacher network generates pseudo-labels based on the weakly enhanced image and obtains a pseudo-label set by filtering according to a threshold, and uses Hungarian matching to pair the pseudo-label set with the output of the student network, while simultaneously supervising the training of the student network based on the source domain image labels; A recurrent self-training unit is used to alternately perform adversarial feature alignment and hybrid pseudo-label training during the self-training phase. Both the teacher network and the student network are initialized by the best-performing model in the pre-training phase. The adversarial feature alignment involves adding a domain discriminator and setting a gradient inversion layer after the student network. The hybrid pseudo-label training involves setting a target domain sample cache. The teacher network generates pseudo-labels based on the weakly enhanced image and combines them with the corresponding strongly enhanced image. It samples cached samples from the target domain sample cache and performs hybrid data augmentation and mosaic data augmentation on each sample before training the student network. Finally, it updates the target domain sample cache with the strongly enhanced image containing pseudo-labels. The loop organization and output unit is used to divide the self-training phase into multiple small cycles. Each small cycle includes several adversarial feature alignments and a single hybrid pseudo-label training. After completing all the small cycles, the student network for target domain object detection is output.

[0018] The present invention discloses the following technical effects: (1) This invention introduces source and target domain information simultaneously during the pre-training stage based on complementary and non-overlapping binary masks, enabling the teacher network to obtain discriminative knowledge of the target domain before self-training. This alleviates the problem of sparse and distorted pseudo-labels generated by existing teacher-student frameworks due to insufficient understanding of the target domain by the teacher network. This technical solution improves the quality of pseudo-labels, reduces background misclassification and class bias, and helps to improve the stability and performance ceiling of subsequent self-training convergence.

[0019] (2) This invention employs an alternating cyclical strategy of adversarial feature alignment and hybrid pseudo-label training to decouple the feature alignment process from the pseudo-label utilization process in time, avoiding mutual interference caused by direct coupling between the two, and significantly suppressing the performance collapse problem of existing self-training methods in the process of erroneous pseudo-label accumulation and propagation. This scheme reduces class noise response, enhances the robustness of target domain recognition, improves the utilization rate of correct pseudo-labels, and helps to improve cross-domain generalization ability and error tolerance.

[0020] (3) This invention integrates the above training process through a small-cycle cyclic organization method, which increases the number of correct pseudo-labels and reduces the negative impact of incorrect pseudo-labels in each small cycle, overcoming the network degradation and performance decline caused by residual noise pseudo-labels propagating through the exponential moving average parameter in existing teacher-student framework methods. This scheme improves the convergence stability of the target domain scene and maintains the continuous improvement trend of model performance in complex domain offset scenes. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of the method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the system structure provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The purpose of this invention is to provide a domain-adaptive target detection method and system based on complementary masks and cyclic training. By pre-training with complementary masks and self-training with short-cycle cyclic training, the domain alignment stability is enhanced and the robustness of the model in cross-domain detection tasks is significantly improved.

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Figure 1 The method flowchart provided in the embodiments of the present invention is as follows: Figure 1 As shown, this invention provides a domain-adaptive target detection method based on complementary masks and cyclic training, comprising: Step 100: Obtain the labeled source domain image and the unlabeled target domain image, and generate a weakly enhanced image and a strongly enhanced image based on the target domain image; Step 200: Construct teacher and student networks; Step 300: In the pre-training phase, a pair of complementary and non-overlapping binary masks are generated based on the target domain image and applied to the teacher network and student network respectively; both the teacher network and student network are randomly initialized; the teacher network generates pseudo-labels based on the weakly enhanced image and obtains a pseudo-label set by filtering according to a threshold, and uses Hungarian matching to pair the pseudo-label set with the output of the student network, while the student network is trained under supervision based on the source domain image labels.

[0027] Step 400: During the self-training phase, adversarial feature alignment and hybrid pseudo-label training are performed alternately. Both the teacher network and the student network are initialized with the best-performing model from the pre-training phase. Adversarial feature alignment involves adding a domain discriminator and setting a gradient inversion layer after the student network. Hybrid pseudo-label training involves setting up a target domain sample cache. The teacher network generates pseudo-labels based on weakly enhanced images and combines them with the corresponding strongly enhanced images. Cache samples are obtained from the target domain sample cache, and hybrid data augmentation and mosaic data augmentation are performed respectively before training the student network. The target domain sample cache is then updated with strongly enhanced images containing pseudo-labels. Step 500: Divide the self-training phase into multiple small cycles. Each small cycle contains several adversarial feature alignments and a single mixed pseudo-label training. After completing all small cycles, output the student network for target domain object detection.

[0028] Specifically, in this embodiment, to verify the applicability and effectiveness of the domain adaptive target detection method based on complementary masks and cyclic training in typical cross-domain scenarios, three representative domain adaptive tasks were selected for experimental evaluation: Cityscapes→FoggyCityscapes (weather adaptation scenario), Cityscapes→BDD100K-Daytime (street scene change scenario), and Sim10k→Cityscapes (car) (synthesized to a real-world scenario).

[0029] Specifically, Cityscapes is a real street scene dataset, collected from street scenes of 50 cities under clear weather conditions, containing 2975 training images. The labeled categories cover eight core categories, including pedestrians, cars, trucks, buses, trains, motorcycles, and bicycles. When used as the source or target domain, it can provide the network with fine-grained annotation information for multi-category urban traffic scenes. Foggy Cityscapes is a synthetic foggy dataset generated based on Cityscapes, obtained by applying simulated fog effects with densities of 0.005, 0.01, and 0.02 to Cityscapes images. Its training and validation set partitioning method is consistent with Cityscapes. In this embodiment, the version with a fog density of 0.02 is selected as the target dataset for constructing the Cityscapes→FoggyCityscapes weather adaptation task.

[0030] BDD100K is a large-scale autonomous driving benchmark dataset covering various weather conditions and urban scenes. This embodiment uses its daytime subset, containing 36,728 training images and 5,258 test images. During evaluation, seven core categories consistent with Cityscapes (excluding the "train" category) are selected for alignment to construct the Cityscapes→BDD100K-Daytime street scene change domain adaptation task. Sim10k is a synthetic dataset generated by the GTA5 game engine, containing 10,000 images and 58,701 labeled instances, all of which belong to the "car" category. This embodiment uses Sim10k as the source domain and the "car" category in Cityscapes as the target domain to construct the Sim10k→Cityscapes (car) scene to evaluate the transfer capability of this invention for single-class vehicle detection tasks under synthetic to real-world cross-domain conditions.

[0031] In a specific implementation of the present invention, the domain adaptive target detection method based on complementary masks and cyclic training is executed according to steps 100 to 500. The source domain images and target domain images in the three domain adaptive scenarios of Cityscapes→Foggy Cityscapes, Cityscapes→BDD100K-Daytime, and Sim10k→Cityscapes (car) are respectively used as the labeled source domain image and unlabeled target domain image input in step 100. Subsequently, in step 200, a teacher network and a student network are constructed. In step 300, complementary and non-overlapping binary masks are applied to the target domain images and pre-training is completed. In step 400, adversarial feature alignment and hybrid pseudo-label training are performed on the three domain adaptive scenarios respectively. In step 500, the self-training phase is divided into multiple small cycles to complete cyclic training and output the student network for target detection in the corresponding target domain.

[0032] During the training phase, based on the teacher and student networks constructed in step 200, this invention introduces a complementary masking strategy in the pre-training phase of step 300. Specifically, using the target domain image as input, multi-scale feature maps are extracted by the ResNet50 backbone network, which shares a structure with the student and teacher networks. These multi-scale features are used as the basic feature representation for complementary masking processing, and are subsequently used to generate mask multi-scale features and participate in the consistency constraints between the teacher and student networks.

[0033] In step 300, a pair of complementary and non-overlapping binary masks are generated for the same target domain image. These masks are fused with the target domain multi-scale features of the teacher network and student network, respectively, to obtain corresponding mask multi-scale features. These mask multi-scale features are then input into the encoder-decoder structure of Deformable DETR to complete feature modeling. During the pre-training phase, the teacher network generates pseudo-labels based on the weakly enhanced image and selects a pseudo-label set according to a threshold. Hungarian matching is used to pair the pseudo-label set with the prediction results and corresponding feature vectors output by the student network. Simultaneously, the source domain image is input into the student network to calculate the supervised loss, thereby jointly optimizing the student network under the constraint of complementary masks. The teacher network is then initialized with the parameters of the student network for subsequent self-training phases.

[0034] Compared to most existing teacher-student framework methods that only use source domain data during the pre-training phase or require the introduction of additional sub-networks to learn target domain knowledge, this invention directly injects target domain information into the existing teacher and student networks in step 300 through a complementary masking strategy. This explicitly improves the pseudo-label quality of the teacher network in the target domain without adding extra network parameters, avoiding the accumulation of errors caused by inaccurate pseudo-labels. Under the same network structure and parameter scale, this invention achieves an mAP improvement of approximately 9.9% compared to the baseline network on typical unsupervised adaptive object detection tasks, effectively balancing target domain knowledge learning ability and model complexity control.

[0035] In the self-training phase of this invention, the training process corresponding to step 400 includes two complementary processes: adversarial feature alignment and hybrid pseudo-label training. Specifically, the adversarial feature alignment process connects to the domain discriminator after the student network's backbone network ResNet50, Deformable DETR encoder, and Deformable DETR decoder, and a gradient inversion layer is placed between the student network and the domain discriminator. During self-training, the gradient inversion layer inverts the gradient direction fed back to the student network, guiding the student network to learn feature representations that can deceive the domain discriminator—i.e., domain-invariant features—while simultaneously receiving inputs from both the source and target domain images. This effectively alleviates the domain gap between the source and target domains, improving the student network's target detection performance in the target domain.

[0036] In the hybrid pseudo-label training process of step 400, a target domain sample cache is set up to store target domain images and their corresponding pseudo-labels. Each time a target domain image is trained, a weakly enhanced image of the target domain image is input into the teacher network to generate pseudo-labels. The generated pseudo-labels are then combined with the corresponding strongly enhanced images to form strongly enhanced target domain samples with pseudo-labels. Subsequently, two cached samples are sampled from the target domain sample cache. One cached sample is combined with the aforementioned strongly enhanced target domain samples with pseudo-labels to perform MixUp data augmentation, while the other cached sample undergoes Mosaic data augmentation, resulting in two types of enhanced target domain training samples. These enhanced target domain training samples, along with the source domain images, are input into the student network for supervised training, while the target domain sample cache is updated using the latest trained pseudo-labels. Through this process, the student network fully utilizes high-quality pseudo-labels while being less likely to learn incorrect pseudo-labels, significantly reducing the negative impact of incorrect pseudo-labels on network performance. Compared to existing methods that simply filter out some erroneous pseudo-labels through thresholds but still cause errors to accumulate due to residual noise, the hybrid pseudo-label training of this invention can effectively allow the model to ignore or mitigate the negative impact of erroneous pseudo-labels, significantly improving the robustness of the network.

[0037] In step 500, to coordinate the roles of adversarial feature alignment and hybrid pseudo-label training in the self-training phase, this invention introduces a cyclic training strategy, dividing the self-training phase into multiple small cycles. Each small cycle includes several adversarial feature alignments and a single hybrid pseudo-label training. Specifically, within each small cycle, adversarial feature alignment is first performed multiple times consecutively to reduce the feature distribution difference between the source and target domains. Then, hybrid pseudo-label training is performed once to increase the number of correct pseudo-labels and mitigate the negative impact of incorrect pseudo-labels. By repeating the above sequential training method in small cycles, the mutual interference caused by the direct superposition of adversarial feature alignment and hybrid pseudo-label training can be reduced. On the other hand, the respective advantages of both in reducing inter-domain gaps and improving pseudo-label quality can be fully utilized. This allows the student network to obtain higher-quality domain-invariant feature representations and more stable target domain detection performance after completing all small cycles. This cyclic training strategy is an original design in existing domain-adaptive object detection methods.

[0038] As an optional implementation, in the pre-training stage of this invention, corresponding to step 300, the student network is first trained using source domain data to acquire basic category recognition capabilities. Then, the trained student network parameters are used to initialize the teacher network for pseudo-label generation in the subsequent self-training stage. However, because the teacher network has not been sufficiently exposed to target domain data during the pre-training stage and lacks adequate target domain knowledge, its pseudo-labels on the target domain image are of low quality, easily leading to incorrect category predictions or misclassification of background regions as foreground regions. This results in the student network learning incorrect category representations during the self-training stage.

[0039] To address the aforementioned limitations, this invention introduces both source domain data and target domain data into the pre-training process in step 300, and employs a complementary masking strategy to enable the student network to learn discriminative knowledge of objects in the target domain even when target domain information is insufficient. Specifically, the teacher network receives more complete target domain information and generates pseudo-labels, while the student network, under the constraint of complementary masking, only receives partial target domain features but must still be consistent with the teacher network's output, allowing the student network to gradually build the ability to recognize target objects in the target domain. Through this method, the teacher network can output higher-quality target domain pseudo-labels based on further learning of target domain information, thereby providing more accurate pseudo-supervision signals for the subsequent self-training stage.

[0040] Furthermore, this embodiment uses a Deformable DETR to instantiate the teacher network. and student network The detector comprises a ResNet50 backbone network, a deformable Transformer encoder, and a deformable Transformer decoder. This embodiment inputs a weakly enhanced target domain image. Simultaneously input the source domain image and the strongly enhanced target domain image. Since both networks use the same processing flow, the following explanation will use the teacher branch as an example. The backbone network extracts multi-scale feature maps, denoted as... ,in k represents the feature level. C, H, and W represent the number of channels, height, and width, respectively. Subsequently, a mask with a specified aspect ratio is generated. and patch size A random binary mask is obtained, and its size is adjusted by bilinear interpolation to obtain a multi-scale mask. ,in Next, multi-scale masks will be applied. With the corresponding feature map The input is a Deformable DETR encoder. During the deformable attention computation, each mask is applied to the value matrix to achieve zero-padding.

[0041] Next, the decoder outputs a semantic sequence to generate the prediction result. To further obtain target-level representations and mitigate potential interference, this embodiment introduces an additional multilayer perceptron (MLP) to project the output sequence into feature embeddings. ,in Indicates the number of embedding vectors. This represents the embedding dimension. Therefore, the teacher network... It will output feature embeddings and their corresponding prediction results Meanwhile, student networks It follows the exact same process, but uses complementary multi-scale masks. Generate feature embeddings and prediction results .

[0042] Subsequently, this embodiment sets a threshold. To filter out pseudo-labels of relatively high quality Each Each corresponds to a feature embedding Next, Hungarian matching is used to identify the pseudo-tags. Student network prediction results Perform matching to obtain the prediction results after pairing. And obtain the paired feature embeddings accordingly. By minimizing and The mean squared error between them is defined in this embodiment as unsupervised feature loss. as follows: In addition to consistency between paired feature embeddings, the teacher network With student network Alignment in classification results is equally crucial for object detection tasks. However, complementary masking strategies can lead to discrepancies in bounding box predictions, hindering student networks. This approach integrates complete target context information. To mitigate this issue, this embodiment introduces unsupervised bounding box regression loss. Loss of Generalized Intersection over Union (GIoU) This is to impose additional spatial constraints.

[0043] The overall definition of unsupervised detection loss is as follows: in Includes classification prediction results and bounding box prediction results. This represents the unsupervised classification loss. Therefore, the total unsupervised loss during the pre-training phase combines the feature-level loss and the detection-level loss, and is defined as: In the formula This is the balance coefficient.

[0044] Furthermore, to achieve the transfer of source domain knowledge, this embodiment also introduces a standard supervised detection loss. .

[0045] Finally, the overall loss function in the pre-training phase combines supervised and unsupervised loss, and is defined as follows: in The weighting factor for unsupervised loss.

[0046] Furthermore, the pre-training phase based on recurrent training in this embodiment includes adversarial feature alignment and hybrid pseudo-label training, as detailed below: (1) Adversarial feature alignment: because To simultaneously process source and target domain images, this embodiment introduces adversarial loss into the network to align features between the two domains. For adversarial training, this embodiment treats the backbone network, encoder, and decoder as feature extractors. A lightweight convolutional neural network (CNN) base domain discriminator is connected after each component. This is used to identify the source domain of a feature. Specifically, in this embodiment, a domain label is assigned to the source domain. Assign domain labels to the target domain The discriminator is trained using binary cross-entropy loss. To ensure the feature extractor Generate a domain discriminator To address this characteristic, this embodiment embeds a Gradient Reversal Layer (GRL) between the two layers to achieve mini-max optimization. During backpropagation, the GRL reverses the gradient direction to support adversarial training, and its optimization objective function is defined as follows: This adversarial optimization implicitly performs density ratio estimation, where the discriminator aligns the feature distributions of the source and target domains to a consistent level, thereby facilitating the learning of domain-invariant representations. Furthermore, this embodiment incorporates a standard self-training loss, which includes an unsupervised classification loss. and supervised detection loss Therefore, the overall loss function for Adversarial Feature Alignment (AFA) is defined as follows: in and These represent the weighting factors for unsupervised classification loss and adversarial loss, respectively.

[0047] (2) Hybrid pseudo-label training: First, a high confidence threshold is used. For the self-training phase of the teacher network The prediction results are filtered to obtain reliable pseudo-labels. These pseudo-labels are then combined with the corresponding strongly enhanced target domain images and input into the MixPL (Mix Data Enhancement) module for further processing. This module integrates two strategies: Mixup and Mosaic data enhancement, as detailed below: 1) Mixup Enhancement Given a strongly augmented target domain training batch (in (Indicates batch size), pseudo-label From the self-training phase teacher network Through weak enhancement input Generate. The Mixed Pseudo-Tag Module (MixPL) is cached from the target domain. Extract an equal amount of pseudo-label samples Then, the images and pseudo-labels in the cache and training batches are mixed and concatenated, respectively, according to the following formula: in Indicates a splicing operation, mixing coefficient Set to 0.5. This hybrid data augmentation strategy generates a hybrid sample set. Each blended image is obtained by linearly combining images from two input batches, and the corresponding blended pseudo-label is the union of the pseudo-labels from the two batches. This operation can be understood as introducing uncertainty-aware interpolation into the pseudo-label samples, implicitly modeling the uncertainty between features and labels within the target domain—this aligns with the core idea of ​​uncertainty-aware generative models, which improve robustness to noisy or uncertain supervisory signals by explicitly capturing prediction uncertainty.

[0048] In summary, Mixup data augmentation has two significant advantages: first, it forces student networks to adapt during the self-training phase. First, learning smoother decision boundaries reduces reliance on labels for individual target domain samples, thus enhancing robustness against erroneous labels. Second, mixed samples dilute the features of missed targets, labeling them as hard examples, further reducing the impact of such samples on the student network during self-training. The negative impact.

[0049] 2) Mosaic enhancement The Mixed Pseudo-Label Module (MixPL) caches data from the target domain. Four sets of pseudo-label target sample pairs were sampled. After cropping and downsampling each sample, they are placed in the four quadrants of a shared canvas, and the corresponding pseudo-labels are merged into a new label set. Each quadrant contains a transformed sample, which ultimately forms a synthetic sample. .

[0050] Mosaic data augmentation has two core advantages: First, by fusing four images with diverse semantic content, the model is exposed to a wider range of target appearance and background information, thereby accelerating the training process; second, false negative results can be considered as the product of true positive targets after scale reduction and occlusion transformation. Therefore, the visual features of small targets after mosaic augmentation are similar to the features of small-scale or occluded foreground targets in false negative samples. After learning these features, it is possible to identify the real foreground objects that were originally misclassified as background and to classify them correctly.

[0051] After Mixup and Mosaic data augmentation, the target domain samples in the training batch are replaced with mixed and synthetic samples for optimizing the student network. The confidence threshold used in this embodiment... Higher than the settings in related studies—experimental observations show that higher confidence thresholds lead to better performance.

[0052] The unsupervised loss used in Hybrid Pseudo-Label Training (MPLT) contains only classification terms and is defined as follows: Since the confidence level of pseudo-labels primarily reflects classification certainty rather than localization accuracy, introducing a localization loss may impair the effectiveness of Hybrid Pseudo-Label Training (MPLT). Furthermore, to achieve source domain knowledge transfer, this embodiment incorporates a supervised detection loss based on labeled source domain data. The final overall loss function is defined as follows: in The weighting factor for unsupervised loss.

[0053] After the Hybrid Pseudo-Label Training (MPLT) process is completed, the teacher network in the self-training phase... Generated pseudo tags and its corresponding strongly enhanced target domain image The update will be performed in the cache.

[0054] (3) Circuit training: AFA and MPLT play complementary but potentially conflicting roles. AFA learns domain-invariant features by minimizing domain differences through adversarial loss; while MPLT regulates training through sample mixing to combat the influence of noisy pseudo-labels. Simultaneously optimizing these conflicting objectives can lead to mutual interference, with the mixed features introduced by MPLT contributing to this interference. Therefore, to resolve this mutual interference, this embodiment introduces a cyclic training strategy.

[0055] Specifically, this embodiment first runs MPLT training. After one epoch, the teacher network for the self-training phase is initialized using the best-performing model. and student network This is to enhance the robustness of the self-training (TS) framework to noisy pseudo-labels. The total number of epochs for cyclic training is defined as... and divided into 1 cycle (where (This represents the number of epochs in each loop). Within each loop, execution alternates. AFA and epoch MPLT is performed for several epochs to reduce the negative impact of noisy pseudo-labels and increase the number of correct pseudo-labels. The loss function for cyclic training is defined as follows: } in This indicates the current epoch number of the training. Furthermore, to reduce the number of parameters, the teacher network during the training phase... With student network Parameters are shared in MPLT and AFA.

[0056] The Cyclic Training (CT) strategy effectively leverages the complementary advantages of MPLT and AFA while mitigating potential interference between the two.

[0057] Corresponding to the above method, this embodiment also provides a domain adaptive target detection system based on complementary masks and cyclic training, including: The source domain and target domain data acquisition and enhancement unit is used to acquire labeled source domain images and unlabeled target domain images, and generate weakly enhanced images and strongly enhanced images based on the target domain images. The teacher network and student network construction unit is used to build teacher networks and student networks; A complementary mask pre-training unit is used to generate a pair of complementary and non-overlapping binary masks based on the target domain image during the pre-training phase and apply them to the teacher network and the student network respectively; both the teacher network and the student network are randomly initialized; the teacher network generates pseudo-labels based on the weakly enhanced image and obtains a pseudo-label set by filtering according to a threshold, and uses Hungarian matching to pair the pseudo-label set with the output of the student network, while simultaneously supervising the training of the student network based on the source domain image labels; A recurrent self-training unit is used to alternately perform adversarial feature alignment and hybrid pseudo-label training during the self-training phase. Both the teacher network and the student network are initialized by the best-performing model in the pre-training phase. The adversarial feature alignment involves adding a domain discriminator and setting a gradient inversion layer after the student network. The hybrid pseudo-label training involves setting a target domain sample cache. The teacher network generates pseudo-labels based on the weakly enhanced image and combines them with the corresponding strongly enhanced image. It samples cached samples from the target domain sample cache and performs hybrid data augmentation and mosaic data augmentation on each sample before training the student network. Finally, it updates the target domain sample cache with the strongly enhanced image containing pseudo-labels. The loop organization and output unit is used to divide the self-training phase into multiple small cycles. Each small cycle includes several adversarial feature alignments and a single hybrid pseudo-label training. After completing all the small cycles, the student network for target domain object detection is output.

[0058] The beneficial effects of this invention are as follows: (1) This invention can simultaneously improve the quality of target domain pseudo-labels and reduce the negative impact of erroneous pseudo-labels within a teacher-student framework, thereby achieving stable transfer even when there are significant distribution differences between the source and target domains. In the pre-training stage, this invention utilizes complementary masks to enable the student network to learn discriminative knowledge of the target domain, allowing the teacher network to output higher-quality target domain pseudo-labels. In the self-training stage, it improves the number of correct pseudo-labels and suppresses the accumulation of erroneous pseudo-labels through a cyclical organization method of adversarial feature alignment and mixed pseudo-label training, enabling the model to achieve better generalization performance and more stable detection output in the target domain scenario. Compared with methods that only use source domain pseudo-labels or rely on threshold filtering, this invention can effectively alleviate the performance degradation caused by unreliable pre-trained models and erroneous pseudo-labels.

[0059] (2) This invention is applicable to typical domain shift scenarios in intelligent driving, such as weather changes, changes in city streets, and migration of synthetic images to real street scenes. It achieves superior performance results in multiple unsupervised domain adaptive object detection tasks, such as Cityscapes→Foggy Cityscapes, Cityscapes→BDD100K-Daytime, and Sim10k→Cityscapes (car). This invention can improve the accuracy and robustness of domain adaptive object detection, meet the high-reliability target perception requirements of autonomous driving systems in complex dynamic environments, provide stable detection support for functions such as forward collision warning, lane keeping, and traffic light recognition, and help reduce the safety risks caused by target recognition failure. This invention can improve the generalization ability and training stability of cross-domain detection, making it easier to integrate into the actual deployment process of intelligent driving systems.

[0060] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0061] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A domain-adaptive target detection method based on complementary masks and cyclic training, characterized in that, include: Acquire labeled source domain images and unlabeled target domain images, and generate weakly enhanced images and strongly enhanced images based on the target domain images; Build teacher and student networks; During the pre-training phase, a pair of complementary and non-overlapping binary masks are generated based on the target domain image and applied to the teacher network and the student network respectively; both the teacher network and the student network are randomly initialized; the teacher network generates pseudo-labels based on the weakly enhanced image and obtains a pseudo-label set by filtering according to a threshold, and uses Hungarian matching to pair the pseudo-label set with the output of the student network, while the student network is trained under supervision based on the source domain image labels; During the self-training phase, adversarial feature alignment and hybrid pseudo-label training are performed alternately. Both the teacher network and the student network are initialized with the best-performing model from the pre-training phase. The adversarial feature alignment involves adding a domain discriminator and setting a gradient inversion layer after the student network. The hybrid pseudo-label training involves setting a target domain sample cache. The teacher network generates pseudo-labels based on the weakly enhanced image and combines them with the corresponding strongly enhanced image. Cache samples are obtained from the target domain sample cache, and hybrid data augmentation and mosaic data augmentation are performed respectively before training the student network. The target domain sample cache is then updated with the strongly enhanced image with pseudo-labels. The self-training phase is divided into multiple small cycles, each of which includes several adversarial feature alignments and a single hybrid pseudo-label training. After completing all the small cycles, the student network for target domain object detection is output.

2. The domain adaptive target detection method based on complementary masks and cyclic training according to claim 1, characterized in that, Generating weakly enhanced and strongly enhanced images based on the target domain image includes: Perform a first enhancement operation on the target domain image to generate the weakly enhanced image; A second enhancement operation is performed on the target domain image to generate the strongly enhanced image; the second enhancement operation includes more enhancement types than the first enhancement operation.

3. The domain adaptive target detection method based on complementary masks and cyclic training according to claim 1, characterized in that, During the pre-training phase, a pair of complementary and non-overlapping binary masks are generated based on the target domain image and applied to the teacher network and the student network respectively, including: Randomly select several mask positions in the spatial location of the target domain image to generate a first binary mask; A second binary mask is generated based on the first binary mask, such that the first binary mask and the second binary mask have opposite values ​​at the same spatial location, thereby forming a complementary and non-overlapping binary mask pair; The first binary mask and the second binary mask are respectively input into the teacher network and the student network as mask constraints for the corresponding networks.

4. The domain adaptive target detection method based on complementary masks and cyclic training according to claim 1, characterized in that, During the pre-training phase, a pair of complementary and non-overlapping binary masks are generated based on the target domain image and applied to the teacher network and the student network respectively, including: Feature maps of the target domain image are extracted from the backbone networks of the teacher network and the student network, respectively. Map the binary mask to a size consistent with the feature map; The feature values ​​at the masked locations are set to zero based on the binary mask, so that the zeroed features do not participate in subsequent feature fusion and prediction calculations.

5. The domain adaptive target detection method based on complementary masks and cyclic training according to claim 1, characterized in that, The teacher network generates pseudo-labels based on the weakly enhanced image and filters them according to a threshold to obtain a pseudo-label set, including: The teacher network outputs candidate detection results and corresponding feature embeddings for the weakly enhanced image; the candidate detection results include category prediction results and bounding box prediction results. A confidence threshold is set based on the candidate detection results, and the pseudo-label set is obtained by filtering according to the confidence threshold; Each pseudo-label in the pseudo-label set is associated with its corresponding category information, bounding box information, and corresponding feature embedding to form a pseudo-label representation for training.

6. The domain adaptive target detection method based on complementary masks and cyclic training according to claim 1, characterized in that, Pairing the pseudo-label set with the student network output using Hungarian matching includes: Construct a matching cost matrix; the matching cost matrix includes a classification cost term, a bounding box regression cost term, and an intersection-union ratio (IU / UCR) cost term. Hungarian matching is performed based on the matching cost matrix to obtain pairing relationships, and the paired student network output is determined based on the pairing relationships; The unsupervised training loss is calculated based on the paired student network output and the pseudo-label set, and is used together with the supervised training loss calculated based on the source domain image labels to train the student network; the teacher network updates its parameters using an exponential moving average mechanism.

7. The domain adaptive target detection method based on complementary masks and cyclic training according to claim 1, characterized in that, The adversarial feature alignment involves adding a domain discriminator after the student network and setting a gradient inversion layer, including: Set domain labels for the source domain image and the target domain image respectively; The features output by the student network are input into the domain discriminator to output the domain discrimination result, and the domain discrimination loss is calculated based on the domain discrimination result and the domain label; The gradient reversal layer is set between the student network and the domain discriminator to reverse the gradient direction and update the parameters of the student network during backpropagation, thereby enabling the student network to learn domain-invariant features; the teacher network updates its parameters through an exponential moving average mechanism.

8. The domain adaptive target detection method based on complementary masks and cyclic training according to claim 1, characterized in that, The hybrid pseudo-label training involves setting up a target domain sample cache. The teacher network generates pseudo-labels based on the weakly enhanced images and combines them with the corresponding strongly enhanced images. Cache samples are obtained from the target domain sample cache, and hybrid data augmentation and mosaic data augmentation are performed respectively before training the student network. The target domain sample cache is then updated with strongly enhanced images bearing pseudo-labels. This includes: The target domain samples and their corresponding pseudo-labels are stored in the target domain sample cache, and the cached samples are output from the target domain sample cache based on a preset sampling rule; Perform hybrid data augmentation on the cached samples and the strongly enhanced images. The hybrid data augmentation includes weighting the two images according to the hybrid coefficient and merging the pseudo-labels corresponding to the two images to generate hybrid training samples. Mosaic data augmentation is performed on the cached samples. The mosaic data augmentation includes sampling four cached samples and stitching the four cached samples into a synthetic image, while merging the pseudo-labels corresponding to the four cached samples to generate synthetic training samples. The mixed training samples and the synthetic training samples are input into the student network to complete the training, and the target domain sample cache is updated based on the latest strongly enhanced image with pseudo-labels; the teacher network updates its parameters through an exponential moving average mechanism.

9. The domain adaptive target detection method based on complementary masks and cyclic training according to claim 1, characterized in that, The self-training phase is divided into multiple short cycles, each containing several adversarial feature alignments and a single hybrid pseudo-label training iteration. After completing all the short cycles, the student network for target domain object detection is output, including: Within each of the aforementioned small cycles, the adversarial feature alignment is first performed a preset number of times to update the student network; Then perform a single training iteration of the hybrid pseudo-label to update the student network and update the target domain sample cache; After completing the current small cycle, the next small cycle is entered, and in the next small cycle, the processes of "continuously performing the adversarial feature alignment a preset number of times to update the student network" and "performing a single hybrid pseudo-label training to update the student network and update the target domain sample cache" are repeated until all the small cycles are completed, and then the student network for target domain object detection is output.

10. A domain-adaptive target detection system based on complementary masks and cyclic training, characterized in that, include: The source domain and target domain data acquisition and enhancement unit is used to acquire labeled source domain images and unlabeled target domain images, and generate weakly enhanced images and strongly enhanced images based on the target domain images. The teacher network and student network construction unit is used to build teacher networks and student networks; A complementary mask pre-training unit is used to generate a pair of complementary and non-overlapping binary masks based on the target domain image during the pre-training phase and apply them to the teacher network and the student network respectively; both the teacher network and the student network are randomly initialized; the teacher network generates pseudo-labels based on the weakly enhanced image and obtains a pseudo-label set by filtering according to a threshold, and uses Hungarian matching to pair the pseudo-label set with the output of the student network, while simultaneously supervising the training of the student network based on the source domain image labels; A recurrent self-training unit is used to alternately perform adversarial feature alignment and hybrid pseudo-label training during the self-training phase. Both the teacher network and the student network are initialized by the best-performing model in the pre-training phase. The adversarial feature alignment involves adding a domain discriminator and setting a gradient inversion layer after the student network. The hybrid pseudo-label training involves setting a target domain sample cache. The teacher network generates pseudo-labels based on the weakly enhanced image and combines them with the corresponding strongly enhanced image. It samples cached samples from the target domain sample cache and performs hybrid data augmentation and mosaic data augmentation on each sample before training the student network. Finally, it updates the target domain sample cache with the strongly enhanced image containing pseudo-labels. The loop organization and output unit is used to divide the self-training phase into multiple small cycles. Each small cycle includes several adversarial feature alignments and a single hybrid pseudo-label training. After completing all the small cycles, the student network for target domain object detection is output.

Images