Unsupervised domain adaptive target detection method and device, equipment and storage medium

Abstract

The invention relates to an unsupervised domain adaptive target detection method and device, equipment and a storage medium. The method comprises the following steps: proposing a beneficial information maximization and decoupling framework, and realizing effective domain alignment by extracting sufficient beneficial information and minimizing redundant information. The framework comprises three units: a potential beneficial information mining unit for estimating statistical characteristic offset containing semantic change to modulate characteristic distribution and maintaining / enhancing potential beneficial information under distribution offset; the beneficial information decoupling unit introduces an information bottleneck principle, maximizes latent variables and task output mutual information, and minimizes latent variables and input mutual information to filter redundancy; and the beneficial information keeping and domain aligning unit is used for generating a target domain pseudo-supervision signal by means of a teacher detector so as to realize target domain beneficial information keeping and efficient alignment of the source domain and the target domain. The method does not have extra learnable parameters, avoids the increase of calculation overhead, and achieves precise target detection at the same time.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an unsupervised adaptive target detection method, apparatus, device, and storage medium. Background Technology

[0002] With breakthroughs in deep learning technology, object detection has achieved remarkable success. However, when the distribution of test data differs from that of training data (i.e., domain shift), detector performance suffers a severe decline, hindering its deployment in practical applications such as autonomous driving, video surveillance, and medical image analysis. Furthermore, the expensive and time-consuming instance-level annotation work limits the possibility of achieving satisfactory model generalization capabilities in new domains through supervised learning.

[0003] To address these challenges, unsupervised domain adaptive object detection has been proposed and has attracted widespread attention in recent years. This method adapts a detector trained in the labeled source domain to the unlabeled target domain. To achieve unsupervised domain adaptive object detection, current research typically employs unsupervised domain adaptation (UDA) to align cross-domain feature distributions. Representative methods include adversarial training, graph-based modeling, clustering, contrastive learning, and mean-teacher methods.

[0004] While these methods have achieved satisfactory performance, they primarily focus on aligning raw features extracted by pre-trained models in the source domain, neglecting the problems of insufficient useful information and feature redundancy. In object detection, useful information such as spatial distribution, semantic association, object structure, and texture is crucial for learning discriminative feature representations and can facilitate knowledge transfer. However, detectors trained using source datasets exhibit severe bias in the target domain, leading to the loss of useful information. This can cause the model to suffer from discriminative dimension degradation, forcing it to perform domain alignment in an overcompressed manifold space, ultimately resulting in suboptimal alignment. For example, the lack of contour details and the blurring of textures mean that the aligned model can only rely on coarse-grained cues such as shape and size, inevitably leading to poor generalization performance of the detector. Furthermore, when disturbed by redundant information (such as background and illumination variations), the model is prone to overfitting these noisy or non-discriminative features, which can also severely impair its cross-domain generalization ability and may even lead to negative transfer. Summary of the Invention

[0005] Therefore, it is necessary to provide an unsupervised domain adaptive target detection method, apparatus, device, and storage medium that can improve the target domain detection accuracy in response to the above-mentioned technical problems.

[0006] An unsupervised adaptive target detection method, the method comprising: Obtain a source domain dataset and a target domain dataset. The source domain dataset includes multiple source domain image samples with ground truth labels, and the target domain dataset includes multiple unlabeled target domain image samples. A target detection training network is constructed based on the pre-trained student detector and teacher detector. Both the student detector and the teacher detector include a backbone network and a detection head. In the student detector, the end of the backbone network integrates a useful information mining unit, and the detection head integrates a useful information decoupling unit. The source domain image samples and target domain image samples are input as a set of samples into the target detection training network. The teacher detector generates pseudo-labels for the target domain image samples. The beneficial information mining unit modulates the feature distribution by estimating the potential shift of the feature statistics, thereby obtaining the target domain and source domain optimized features containing beneficial information. Then, the beneficial information decoupling unit performs information bottleneck constraints on the first layer features of the detection head to obtain the beneficial information decoupling loss of the target domain and source domain. At the same time, based on the target domain and source domain prediction results output by the detection head, the classification loss and bounding box loss are calculated according to the corresponding pseudo-labels and ground truth labels, respectively. The total loss function is calculated based on the decoupling loss, classification loss, and bounding box loss of the source domain image samples and target domain image samples. The adjustable parameters of the student detector and teacher detector are optimized using the total loss function until convergence, and the trained student detector and teacher detector are obtained. The target domain image is acquired and input into the trained teacher detector to obtain the target detection result.

[0007] In one embodiment, in the beneficial information mining unit: Channel-by-channel calculation: The backbone network extracts feature statistics based on a batch of sample data, including the mean and standard deviation. The potential bias is estimated by utilizing the variance of characteristic statistics among multiple batches of sample data. Based on the multivariate Gaussian distribution corresponding to the potential offset, a new feature statistic is generated to modulate the features extracted by the backbone network, thereby obtaining optimized features.

[0008] In one embodiment, the optimization feature is represented as:

[0009] In the above formula, This represents the features extracted from the backbone network. and Representation of features The characteristic statistics, and This represents the statistical measure of the new feature.

[0010] In one embodiment, during pre-training of the student detector and the teacher detector: The student detector is pre-trained using the source domain dataset to obtain the pre-trained student detector. The parameters of the pre-trained student detector are copied to the teacher detector as initial parameters.

[0011] In one embodiment, the adjustable parameters of the student detector and the teacher detector are optimized using the total loss function: The adjustable parameters in the student detector are optimized using the total loss function, and the adjustable parameters in the teacher detector are optimized using an exponential moving average.

[0012] In one embodiment, during training, the prediction output of the detection head is a semantic output that has not undergone nonmaximum suppression processing.

[0013] In one embodiment, the classification loss is a quality focus loss.

[0014] This application also provides an unsupervised domain adaptive target detection device, the device comprising: The sample acquisition module is used to acquire source domain datasets and target domain datasets. The source domain dataset includes multiple source domain image samples with ground truth labels, and the target domain dataset includes multiple unlabeled target domain image samples. The target detection training network construction module is used to construct a target detection training network based on the pre-trained student detector and teacher detector. Both the student detector and the teacher detector include a backbone network and a detection head. In the student detector, the end of the backbone network integrates a useful information mining unit, and the detection head integrates a useful information decoupling unit. The loss calculation module is used to input the source domain image samples and target domain image samples as a set of samples into the target detection training network. The teacher detector generates pseudo labels for the target domain image samples. The beneficial information mining unit modulates the feature distribution by estimating the potential shift of the feature statistics, thereby obtaining the target domain and source domain optimized features containing beneficial information. Then, the beneficial information decoupling unit performs information bottleneck constraints on the first layer features of the detection head to obtain the beneficial information decoupling loss of the target domain and source domain. At the same time, based on the target domain and source domain prediction results output by the detection head, the classification loss and bounding box loss are calculated according to the corresponding pseudo labels and ground truth labels, respectively. The network training module is used to calculate the total loss function based on the beneficial information of the source domain image samples and the target domain image samples, including decoupling loss, classification loss, and bounding box loss. The total loss function is then used to optimize the adjustable parameters of the student detector and the teacher detector until convergence, resulting in a trained student detector and a teacher detector. The target detection module is used to acquire a target domain image, input the target domain image into a trained teacher detector, and obtain the target detection result.

[0015] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the specific steps of the above-described unsupervised adaptive target detection method.

[0016] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the specific steps of the above-described unsupervised adaptive target detection method.

[0017] The aforementioned unsupervised adaptive target detection method, apparatus, device, and storage medium construct a target detection training network based on pre-trained student and teacher detectors. Both the student and teacher detectors include a backbone network and a detection head. In the student detector, a useful information mining unit is integrated at the end of the backbone network, and a useful information decoupling unit is integrated in the detection head. Source domain image samples and target domain image samples are input as a set of samples into the target detection training network. The teacher detector generates pseudo-labels for the target domain image samples, and the useful information mining unit modulates the feature distribution by estimating the latent shift of feature statistics, thereby obtaining the target domain and target domain containing implicit useful information. The source domain features are optimized, and then an information bottleneck constraint is applied to the first-layer features of the detection head through a beneficial information decoupling unit to obtain the beneficial information decoupling loss between the target and source domains. Simultaneously, based on the target and source domain prediction results output by the detection head, classification loss and bounding box loss are calculated according to the corresponding pseudo-labels and ground truth labels, respectively. A total loss function is calculated based on the beneficial information decoupling loss, classification loss, and bounding box loss of the source and target domain image samples. This total loss function is used to optimize the adjustable parameters of the student and teacher detectors until convergence, resulting in trained student and teacher detectors. The target domain image is then input into the trained teacher detector to obtain the target detection result. This method can effectively improve the accuracy of unsupervised target detection. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an unsupervised adaptive target detection method in one embodiment; Figure 2This is a schematic diagram illustrating how the method in this embodiment achieves domain alignment by extracting sufficient useful features and filtering out redundant information. Figure 3 This is a schematic diagram of the beneficial information maximization and decoupling framework proposed in this method in one embodiment; Figure 4 This is a schematic diagram of the qualitative analysis results in an experiment. Figure 5 This is a visual comparison of image-level features in an experiment comparing the adaptation tasks from Cityscapes to Foggy Cityscapes domains. Figure 6 A visualization of instance-level feature t-SNE for an adaptive task from Cityscapes to Foggy Cityscapes in an experiment; Figure 7 This is a structural block diagram of an unsupervised domain adaptive target detection device in one embodiment; Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] In existing unsupervised adaptive object detection, the primary approach to mitigate domain disparities is by aligning the original features extracted from pre-trained models on the source domain dataset. However, this neglects the alignment bias and even negative transfer issues that may result from insufficient useful information and feature redundancy. In this application, as... Figure 1 As shown, an unsupervised adaptive target detection method is provided, including the following steps: Step S100: Obtain the source domain dataset and the target domain dataset. The source domain dataset includes multiple source domain image samples with ground truth labels, and the target domain dataset includes multiple unlabeled target domain image samples.

[0021] Step S110: Construct a target detection training network based on the pre-trained student detector and teacher detector. Both the student detector and the teacher detector contain a backbone network and a detection head. In the student detector, the useful information mining unit is integrated at the end of the backbone network, and the useful information decoupling unit is integrated in the detection head.

[0022] In step S120, source domain image samples and target domain image samples are input as a set of samples into the target detection training network. The teacher detector generates pseudo-labels for the target domain image samples. The useful information mining unit modulates the feature distribution by estimating the potential shift of the feature statistics, thereby obtaining the target domain and source domain optimized features containing useful information. Then, the useful information decoupling unit performs information bottleneck constraints on the first layer features of the detection head to obtain the useful information decoupling loss of the target domain and source domain. At the same time, based on the target domain and source domain prediction results output by the detection head, the classification loss and bounding box loss are calculated according to the corresponding pseudo-labels and ground truth labels, respectively.

[0023] Step S130: Calculate the total loss function based on the decoupling loss, classification loss, and bounding box loss of the useful information of the source domain image samples and the target domain image samples. Optimize the adjustable parameters of the student detector and the teacher detector using the total loss function until convergence, and obtain the trained student detector and teacher detector.

[0024] Step S140: Obtain the target domain image, input the target domain image into the trained teacher detector, and obtain the target detection result.

[0025] This application proposes a novel Beneficial Information Maximization and Disentanglement (BIMD) framework, which achieves effective domain alignment in object detection by extracting sufficient beneficial information and minimizing redundant information. Specifically, the framework includes a Latent Beneficial Information Mining (LBIM) unit, which modulates the feature distribution by estimating statistical property shifts that include semantic changes, thereby preserving or enhancing latent beneficial information under distribution shift conditions. Simultaneously, a Beneficial Information Disentanglement (BID) unit is proposed, which introduces the information bottleneck principle to maximize the mutual information between latent variables and task outputs while minimizing their mutual information with inputs to achieve redundancy filtering. A Beneficial Information Preservation and Domain Alignment (BIRDA) module is also included, which generates pseudo-supervisory signals in the target domain using a teacher detector, achieving beneficial information preservation in the target domain based on the information bottleneck and efficient source and target domain alignment. Notably, this method does not introduce additional learnable parameters, avoiding increased computational overhead. Extensive experiments across multiple widely used scenarios demonstrate that this method improves baseline performance and achieves state-of-the-art performance.

[0026] Specifically, the beneficial information maximization and decoupling framework proposed in this application is used for unsupervised adaptive object detection. Its core idea is to achieve domain alignment by extracting sufficient beneficial features and filtering out redundant information, such as... Figure 2 As shown. Specifically, inspired by the idea that the variance between features can encode implicit semantic information, feature statistics are calculated and their variances are used as latent offset estimates for feature distribution modulation. This method can preserve or enhance potentially beneficial information under domain offset conditions. This method is called Latent Beneficial Information Mining (LBIM).

[0027] Furthermore, a Beneficial Information Disentanglement (BID) is proposed. An information bottleneck criterion is introduced, treating the first-layer convolutional features of the detection head as latent variables. By maximizing the mutual information between the latent variables and the task output, while minimizing the mutual information between the latent variables and the input, redundant information is filtered out while retaining the features most beneficial to downstream tasks.

[0028] Finally, a Beneficial Information Retention and Domain Alignment (BIRDA) method is proposed. Considering that the information bottleneck principle relies on supervisory signals for constraint, a Mean Teacher (MT) is used to generate pseudo-supervisory signals in the target domain, achieving beneficial information retention based on the information bottleneck in the target domain and efficient alignment between the source and target domains. Notably, this method does not introduce additional learnable parameters, thus avoiding increased computational overhead.

[0029] Therefore, this method proposes a training framework and a network architecture, such as Figure 3 As shown. During the training process, the training framework includes a student detector and a teacher detector. Both detectors have the same network structure, consisting of a backbone network and a detection head. The difference is that the backbone network of the student detector integrates a useful information mining unit, while the first layer of the detection head integrates a useful information decoupling unit after the feature convolution, which is the target detection training network constructed in step S110.

[0030] In step S120, when training the object detection training network, source domain image samples and target domain image samples are input as a set of samples into the object detection training network. The source domain image samples and target domain image samples are respectively from the source domain dataset and the target domain dataset. Formally, given a set of samples... Source domain datasets containing samples with truth labels ,in, Indicates the first Source domain sample images Bounding box annotations for instances in the example. This indicates the corresponding category label and contains Target domain dataset of unlabeled samples ,in, Indicates the first The ultimate goal of unsupervised adaptive object detection is to jointly utilize sample images of the target domain. and To construct a detector that learns domain-invariant features.

[0031] like Figure 3 As shown, during unsupervised training, the target domain sample images are weakly enhanced and then input into the teacher detector to obtain pseudo-labels for the unlabeled target domain sample images. Simultaneously, the target domain sample images and source domain sample images are strongly enhanced and then input as a sample group into the student detector. Next, we will introduce the beneficial information mining unit and the beneficial information decoupling unit set up in the student network.

[0032] Enhancing the generalization ability of detectors by fully acquiring useful information from the training dataset is crucial for domain alignment. However, detectors trained on source domain datasets often exhibit significant bias when applied to the target domain, leading to the loss of useful information and feature degradation, thus affecting domain alignment performance. To address this issue, our method proposes a useful information mining unit, which effectively mines useful information by capturing and adjusting potential changes in feature statistics.

[0033] Inspired by the idea that differences between features can encode implicit semantic information, the beneficial information mining unit first calculates feature statistics (mean and standard deviation) as an abstract representation of the features. Then, it uses the variance of the mini-batch data statistics as an estimate of the potential bias and maps it to each feature channel. In this way, the beneficial information mining unit can retain or enhance the potential beneficial information required for domain adaptation learning.

[0034] In this embodiment, in the beneficial information mining unit, feature statistics, including the mean and standard deviation, are calculated channel by channel by the backbone network based on a batch of sample data. The variance of the feature statistics across multiple batches of sample data is used to estimate the potential offset. Then, new feature statistics are generated based on the multivariate Gaussian distribution corresponding to the potential offset to modulate the features extracted by the backbone network, resulting in optimized features.

[0035] Specifically, in the beneficial information mining unit, feature statistics that can capture information features on the corresponding data domain are calculated. Given... Input batch of samples By feature extractor (i.e., the feature extraction part in the backbone network) generates features Then, characteristic statistics, including the mean, are calculated channel by channel. and standard deviation The calculation method is as follows: (1) (2) In formulas (1) and (2), and These represent the height and width of the input feature, respectively.

[0036] Furthermore, we assume that each feature statistic follows a multivariate Gaussian distribution rather than taking deterministic values, so that the features can capture potentially useful information. The mean of this distribution can be considered as the original feature statistic, while the standard deviation characterizes the possible range of bias. Early research has shown that the variance between features can encode implicit semantic information, and directions with larger variances may point to more valuable semantic changes. Based on this finding, we use the variance of feature statistics between batch samples to estimate the potential bias: (3) (4) In formulas (3) and (4), This represents the mean of the feature statistics within the batch. It is worth noting that the calculation rules provided by formulas (3) and (4) play a key role in characterizing useful information about features, and they can provide appropriate and meaningful transformation representations for each feature channel.

[0037] Furthermore, after obtaining the latent bias estimate, a corresponding multivariate Gaussian distribution can be constructed to generate new statistics to represent the characteristic distribution. Specifically, the mean... Follows Gaussian distribution ,variance Follows Gaussian distribution Subsequently, information enhancement is achieved by sampling from a Gaussian distribution and randomly selecting the direction and intensity of semantic transformation. (5) (6) In formulas (5) and (6), and Follows a normal distribution Finally, affine transformation is used to transform the features Statistic and A new style of conversion to random sampling and : (7) In this embodiment, the aforementioned beneficial information mining unit is applied to the end of the backbone network to modulate high-level semantic features. This unit is selectively incorporated into the training process with a preset probability threshold to enhance the model's ability to learn stable features and its robustness. Simultaneously, noting that a similar distribution modulation method is used in supervised learning, this method employs a unified strategy for labeled source images and unlabeled target images. It is worth noting that this unit does not contain any learnable parameters, thereby reducing the model's computational complexity to some extent. Furthermore, it only operates during the training phase and can be completely removed during the testing phase to improve inference efficiency. By introducing this unit, the variability of feature distribution can be captured, helping the model discover and enhance beneficial information.

[0038] While fully acquiring useful information, effective domain alignment must focus on minimizing redundant signals. Such redundancy may cause the model to be biased towards noisy or irrelevant features, leading to overfitting or even negative transfer. To address this, our method introduces the Information Bottleneck (IB) principle to decouple useful information and filter out useless information, providing a sufficient and compact feature representation for the prediction task.

[0039] In this embodiment, based on the formal formulation of the information bottleneck theory, let... , and Let the input data, latent variables, and prediction target be represented respectively. Then, the objective function to be optimized is defined as follows: (8) In formula (8), Indicates mutual information, The weight hyperparameters are used as the features extracted by the first convolutional layer of the detection head. In this equation, the first term encourages the latent variables to learn sufficient information, while the second term ensures the filtering of redundant information. Further expansion of the mutual information yields: (9) In formula (9), since and This is difficult to solve here, now let and These are respectively used as their variational approximations. For the model's decoder, It can be regarded as a standard Gaussian distribution Given the nonnegativity of the Kullback-Leibler divergence, we obtain the inequality: and Therefore, we obtain The lower bound is as follows: (10) Furthermore, according to , and Markov chain hypothesis: given Under the conditions and Since they are independent, we obtain the joint probability distribution: Therefore, formula (10) can be written in the following form: (11) In formula (11), This indicates the number of samples in the dataset.

[0040] Therefore, the loss function that needs to be maximized under the information bottleneck constraint is obtained, as shown in formula (11). Here, the first term can be considered as the classification cross-entropy in the detection loss (omitting the one-hot encoding of the label), while... Let represent the entropy of the entire dataset labels, which is independent of the optimization process in the supervised learning paradigm. Since the first two terms of the loss function do not need to be considered, the objective is to minimize: Given that a single image generates multiple instance features in object detection, the useful information decoupling loss is ultimately expressed as: (12) In formula (12), Indicates the number of samples. and These represent the height and width of the feature map, respectively, and the loss is calculated using a sampling strategy. It's worth noting that this method does not introduce any additional parameters, thus not increasing the computational cost.

[0041] As shown in Equation (8), the information bottleneck constraint requires the target output to guide the retention of useful information. Although the source domain data is labeled, the target domain remains unlabeled. In this embodiment, the Mean Teacher (MT) method is used to generate pseudo-labels for the target domain to impose the information bottleneck constraint, and context-aware feature alignment is combined to achieve domain adaptation.

[0042] In this embodiment, the mean-teacher method typically comprises two structurally identical models: a student detector and a teacher detector, i.e., the object detection training network proposed above. The teacher model processes weakly enhanced target domain images to generate pseudo-labels, which serve as supervision signals for the corresponding strongly enhanced images. The student model is trained based on the pseudo-labeled target domain data. Through supervised learning of the target domain, this method achieves domain alignment while preserving useful information. Furthermore, the student model is also trained using the source domain dataset to retain useful information from the source domain and mitigate confirmation bias. Therefore, the overall training objective includes source domain loss. and target domain loss : (13) (14) In formulas (13) and (14), and These represent the classification loss and the bounding box regression loss, respectively. For information decoupling loss, Represents the truth label (GT). This represents a pseudo-label. The overall loss function is defined as follows: (15) In formula (15), Balance the loss items.

[0043] It is worth noting that in this embodiment, pseudo-supervised learning is performed using the semantic output without non-maximum suppression (NMS) processing, in order to preserve useful information to the greatest extent possible, i.e., the prediction result of the detection head output is the semantic output without NMS processing. To this end, the classification loss is... Replace it with Quality Focal Loss, which is defined as follows: (16) In formula (16), Indicates inhibitory factor, and These represent the category scores predicted by the teacher detector and the student detector, respectively.

[0044] In this embodiment, in each iteration, after the student detector has been optimized, its parameters are updated using the exponential moving average (EMA) to obtain a more accurate model. The EMA update process can be described as follows: ,in The parameters represent the detector model. This refers to the update rate.

[0045] In this embodiment, during the pre-training of the student detector and the teacher detector: the student detector is pre-trained using the source domain dataset to obtain the pre-trained student detector. The parameters of the pre-trained student detector are then copied to the teacher detector as initial parameters.

[0046] Specifically, when pre-training the student detector using the source domain dataset, the loss function shown in formula (13) can be used to train the student detector.

[0047] In this embodiment, the backbone networks in the teacher detector and student detector can be selected from multiple options and are not limited to one. In the subsequent experimental section, the performance of this method in unsupervised adaptive target detection is evaluated by using two commonly used backbone networks, ResNet50 and VGG16, to demonstrate that the framework proposed in this method has good compatibility.

[0048] In step S140, the teacher detector in the trained object detection training network is directly used as the final cross-domain object detector to perform object detection on the object domain image.

[0049] In this embodiment, pseudocode for the complete optimization process is also provided, as shown in Algorithm 1. During the inference phase, a standard object detector structure is used, which does not introduce any additional network parameters or computational overhead.

[0050] Algorithm 1

[0051] In this paper, the effectiveness of the proposed method is also demonstrated through experiments. The method was extensively tested on five public datasets, including Cityscapes, Foggy Cityscapes, BDD100K, KITTI, and Sim10K. Cityscapes and Foggy Cityscapes each contain 2,975 training images and 500 validation images, labeled with 8 target classes. Foggy Cityscapes is a foggy scene dataset synthesized based on Cityscapes. BDD100K has a similar scene to Cityscapes but a different camera configuration, containing 7 overlapping classes. 36,278 training images and 5,258 validation images taken during the day were used as the target domain data. KITTI provides 7,481 labeled images of autonomous vehicles; although it shares road scenes with Cityscapes, the camera parameters differ. Sim10K contains 10,000 synthetic images rendered using the Grand Theft Auto engine, with only vehicle annotations.

[0052] Specifically, the experiments follow previous work, implementing a RetinaNet-based detector and using ImageNet pre-trained VGG16 as the backbone network. The student detector is trained using the SGD optimizer with a learning rate of 0.01, momentum of 0.9, and weight decay of 0.0001. The target domain loss weights are... and inhibitory factors All values ​​were set to 2. The teacher detector was updated using EMA with a smoothing coefficient of 0.9995. The student detector was first trained for 10k iterations on the source domain to initialize the teacher model, followed by 40k iterations of training using combined source and target domain data. Data preprocessing and augmentation followed the MGCAMT approach. For fair comparison, all experiments were performed on two RTX 3090 GPUs, with each GPU processing eight images (four labeled and four unlabeled).

[0053] In the experiments, our method is first compared with state-of-the-art methods. This experiment evaluates the model's adaptability to dynamic natural environments in domain adaptation under different weather conditions. Experiments were conducted using the Cityscapes and Foggy Cityscapes training sets, and results are reported on the Foggy Cityscapes validation set. As shown in Table 1, our method achieves improvements of 5.0%, 1.6%, and 6.3% compared to state-of-the-art methods based on two-stage, one-stage, and ensemble prediction detectors, respectively. Even compared to DA-Pro, which uses a large base model, our method maintains a 1.6% advantage. Furthermore, experimental results show that our method achieves the highest average accuracy (AP) across four target classes: pedestrians, cars, motorcycles, and bicycles.

[0054] Table 1 shows the experimental results of migrating from Cityscapes to Foggy Cityscapes. In this paper, VGG16 is used as the backbone network, and the average accuracy (%) is evaluated on the target domain image.

[0055] Furthermore, domain adaptation experiments were conducted across different cameras. This experiment investigated the domain adaptation problem across cameras, where differences in image distribution mainly stemmed from variations in camera intrinsic parameters (such as distortion and resolution) and extrinsic parameters (such as mounting height and orientation). Using Cityscapes as the source domain and BDD100K daytime images as the target domain, the adaptation results are recorded in Table 2. The experiments show that this method outperforms the two-stage method with a 4.7% mAP, surpasses the single-stage detector with a 1.7% mAP, and significantly exceeds the DETR detector with a 9.2% mAP, achieving the best performance among all recent studies.

[0056] To fully validate the performance of this method, a domain adaptation experiment was conducted from KITTI to Cityscapes. This experiment used KITTI as the source domain and Cityscapes as the target domain, and evaluated the detection performance on vehicle categories in the Cityscapes validation set. The results in Table 3 show that this method outperforms all state-of-the-art methods (including those based on the base model) with a mAP of 62.8%, achieving the best performance in this benchmark.

[0057] Table 2 shows the experimental results of migrating from Cityscapes to BDD100K, where the average accuracy (%) was evaluated on the target domain image using VGG16 as the backbone network.

[0058] Table 3 shows the experimental results of migrating from KITTI to Cityscapes, where the average accuracy (%) was evaluated on the target domain image using VGG16 as the backbone network.

[0059] Next, a domain adaptation experiment was conducted, moving from synthetic images to real images. This experiment investigated the cross-domain vehicle instance detection task, from computer-rendered images to real data. During training, Sim10K was used as the labeled source dataset, and Cityscapes as the unlabeled target dataset. The results were tested on the Cityscapes validation set. As shown in Table 4, our method achieved a new record of 68.8% mAP, a 1.3% improvement over the previous best result. This result fully demonstrates that our method can effectively extract sufficient useful features with minimal redundancy and achieve excellent feature alignment.

[0060] Table 4 shows the experimental results of migrating from Sim10K to Cityscapes, where the average accuracy (%) is evaluated on the target domain image using VGG16 as the backbone network.

[0061] Furthermore, ablation experiments were conducted on the proposed framework, as shown in Table 5. These experiments comprehensively validated the contributions of each component of the proposed BIMD framework to the Cityscapes to Foggy Cityscapes domain adaptation task. Based on training using only the source domain, integrating the BID and LBIM units sequentially improved detection performance to 40.1% mAP and 42.0% mAP, respectively. This demonstrates that extracting beneficial features while filtering domain-specific noise effectively promotes the learning of generalizable representations. When BID and LBIM work together in both the source and target domains, detection performance is further improved. Finally, after integrating all three components, the best performance of 57.5% mAP demonstrates the complementary advantages of each unit in this method.

[0062] Table 5 shows the ablation experimental results of transferring Cityscapes to Foggy Cityscapes, where VGG16 was used as the backbone network, and the average accuracy (%) was evaluated on the target domain images.

[0063] The impact of the BID unit's application location was further investigated. The RetinaNet detection head contains four hidden layers and one output layer. Let... Represents the hidden layer index, BID will be the first... Features generated by each hidden layer are used as latent variables. Domain adaptation experiments based on Cityscapes to Foggy Cityscapes (results shown in Table 6) show that: The highest detection accuracy is achieved when the value is 0, and the accuracy increases with... Increase and decrease; when When the value is 3, the performance is even worse than that without using BID. This indicates that decoupling operations based on information bottlenecks require the network to have sufficient capacity to decode hidden representations. In this method, =0 is set as the default configuration for all scenarios.

[0064] Table 6 shows the effectiveness analysis of BID application locations, where VGG16 is used as the backbone network, and the average accuracy (%) is evaluated on the target domain image.

[0065] Furthermore, a compatibility analysis of different backbone networks was conducted. The performance of this method in unsupervised adaptive object detection was evaluated using two commonly used backbone networks, ResNet50 and VGG16. As shown in Table 7, on the FoggyCityscapes dataset, this method achieved stable improvements under both architectures. A performance gain of 16.8% mAP was achieved based on ResNet50, while a 20.1% improvement in detection accuracy was maintained when using VGG16. This backbone network independence characteristic indicates that this method can effectively learn transferable representations, is not limited by the underlying network architecture, and can adapt to different computational constraints and application scenarios. The observed stable performance improvements under each backbone network further validate the adaptability and robustness of the method.

[0066] Table 7. Applicability of different backbone networks, evaluated by average accuracy (%) on the target domain image.

[0067] Furthermore, the qualitative detection results of this method are presented. Figure 4 shows an example of the detection results from the Foggy Cityscapes dataset. Figure 4 As shown, our method achieves more accurate classification (e.g., buses and trucks at the top). The middle row results demonstrate that our proposed method provides more precise localization (e.g., small vehicles, ambiguous buses, and pedestrians). Furthermore, in the harsh environment shown at the bottom, our method effectively avoids missed detections. These results collectively demonstrate the effectiveness of our method.

[0068] Furthermore, the features of this method are visualized. To explore the properties of the learned features, in Figure 5 The proposed method visualizes image-level heatmaps. Compared to Source Only and MT methods, it focuses on the entire image target while maintaining relatively complete coverage, effectively suppressing background interference and noise. These feature visualization results clearly demonstrate that the proposed method can extract a large amount of useful information while filtering out irrelevant noise, ultimately learning transferable discriminative features.

[0069] Furthermore, t-SNE visualization of this method is performed. Figure 6 In this paper, the t-SNE method is used to visualize the distribution of instance features in the source and target domains. Although the MT method can effectively cluster similar instances and show good feature alignment, feature entanglement still exists for vehicle categories such as trucks, buses, and trains. Compared with the MT and Source Only methods, this method shows better separability between similar categories. This visualization result verifies that this method can effectively enhance feature alignment and model discriminative power.

[0070] Among the aforementioned unsupervised adaptive target detection methods, this paper proposes a novel BIMD framework to address the problem of unsupervised adaptive target detection by enhancing useful information and suppressing redundant information. This method proposes an LBIM unit, which guides feature distribution modulation through statistical characteristic shift estimation to uncover potential useful information. Simultaneously, a BID unit is designed to introduce information bottleneck constraints to filter redundant information and retain the features most beneficial to the output. Finally, a BIRDA component is developed to generate a surrogate supervision signal in the target domain to achieve useful information preservation and domain alignment. Extensive experimental results demonstrate that the proposed method not only significantly improves baseline performance but also surpasses existing state-of-the-art methods.

[0071] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0072] In one embodiment, such as Figure 7 As shown, an unsupervised adaptive target detection method apparatus is provided, comprising: a sample acquisition module 200, a target detection training network construction module 210, a loss calculation module 220, a network training module 230, and a target detection module 240, wherein: The sample acquisition module 200 is used to acquire a source domain dataset and a target domain dataset. The source domain dataset includes multiple source domain image samples with ground truth labels, and the target domain dataset includes multiple unlabeled target domain image samples. The target detection training network construction module 210 is used to construct a target detection training network based on the pre-trained student detector and teacher detector. Both the student detector and the teacher detector include a backbone network and a detection head. In the student detector, the end of the backbone network integrates a beneficial information mining unit, and the detection head integrates a beneficial information decoupling unit. The loss calculation module 220 is used to input the source domain image samples and target domain image samples as a set of samples into the target detection training network. The teacher detector generates pseudo labels for the target domain image samples. The beneficial information mining unit modulates the feature distribution by estimating the potential shift of the feature statistics, thereby obtaining the target domain and source domain optimized features containing beneficial information. Then, the beneficial information decoupling unit performs information bottleneck constraints on the first layer features of the detection head to obtain the beneficial information decoupling loss of the target domain and source domain. At the same time, based on the target domain and source domain prediction results output by the detection head, the classification loss and bounding box loss are calculated according to the corresponding pseudo labels and ground truth labels, respectively. The network training module 230 is used to calculate the total loss function based on the beneficial information of the source domain image samples and the target domain image samples, including decoupling loss, classification loss, and bounding box loss. The total loss function is then used to optimize the adjustable parameters of the student detector and the teacher detector until convergence, thus obtaining the trained student detector and teacher detector. The target detection module 240 is used to acquire a target domain image, input the target domain image into a trained teacher detector, and obtain the target detection result.

[0073] Specific limitations regarding the unsupervised adaptive target detection device can be found in the limitations of the unsupervised adaptive target detection method described above, and will not be repeated here. Each module in the aforementioned unsupervised adaptive target detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0074] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an unsupervised domain adaptive target detection method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0075] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0076] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: Obtain a source domain dataset and a target domain dataset. The source domain dataset includes multiple source domain image samples with ground truth labels, and the target domain dataset includes multiple unlabeled target domain image samples. A target detection training network is constructed based on the pre-trained student detector and teacher detector. Both the student detector and the teacher detector include a backbone network and a detection head. In the student detector, the end of the backbone network integrates a useful information mining unit, and the detection head integrates a useful information decoupling unit. The source domain image samples and target domain image samples are input as a set of samples into the target detection training network. The teacher detector generates pseudo-labels for the target domain image samples. The beneficial information mining unit modulates the feature distribution by estimating the potential shift of the feature statistics, thereby obtaining the target domain and source domain optimized features containing beneficial information. Then, the beneficial information decoupling unit performs information bottleneck constraints on the first layer features of the detection head to obtain the beneficial information decoupling loss of the target domain and source domain. At the same time, based on the target domain and source domain prediction results output by the detection head, the classification loss and bounding box loss are calculated according to the corresponding pseudo-labels and ground truth labels, respectively. The total loss function is calculated based on the decoupling loss, classification loss, and bounding box loss of the source domain image samples and target domain image samples. The adjustable parameters of the student detector and teacher detector are optimized using the total loss function until convergence, and the trained student detector and teacher detector are obtained. The target domain image is acquired and input into the trained teacher detector to obtain the target detection result.

[0077] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain a source domain dataset and a target domain dataset. The source domain dataset includes multiple source domain image samples with ground truth labels, and the target domain dataset includes multiple unlabeled target domain image samples. A target detection training network is constructed based on the pre-trained student detector and teacher detector. Both the student detector and the teacher detector include a backbone network and a detection head. In the student detector, the end of the backbone network integrates a useful information mining unit, and the detection head integrates a useful information decoupling unit. The source domain image samples and target domain image samples are input as a set of samples into the target detection training network. The teacher detector generates pseudo-labels for the target domain image samples. The beneficial information mining unit modulates the feature distribution by estimating the potential shift of the feature statistics, thereby obtaining the target domain and source domain optimized features containing beneficial information. Then, the beneficial information decoupling unit performs information bottleneck constraints on the first layer features of the detection head to obtain the beneficial information decoupling loss of the target domain and source domain. At the same time, based on the target domain and source domain prediction results output by the detection head, the classification loss and bounding box loss are calculated according to the corresponding pseudo-labels and ground truth labels, respectively. The total loss function is calculated based on the decoupling loss, classification loss, and bounding box loss of the source domain image samples and target domain image samples. The adjustable parameters of the student detector and teacher detector are optimized using the total loss function until convergence, and the trained student detector and teacher detector are obtained. The target domain image is acquired and input into the trained teacher detector to obtain the target detection result.

[0078] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An unsupervised domain adaptation object detection method, characterized in that, The method comprises: obtaining a source domain data set and a target domain data set, the source domain data set comprising a plurality of source domain image samples with true value labels, and the target domain data set comprising a plurality of unlabeled target domain image samples; constructing a target detection training network according to a pre-trained student detector and a teacher detector, the student detector and the teacher detector both comprising a backbone network and a detection head, and in the student detector, a beneficial information mining unit is integrated at the end of the backbone network, and a beneficial information decoupling unit is integrated in the detection head; inputting the source domain image samples and the target domain image samples as a group of samples into the target detection training network, generating pseudo labels of the target domain image samples by the teacher detector, modulating feature distribution by estimating potential offset of feature statistics by the beneficial information mining unit, so as to obtain target domain and source domain optimization features with implicit beneficial information, and then performing information bottleneck constraint on the first layer features of the detection head by the beneficial information decoupling unit to obtain target domain and source domain beneficial information decoupling loss, and meanwhile, according to the target domain and source domain prediction results output by the detection head, classification loss and bounding box loss are calculated according to the corresponding pseudo labels and true value labels respectively; calculating a total loss function according to the beneficial information decoupling loss, the classification loss and the bounding box loss of the source domain image samples and the target domain image samples, and optimizing adjustable parameters of the student detector and the teacher detector by using the total loss function until convergence, so as to obtain trained student detector and teacher detector; obtaining a target domain image, and inputting the target domain image into the trained teacher detector to obtain a target detection result.

2. The unsupervised domain adaptation object detection method of claim 1, wherein, In the beneficial information mining unit: channel-by-channel calculation is performed, and feature statistics of features extracted by the backbone network according to a batch of sample data are calculated, the feature statistics comprising mean and standard deviation; potential offset is estimated by using variance of feature statistics between a plurality of batches of sample data; new feature statistics are generated based on a multivariate Gaussian distribution corresponding to the potential offset to modulate the features extracted by the backbone network, so as to obtain optimization features.

3. The unsupervised domain adaptation object detection method of claim 2, wherein, The optimization features are expressed as: In the above formula, denotes a feature extracted by a backbone network, and denotes a feature a feature statistic of the feature and denotes the new feature statistic.

4. The unsupervised domain adaptation object detection method according to any one of claims 1-3, characterized in that, when pre-training the student detector and the teacher detector: the student detector is pre-trained by using the source domain data set to obtain a pre-trained student detector; parameters of the pre-trained student detector are copied to the teacher detector as initial parameters.

5. The unsupervised domain adaptation object detection method of claim 4, wherein, when optimizing the adjustable parameters of the student detector and the teacher detector by using the total loss function: the adjustable parameters in the student detector are optimized by using the total loss function, and the adjustable parameters of the teacher detector are optimized by using exponential moving average.

6. The unsupervised domain adaptation object detection method of claim 5, wherein, In the training process, the prediction result output by the detection head is semantic output without non-maximum suppression processing.

7. The unsupervised domain adaptation object detection method of claim 5, wherein, The classification loss adopts quality focal loss.

8. An unsupervised domain adaptation object detection apparatus, characterized by, The device comprises: a sample obtaining module, configured to obtain a source domain data set and a target domain data set, the source domain data set comprising a plurality of source domain image samples with true value labels, and the target domain data set comprising a plurality of unlabeled target domain image samples; The target detection training network construction module is configured to construct a target detection training network according to a pre-trained student detector and a teacher detector, wherein the student detector and the teacher detector both include a backbone network and a detection head, and in the student detector, a beneficial information mining unit is integrated at the end of the backbone network, and a beneficial information decoupling unit is integrated in the detection head; The loss calculation module is configured to input the source domain image sample and the target domain image sample into the target detection training network as a group of samples, generate a pseudo label of the target domain image sample by the teacher detector, modulate a feature distribution by estimating a potential offset of a feature statistic by the beneficial information mining unit, so as to obtain target domain and source domain optimization features with implicit beneficial information, perform information bottleneck constraint on the first layer feature of the detection head by the beneficial information decoupling unit, so as to obtain a target domain and source domain beneficial information decoupling loss, and calculate a classification loss and a bounding box loss according to a target domain and source domain prediction result output by the detection head and a corresponding pseudo label and true value label respectively; The network training module is configured to calculate a total loss function according to the beneficial information decoupling loss, the classification loss and the bounding box loss of the source domain image sample and the target domain image sample, optimize adjustable parameters of the student detector and the teacher detector by using the total loss function until convergence, and obtain a trained student detector and a trained teacher detector. The target detection module is configured to obtain a target domain image, input the target domain image into the trained teacher detector, and obtain a target detection result. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7.

Images