A dual-domain pseudo-label generation cross-domain target detection method based on detr

Abstract

The application relates to the technical field of computer vision, and proposes a dual-domain pseudo-label generation cross-domain target detection method based on DETR. A multi-scale decoding query clustering module can accurately capture the feature information of targets of different scales by performing multi-scale clustering on the query output by the decoder, thereby effectively evaluating the confidence of the pseudo-label. The method has strong cross-domain migration capability, can generate higher-quality pseudo-labels between target domain and source domain style data, optimizes the confidence of the pseudo-labels, and further improves the accuracy and robustness of target detection, significantly reducing the missed detection and false detection problems of pseudo-labels of different scales. The dual-domain cooperation and dual-domain verification module combines the pseudo-labels under different confidence levels of the source domain and target domain style images, enhances the reliability of the pseudo-label generation, effectively filters noise, improves the adaptability of cross-domain data, ensures that the model can smoothly migrate between different fields, and further improves the detection precision and generalization capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a dual-domain pseudo-label generation cross-domain target detection method based on DETR. BACKGROUND

[0002] Target detection technology has made significant progress with the development of deep learning, but most target detection models are difficult to effectively generalize to target domains with different visual features and data distributions after being trained on specific datasets (such as COCO, Pascal VOC, etc.). The visual differences (such as object appearance, background environment, lighting conditions, etc.) between the source domain and the target domain lead to the problem of "inter-domain difference" or "domain shift", which affects the performance of the target detection model in the target domain. In order to solve this problem, domain adaptive target detection has emerged, which aims to transfer the knowledge of the source domain to the target domain through unsupervised learning or weakly supervised learning, and reduce the negative impact of inter-domain differences on the target detection model.

[0003] Domain adaptive target detection methods can be mainly divided into two categories: feature alignment methods and pseudo-label generation methods. Feature alignment methods reduce the feature distribution difference between the source domain and the target domain through adversarial training and other means, so as to extract domain-invariant features and realize cross-domain detection. However, feature alignment is difficult to achieve perfectly when there are significant visual differences, and the supervision signal of the source domain is too much to affect the learning of the target domain, resulting in insufficient generalization ability of the model. Pseudo-label generation methods first train a detector on the source domain, then generate pseudo-labels for the target domain, and use the pseudo-labels to train the model. The teacher-student framework is a common strategy in this method, the teacher model is responsible for generating pseudo-labels, and the student model is trained through pseudo-labels, and the parameters of the teacher model are updated through exponential moving average to maintain the stability of the pseudo-labels. Current pseudo-label generation methods have achieved good results in some scenarios, but due to the inter-domain difference, the quality of the pseudo-labels still has great uncertainty, and the existence of false positive and false negative labels seriously affects the training effect of the model.

[0004] Despite the progress made in domain adaptation object detection methods, there are still several challenges. First, the feature alignment method is difficult to achieve perfect alignment in the case of large domain differences, which leads to the performance of the model on the target domain not reaching the ideal effect. Second, the quality of the pseudo-label generation method depends on the confidence estimation of the teacher model, while the confidence of the teacher model is easily affected by the difference between the source domain and the target domain, resulting in inaccurate pseudo-labels. Many existing teacher-student models based on convolutional neural networks for knowledge transfer, but the convolutional neural network model lacks global context information and modeling of inter-instance relationships, which makes the detection results in the target domain, especially the low confidence candidate frame, prone to misjudgment. In addition, the pseudo-label generated by the teacher model often has a large uncertainty when the target domain label is missing, further affecting the training effect of the student model. Therefore, it is crucial to study a method that can improve the generalization ability of the target detection model and improve the quality of the pseudo-label for the unsupervised domain adaptation object detection task. SUMMARY

[0005] In order to solve the problem of feature alignment difficulty between source domain and target domain and unstable quality of pseudo-label in unsupervised domain adaptation object detection, the present application proposes a DETR-based dual-domain pseudo-label generation cross-domain object detection method. The purpose is to solve the problem of feature alignment difficulty between source domain and target domain and unstable quality of pseudo-label in unsupervised domain adaptation object detection. Further improve the generalization ability of the target detection model on the target domain, generate high-quality pseudo-labels, reduce false positives and false negatives, and further enhance the cross-domain detection performance and reduce the dependence on accurate feature alignment.

[0006] In order to achieve the above purpose, the present application provides a DETR-based dual-domain pseudo-label generation cross-domain object detection method applied to the unsupervised domain adaptation object detection task, the method comprising:

[0007] S1: constructing a teacher-student model framework on a DETR-like detector, which is composed of a trained student model and an inferred teacher model;

[0008] S2: obtaining source domain and target domain images, training a CUT model, and inferring the corresponding source domain and target domain style images through the CUT model;

[0009] S3: using a pre-trained model trained on the COCO dataset, further training the student model and the teacher model through the labeled source domain image and the target domain style image, and obtaining the initialized student model and the teacher model;

[0010] S4: a multi-scale decoding query clustering module is constructed, the teacher model inferences the target domain and the source domain style images to obtain a decoding query, and the query similarity is calculated through the multi-scale decoding query clustering module to serve as a confidence level; each query represents a potential object and contains class information and bounding box prediction information;

[0011] S5: a dual-domain collaboration module is constructed, and the high confidence level results of the target domain and the source domain style images are fused together to serve as reliable pseudo labels;

[0012] S6: a dual-domain verification module is constructed, high coincidence samples with the same class detection frame IOU value exceeding a threshold value are extracted from the difficult samples inferred from the target domain and the source domain style images to serve as reliable pseudo labels, and the reliable pseudo labels generated by the dual-domain collaboration module are fused to obtain final pseudo labels;

[0013] S7: the final pseudo labels are used as the labels of the target domain images for training the student model, and the teacher model is continuously updated, and finally a trained teacher-student model is obtained for cross-domain target detection.

[0014] A storage device stores instructions and data for implementing the DETR-based dual-domain pseudo label generation cross-domain target detection method.

[0015] A DETR-based dual-domain pseudo label generation cross-domain target detection device includes a processor and a storage device; the processor loads and executes the instructions and data in the storage device to implement the DETR-based dual-domain pseudo label generation cross-domain target detection method.

[0016] The DETR-based dual-domain pseudo label generation cross-domain target detection method has the advantages that the method is used for unsupervised field self-adaptive target detection. The multi-scale decoding query clustering module introduced can accurately capture the feature information of targets of different scales by performing multi-scale clustering on the queries output by the decoder, effectively improves the robustness of the pseudo label confidence level evaluation in the multi-scale target detection task, and effectively evaluates the confidence level of the pseudo label. The method has strong cross-domain migration ability, can generate higher quality pseudo labels between target domain and source domain style data, optimizes the pseudo label confidence level, and thus improves the accuracy and robustness of target detection, significantly reduces the missed detection and false detection problems of pseudo labels of different scales, and enhances the performance of the model in complex scenes.

[0017] The introduced dual-domain cooperation and dual-domain verification module enhances the reliability of pseudo label generation and the robustness of cross-domain learning by combining pseudo labels under different confidence levels of source domain and target domain style images, improves the adaptability of cross-domain data, ensures the smooth migration of the model between different domains, and further improves the detection accuracy and generalization ability. Effectively improve the quality of pseudo labels and reduce noise and redundancy, improve the adaptability and detection performance of the model in the target domain.

[0018] By introducing the multi-scale decoding query clustering module, the dual-domain cooperation and dual-domain verification module, the generation and filtering process of pseudo labels are optimized, false positives and false negatives are reduced, and the generalization ability of the cross-domain detection model is enhanced. Through this method, the performance of the target detection model in the target domain is improved, the quality of the pseudo label is significantly improved, the dependence on accurate feature alignment is reduced, and the effect of the unsupervised domain adaptation target detection task is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A dual-domain pseudo label generation cross-domain target detection method framework based on DETR is proposed for the present application;

[0020] Figure 2 A multi-scale decoding query clustering module flowchart is provided for the embodiment of the present application;

[0021] Figure 3 A dual-domain cooperation calculation method flowchart is provided for the embodiment of the present application;

[0022] Figure 4 A dual-domain verification calculation method flowchart is provided for the embodiment of the present application;

[0023] Figure 5 A class-based non-maximum suppression flowchart is provided for the embodiment of the present application;

[0024] Figure 6 A schematic diagram of the working of the hardware device is provided for the embodiment of the present application. DETAILED DESCRIPTION

[0025] The principles and features of the present application are described below in conjunction with the accompanying drawings, and the examples are only used to explain the present application and are not used to limit the scope of the present application.

[0026] Embodiment 1

[0027] The present embodiment discloses a dual-domain pseudo label generation cross-domain target detection method based on DETR, which is applied to an unsupervised domain adaptation target detection task, and the method comprises:

[0028] S1: build a teacher-student model framework on a DETR-like detector, which is composed of a trained student model and an inferred teacher model;

[0029] S2: obtain source domain and target domain images, train a CUT model, and infer corresponding source domain and target domain style images through the model;

[0030] S3: use a pre-trained model trained on the COCO dataset to further train the student model and the teacher model through the labeled source domain images and the target domain style images, and obtain the initialized student model and the teacher model.

[0031] S4: build a multi-scale decoding query clustering module, the teacher model infers the target domain and source domain style images to obtain decoding queries, and calculates the query similarity through the module as the confidence; each query represents a potential object and contains class information and bounding box prediction information;

[0032] S5: build a dual-domain collaboration module to fuse the high-confidence results of the target domain and source domain style images together as reliable pseudo-labels;

[0033] S6: build a dual-domain verification module to extract high-overlap samples with class detection box IOU values exceeding a threshold from the difficult samples inferred from the target domain and source domain style images as reliable pseudo-labels, and fuse them with the reliable pseudo-labels generated by the dual-domain collaboration module to obtain the final pseudo-labels;

[0034] S7: use the pseudo-labels as the labels of the target domain images for the training of the student model, while the teacher model is constantly updated, and finally obtain the trained model.

[0035] The following describes each step in detail.

[0036] Further, in step S1, a teacher-student model framework is built on a DETR-like detector, as shown in Figure 1 , which is composed of two DAB Deformable DETR models, both using ResNet-50 as a feature extractor. The steps are as follows:

[0037] S101: build a student model, update the gradient through back propagation and loss value. The teacher model is updated through exponential moving average (EMA) of the student model parameters, as shown in the formula:

[0038]

[0039] where θ t and θ sα represents the parameter weights of the teacher model and the student model, respectively, k is the number of training iterations, and α is the smoothing coefficient of EMA, used to control the speed of parameter updates.

[0040] S102: This framework is mainly used by the teacher model to generate pseudo-labels, which are then used by the student model to train for target localization and classification tasks in the target domain.

[0041] Furthermore, in step S2, source and target domain images are acquired, a CUT model is trained, and the model is used to generate corresponding source and target domain style images. The specific steps are as follows:

[0042] S201: Acquire image data of source and target domains with different visual styles, and train the CUT model based on these data.

[0043] S201: After training, use the model to perform style transfer inference on images in the source and target domains, and generate corresponding source domain style images and target domain style images respectively.

[0044] Furthermore, in step S3, a pre-trained COCO model is used for supervised training on style data from both the source and target domains. The specific steps are as follows:

[0045] S301: Download the detection model trained on the COCO dataset and modify the detection head network according to the needs of the actual cross-domain task to obtain a pre-trained model.

[0046] S302: Further supervised training of the pre-trained model using labeled source domain images and target domain style images, and select the student model or teacher model with the best performance during the process.

[0047] Furthermore, in step S4, a multi-scale decoding query clustering module is constructed, such as... Figure 2 As shown, the teacher model infers the style images of the target domain and the source domain to obtain the decoded query, and calculates the query similarity as the confidence score through this module. The specific steps are as follows:

[0048] S401: Given a target image, the encoder of the teacher model first extracts image features and generates encoded features, as shown in the formula:

[0049]

[0050] in, For the i-th image in the target domain, The Encoder is the encoding feature, and the Backbone is the image feature extractor.

[0051] Next, the decoder utilizes the features output by the encoder. and object A set of decoded objects query is generated, as shown in equation (1):

[0052]

[0053] wherein, denotes the decoded object query, N is the number of query, and d is the dimension of each query. Each query represents a potential object, containing class information and bounding box prediction information, and after processing by the detection head, the inferred target information is obtained, as shown in equation (2):

[0054]

[0055] wherein, is the detection confidence, is the predicted class label, is the refined bounding box;

[0056] S402: Select the detection confidence exceeding threshold τ r as reliable decoded objects Scale information denotes the area proportion of the object, as shown in equation (3):

[0057]

[0058] wherein, is the area of the refined bounding box, is the total area of the input image, denotes the i-th target domain image;

[0059] S403: Aggregate similar object features in the feature space to form a consistent and compact class-scale prototype set. Extract the class information and scale information of reliable decoded objects from equations (4) and (5), and then aggregate features with the same class c and the same scale r to generate a class-scale prototype through feature averaging. The formula is as follows:

[0060]

[0061] wherein, denotes the reliable object output by the decoder, and denote the class and scale information of the corresponding object, respectively. Symbol is an indicator function, which is 1 when the condition is true, and 0 otherwise. N t is the total number of target domain images, For each target domain image The total number of objects in the image. For class scale prototype, where N class is the number of classes, N scale is the number of scale levels, and d is the feature dimension. By clustering the features of the same class and scale, the generated class scale prototype aims to accurately represent the object features under different class and scale conditions. In the teacher-student mutual learning stage, the class scale prototype will be updated continuously as the training progresses. The update process is the same as described in formula (6), and the updated prototype is generated by aggregating and averaging the features of the same class c and scale level r, so as to achieve continuous optimization.

[0062] S404: A new confidence evaluation method is designed by calculating the cosine similarity between the decoded query generated by the decoder and the pre-constructed class scale prototype As shown in formula (7), the prototype with the highest similarity to the query is selected as the confidence source of the query, and the maximum similarity is regarded as the clustering confidence of the object. At the same time, the class index related to the class scale prototype with the highest cosine similarity is assigned as the class label of the query, so as to re-evaluate the confidence of the query and possibly change its initial class.

[0063]

[0064] wherein, is the clustering confidence score calculated by the maximum similarity, is the class label of the jth object assigned by the maximum cosine similarity.

[0065] Further, in step S5, a dual-domain collaborative module is constructed, as shown in Figure 3 , which fuses the high-confidence inference results of the target domain and the source domain style images into reliable pseudo labels. The steps are as follows:

[0066] S501: In the target domain and the source domain style image, a selection strategy based on clustering confidence score is used to screen high-confidence pseudo labels. In the target domain, only the target with clustering confidence score greater than the predefined reliability threshold is selected as the candidate target pseudo label The formula is:

[0067]

[0068] wherein, is the bounding box of the jth object in the ith target domain image, Let j be the category label for the j-th object. This represents the total number of objects in each target domain image;

[0069] S502: Perform a multi-scale decoding query clustering module on source-domain style images to generate source-domain style category-scale prototypes. Subsequently, cluster confidence scores were obtained from the source domain style images. Greater than the preset threshold To filter source domain style images and obtain reliable pseudo-labels The formula is:

[0070]

[0071] in, Let be the bounding box of the j-th object in the i-th source-domain style image. Let be the category label of the j-th object in the i-th source domain style image. For each source domain style image The total number of objects in the array.

[0072] S503: As Figure 3 As shown, the target domain image and corresponding source domain style images The only difference is in the image style; the category and location of the foreground objects remain consistent. Therefore, reliable pseudo-labels generated from both the target and source domain style images can be combined to generate the final high-confidence target domain pseudo-labels. Then, the category-based nonmaximum suppression (NMS) method is applied to these two sets of pseudo-labels to eliminate redundant and excessively overlapping predictions, thereby obtaining the final reliable pseudo-labels for the target domain. The formula is:

[0073]

[0074] Among them, NMS class Represents a category-based nonmaximum suppression operation, and ∪ represents the target domain pseudo-label. Heyuan Domain Style Pseudo-tags The union of .

[0075] Furthermore, in step S6, a two-domain verification module is constructed, such as... Figure 4 As shown, difficult pseudo-tags are mined and fused with reliable pseudo-tags generated by the dual-domain collaboration module to obtain the final pseudo-tags.

[0076] The specific steps are as follows:

[0077] S601: The dual-domain verification module selects reliable pseudo-labels with lower clustering confidence scores by leveraging the consistency between target images and corresponding source domain style images. First, select n candidate bounding boxes for target domain data with clustering confidence scores lower than a pre-set threshold , and n candidate bounding boxes for corresponding source domain style images with clustering confidence scores lower than a threshold , denoted as and Although the clustering confidence of these samples is low, they may still retain valuable information.

[0078] S602: The target domain image and its corresponding source domain style image differ in visual style, but the class and location of the foreground object remain consistent. Then, the Intersection over Union (IoU) metric is used to quantify the overlap between samples of the same class in and , and the pseudo-labels with higher overlap are selected as reliable difficult samples , whose formula is:

[0079]

[0080] where represents the bounding box of the kth object in the ith source domain style image, represents the class label of the kth object in the ith source domain style image, represents the bounding box of the kth object in the ith target domain image. When the Intersection over Union (IoU) exceeds the overlap threshold δ oss , the sample is retained.

[0081] S603: Since a single foreground object may be captured by multiple decoded object queries with different confidence scores, resulting in the same candidate bounding box being selected in both reliable pseudo-labels and difficult pseudo-labels. To reduce confusion, perform class-based Non-Maximum Suppression (NMS) on and , as shown in Figure 5 , with the formula:

[0082]

[0083] where represents the final pseudo-label set of the ith target domain image after merging target domain and source domain style data, NMS class represents the class-level non-maximum suppression operation applied to remove redundant labels.

[0084] Further, in step S7, the pseudo label is taken as the label of the target domain image to guide the student model training, while the teacher model is continuously updated to obtain the finally trained model. The steps are specifically as follows:

[0085] S701: The training of the teacher-student model is divided into two stages. In the initial stage, a dual-source domain supervision training strategy is adopted to gradually guide the model to adapt to the target domain by using the source domain data and the target domain style data, enhance the field-specific learning, and promote effective migration. For the supervised training loss, the formula is:

[0086]

[0087] wherein, represents the detection loss, including the source domain detection loss and the target stylized detection loss Each detection loss includes an L1 loss and a generalized intersection over union (GIoU) loss.

[0088] S702: The source domain and the target domain style data share the same foreground object and label, so the model needs to achieve consistent prediction between the two. To strengthen this goal, a consistency loss is introduced, which is calculated by the L2 distance between the source domain detection loss and the target stylized detection loss The formula is:

[0089]

[0090] S703: In the teacher-student co-learning stage, a distillation loss is introduced to enhance the generalization ability of the model. The teacher model generates pseudo labels t from the target domain data I (s←t) and the source domain stylized data I and guides the student model to train on the target data, and finally obtains the distillation loss The formula is:

[0091]

[0092] S704: The final total loss is the weighted sum of the detection loss, the consistency loss and the distillation loss, wherein the distillation loss is only introduced in the teacher-student co-learning stage. Proper weighting coefficients can balance the influence of each loss term on model optimization.

[0093] The overall total loss expression is:

[0094]

[0095] wherein, α, β and γ are the weighting coefficients of each loss term, and ​Only used in the teacher-student co-learning phase.

[0096] The relevant details of the method are described below:

[0097] (1) Multi-scale decoding query clustering: This method aims to enhance the detection performance of the source domain training model on different target domains and multi-scale targets. This module first extracts features from the multi-scale object queries output by the decoder, groups these queries according to their scale size in terms of image area proportion, and calculates the average value of the features in different categories and different scales to generate multi-scale category prototypes. Each prototype represents the feature center of a target in different scales. These prototypes not only retain the key information of the category, but also adapt to the changes of the target in different scales. Then, the module calculates the cosine similarity between each decoding query and each category prototype, and uses the resulting similarity value as the confidence score of the query. This confidence score is different from the detection score generated by the traditional detection head, and can generate more accurate confidence scores for specific domains, thereby more effectively distinguishing reliable pseudo labels, difficult pseudo labels, and unreliable pseudo labels in different domains. In addition, this similarity calculation based on multi-scale prototypes can effectively capture the features of targets in different scales in different target domain images, thereby improving the performance of the model in cross-domain detection tasks.

[0098] (2) Dual-domain collaboration and dual-domain verification: This module improves the quality of pseudo labels by combining the complementary information of target domain and source domain style images. In the dual-domain collaboration module, the target domain and source domain style images are inferred by a multi-scale decoder to generate high-confidence pseudo labels, and these labels are filtered based on clustering confidence. In this process, the style difference between the source domain and the target domain is taken into account to ensure the reliability of the pseudo labels. In the dual-domain verification module, the consistency between the target domain and the source domain style images is used to further filter out difficult samples with high overlap, and these samples are used as reliable pseudo labels. Finally, the pseudo labels generated by the two modules are fused. By introducing a class-based non-maximum suppression (NMS) method, redundant and excessively overlapping pseudo labels are removed to ensure that the final generated pseudo labels are more accurate and representative. The combination of dual-domain collaboration and verification can effectively improve the detection performance of the target domain, reduce pseudo label noise, and improve the generalization ability of the model.

[0099] To verify the finiteness of the method of the present application, experiments are conducted on four mainstream unsupervised domain adaptive object detection datasets, including Cityscapes, Foggy Cityscapes, Bdd100k, and Sim10k:

[0100] Cityscapes is a high-resolution dataset containing urban street scenes, primarily used for scene understanding, comprising 2,975 training images and 500 validation images, suitable for cross-weather and cross-scene adaptation studies.

[0101] Foggy Cityscapes is a target dataset generated based on the Cityscapes dataset through a haze synthesis algorithm, designed to evaluate cross-weather generalization under low visibility, with a focus on scenes with a haze density of 0.02.

[0102] BDD100k is a large-scale dataset containing 36,278 daytime driving images, used for cross-scene adaptation experiments to evaluate the ability to adapt to 7 target classes under different daytime lighting and visual conditions.

[0103] Sim10k is a synthetic dataset containing 10,000 virtual street scene images labeled with vehicle instances, used for adaptation from synthetic to real scenes, to evaluate the detection ability of models from virtual environments to real-world urban environments.

[0104] Evaluation metrics: This method uses AP50 as the evaluation metric, representing the average precision of the model in the target detection task when the intersection over union (IoU) threshold is 0.5. By weighting the AP50 values of all classes, the mAP metric is obtained, which comprehensively evaluates the overall performance of multi-class target detection models.

[0105] Experimental indicators:

[0106] Table 1: Evaluate experimental performance on Cityscapes to Foggy Cityscapes.

[0107]

[0108]

[0109] Table 2: Evaluate experimental performance on Cityscapes to Bdd100k.

[0110]

[0111] Table 3: Evaluate experimental performance on Sim10k to Cityscapes.

[0112]

[0113]

[0114] Through the experimental results of Table 1, Table 2 and Table 3, it can be seen that the method is obviously superior to the existing method, improves the detection performance under different cross-domain tasks, and proves the superiority of the method.

[0115] Embodiment 2

[0116] A DETR-based dual-domain pseudo-label generation cross-domain target detection device 401, as shown in Figure 6 The processor 402 loads and executes the instructions and data in the storage device 403 to implement the DETR-based dual-domain pseudo-label generation cross-domain target detection method.

[0117] Embodiment 3

[0118] A storage device stores instructions and data for implementing the DETR-based dual-domain pseudo-label generation cross-domain target detection method.

[0119] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A DETR-based dual-domain pseudo-label generation cross-domain object detection method, applied to a computer vision unsupervised field self-adaptive object detection task, the method comprising: S1: constructing a teacher-student model framework on a DETR-like detector, which is composed of a trained student model and an inferred teacher model; S2: obtaining source domain and target domain images, training a CUT model, and inferring corresponding source domain and target domain style images through the CUT model; S3: using a pre-trained model trained on the COCO dataset, further training the student model and the teacher model through the labeled source domain images and the target domain style images to obtain the initialized student model and the teacher model; S4: constructing a multi-scale decoding query clustering module, the teacher model infers the target domain and source domain style images to obtain decoding queries, and the multi-scale decoding query clustering module is used to calculate the query similarity as the confidence; each query represents a potential object and contains class information and bounding box prediction information; S5: constructing a dual-domain collaboration module, and fusing the high confidence results of the target domain and source domain style images together as reliable pseudo-labels; S6: constructing a dual-domain verification module, extracting high overlap samples with class detection box IOU values exceeding a threshold from the difficult samples inferred from the target domain and source domain style images as reliable pseudo-labels, and fusing them with the reliable pseudo-labels generated by the dual-domain collaboration module to obtain the final pseudo-labels; S7: using the final pseudo-labels as the labels of the target domain images for training the student model, while the teacher model is constantly updated, and finally obtaining the trained teacher-student model for cross-domain object detection.

2. The dual-domain pseudo-label generation cross-domain object detection method based on DETR according to claim 1, wherein, Both the teacher model and the student model are composed of DAB Deformable DETR, wherein DAB Deformable DETR uses ResNet-50 as a feature extractor, the student model updates the gradient through back propagation and loss value, and the teacher model updates the parameters of the student model through EMA, the formula is: where, and denote the parameter weights of the teacher model and student model at the kth iteration, respectively, and a is the smoothing factor of EMA, which is used to control the speed of parameter update.

3. The dual-domain pseudo-label generation cross-domain object detection method based on DETR of claim 1, wherein, The specific implementation process of step S2 is as follows: S201: obtain image data of source domain and target domain with different visual styles, and train a CUT model based on the data; S202: after training, use the CUT model to perform style conversion inference on the source domain and target domain images to generate corresponding source domain style images and target domain style images.

4. The dual-domain pseudo-label generation cross-domain object detection method based on DETR of claim 1, wherein, The specific implementation process of step S3 is as follows: S301: download a detection model trained on the COCO dataset, and modify the detection head network according to the actual cross-domain task requirements to obtain a pre-trained model; S302: further supervise the training of the pre-trained model through labeled source domain images and target domain style images, and select the best student model or teacher model in the process.

5. The dual-domain pseudo-label generation cross-domain object detection method based on DETR of claim 1, wherein, The specific implementation process of step S4 is as follows: S401: given a target image, the encoder of the teacher model first extracts image features and generates encoded features: wherein, is the i-th image of the target domain, is the encoding feature, Encoder is the encoder, and Backbone is the image feature extractor. The decoder then utilizes the encoded features output by the encoder and objects to generate a set of decoded objects query: wherein, represents the decoded objects query, N is the number of queries, d is the dimension of each query, and Encoder is the decoder. Decoded object query After the detection head processing, the inferred target information is obtained: wherein, for detecting the confidence, for predicting the class label, for the refined bounding box, and Detection Head represents a detection head; S402: select detection confidence over threshold τ r query as reliable decoding object scale information scale information representing the area proportion of the object The calculation formula is: wherein, is the refined bounding box area, is the total area of the input image, denotes the i-th target domain image; S403: Aggregating similar object features in the feature space to form a consistent and compact class-scale prototype set; extracting reliable decoding objects from formulas (4) and (5) predicted class label and scale information Then the features with the same class c and the same scale r are aggregated to generate a class-scale prototype by feature averaging wherein, represents the reliable objects output by the decoder, and respectively represent the category and scale information of the corresponding objects; the symbol X{·} is an indicator function, χ{·} = 1 when the condition is true, and 0 otherwise; N t is the total number of target domain images, is the total number of objects in each target domain image ; is the category scale prototype, wherein N class is the number of categories, N scale is the number of scale levels, and d is the feature dimension; by clustering the features of the same category and scale, the generated category scale prototype aims to accurately represent the object features under different category and scale conditions; in the teacher-student mutual learning stage, the category scale prototype will be updated continuously as the training progresses; the updating process is the same as described in formula (6), and the updated prototype is generated by aggregating and averaging the features of the same category c and scale level r , thereby achieving continuous optimization; S404: generate cosine similarity between the decoded query generated by the decoder and the pre-built category scale prototype A new confidence evaluation method is designed by calculating the cosine similarity between the decoded query generated by the decoder and the pre-built category scale prototype. The specific process is as follows: as shown in formula (7), the prototype most similar to the query is selected as the confidence source of the query, and the maximum similarity is regarded as the clustering confidence of the object; at the same time, the category index related to the category scale prototype with the highest cosine similarity is assigned as the category label of the query, so as to reevaluate the confidence of the query and possibly change its initial category: wherein, is a cluster confidence score computed by maximum similarity, is a class label assigned to the jth object by maximum cosine similarity.

6. The dual-domain pseudo-label generation cross-domain object detection method based on DETR of claim 1, wherein, The specific implementation process of step S5 is as follows: S501: select high-confidence pseudo-labels in target domain and source domain style images using a selection strategy based on clustering confidence scores, in the target domain, only targets with clustering confidence scores greater than a predefined reliability threshold are selected as candidate target pseudo-labels wherein, is a bounding box of the jth object in the ith target domain image, is a class label of the jth object, is the total number of objects in each target domain image; S502: performing a multi-scale decoding query clustering module in the source domain style image to generate a class scale prototype of the source domain style Subsequently, the source domain style image is filtered according to the clustering confidence score greater than a preset threshold to obtain reliable pseudo labels wherein, is a bounding box of the jth object in the ith source domain style image, is a class label of the jth object in the ith source domain style image, is the total number of objects in each source domain style image . S503: Combine the reliable pseudo labels generated in the target domain and the source domain style image to generate the final high-confidence target domain pseudo label Then, a class-based non-maximum suppression method is applied to the two groups of pseudo labels to eliminate redundant and overly overlapping predictions, thereby obtaining the final target domain reliable pseudo label where NMS class denotes a class-based non-maximum suppression operation, and denotes the union of target domain pseudo labels and source domain style pseudo labels .

7. The dual-domain pseudo-label generation cross-domain object detection method based on DETR of claim 1, wherein, The specific implementation process of step S6 is as follows: S601: The dual-domain verification module selects reliable pseudo labels with lower clustering confidence scores by utilizing the consistency between the target image and the corresponding source domain style image: first, select n candidate bounding boxes with clustering confidence scores lower than a preset threshold for the target domain data, and n candidate bounding boxes with clustering confidence scores lower than the threshold for the corresponding source domain style image, respectively denoted as and . and S602: Use the IoU metric to quantify the overlap between samples in the same category and the overlap degree between samples in the same category, and select the pseudo label with high overlap degree as reliable difficult samples The formula is: wherein, represents a bounding box of the kth object in the ith source domain style image, represents a class label of the kth object in the ith source domain style image, represents a bounding box of the kth object in the ith target domain image; when the intersection over union IoU exceeds δ oss , the sample is retained, wherein, δ oss is an overlap threshold; S603: Since a single foreground object can be captured by multiple decoded object queries with different confidence scores, resulting in the same candidate bounding box being selected in both reliable pseudo-labels and difficult pseudo-labels, in order to reduce confusion, the and perform class-based non-maximum suppression, the formula is: wherein, denotes the final pseudo label set of the i-th target domain image after merging the target domain and source domain style data, NMS class denotes a class level non-maximum suppression operation applied to remove redundant labels, denotes a target domain reliable pseudo label, denotes a high confidence target domain pseudo label.

8. The dual-domain pseudo-label generation cross-domain object detection method based on DETR of claim 1, wherein, The specific implementation process of step S7 is as follows: S701: The training of the teacher-student model is divided into two stages. In the initial stage, a dual-source domain supervision training strategy is adopted to gradually guide the model to adapt to the target domain by using source domain data and target domain style data, enhance domain-specific learning, and promote effective migration. For the supervised training loss, the formula is: wherein, represents a detection loss, represents a source domain detection loss, represents a target domain stylization detection loss; S702: If the source and target domain style data share the same foreground objects and labels, the model needs to achieve consistent predictions between them. To strengthen this goal, a consistency loss is introduced, using source domain detection loss. and target stylization detection loss The consistency loss is calculated using the L2 distance between them. The formula is: S703: In the teacher-student co-learning stage, introduce distillation loss to enhance the generalization ability of the model; the teacher model is generated from the target domain data I t and the source domain stylized data I (s←t) Generate pseudo labels and guide the training of the student model on the target data, and finally obtain the distillation loss The formula is: S704: The final total loss is the weighted sum of the detection loss, the consistency loss, and the distillation loss, wherein the distillation loss is only introduced in the teacher-student joint learning stage; appropriate weighting coefficients are used to balance the influence of each loss term on model optimization; The overall total loss expression is: where a, β and γ are weighting coefficients for each loss term, respectively, and Used only in the teacher-student co-learning phase.

9. A storage device, characterized by: The storage device stores instructions and data for implementing the DETR-based dual-domain pseudo-label generation cross-domain target detection method according to any one of claims 1-8.

10. A dual-domain pseudo-label generation cross-domain object detection device based on DETR, characterized by: Comprise: A processor and a storage device; the processor loads and executes the instructions and data in the storage device to implement the DETR-based dual-domain pseudo-label generation cross-domain target detection method according to any one of claims 1-8.

Images