A target detection method, device, and equipment

Abstract

The present invention provides a target detection method, device and equipment, which relates to the field of domain adaptation. The method includes: respectively inputting the target domain image samples after different processing into the pre-trained teacher model in the source domain and the student model to be trained in the target domain, so as to obtain teacher encoded features, teacher decoded features, student encoded features and student decoded features; obtaining an uncertainty weight based on the student class prediction result; obtaining a feature distillation loss based on the teacher encoded features, teacher decoded features, student encoded features and the uncertainty weight; obtaining a contrastive learning loss based on the pseudo labels, student class prediction results, student bounding box prediction results, student decoded features and a memory bank of multiple categories; updating the model based on the feature distillation loss and the contrastive learning loss to obtain a trained student model, so as to perform target detection on the image to be detected in the target domain, realizing the application of the DETR detection model to domain adaptation target detection in a source-free scenario.

Description

Technical Field

[0001] The present invention relates to the field of domain adaptation, and in particular, to an object detection method, apparatus, and device. Background Art

[0002] In recent years, with the rapid development of deep neural networks, computer vision technology has also experienced rapid development, and its application efficiency and performance in real-world environments have been significantly improved. Among them, object detection, as an important task in computer vision, has received extensive attention and research, and efficient deep object detection models such as Faster R-CNN, YOLO, and DETR have emerged. However, although these deep object detectors have achieved very good results on some datasets and benchmarks, they often require large-scale training datasets with high annotation quality to achieve satisfactory performance. In real-world environments, the collection and annotation of these data often require a large amount of manpower and financial resources, which hinders the widespread application of these object detectors in real-world environments. In addition, when applied in diverse real-world environments, these deep object detectors often experience performance degradation, and one of the important reasons is called domain shift. Domain shift means that there are certain differences in the feature distributions between the source domain and the target domain. For example, in natural images, there are differences in illumination, texture, shape, etc. These domain gaps will cause the model pre-trained on source domain data to be unable to be efficiently applied to target domain data to achieve satisfactory performance. To address this challenge, a large amount of research has been conducted on domain-adaptive object detection, and relatively successful progress has been made in this field: through methods and techniques such as domain difference terms, adversarial discriminators, contrastive feature learning, and robust learning, domain-difference adaptive object detection between domains can effectively bridge the source domain and the target domain, so as to achieve the effect of migrating the model from the source domain to the target domain for application.

[0003] Although domain-adaptive object detection can to some extent alleviate the performance degradation caused by domain shift, however, in more real-world application scenarios, the application and effects of these traditional domain-adaptive object detection methods are still limited. For example, currently most methods use adversarial alignment or intermediate domain generation methods for adaptation, but these methods rely on labeled source domain data. Obviously, when the source domain data cannot be directly obtained due to privacy protection and other reasons, these methods cannot perform their migration performance normally. This scenario is called source-free domain adaptation object detection, that is, in the domain adaptation process, the source domain data and its distribution are not visible, and only the pre-trained model containing source domain information participates in domain adaptation.

[0004] Fortunately, in response to the challenges of domain adaptive object detection in the above-mentioned restricted scenarios, some efficient solutions have been developed on specific architectures, such as semi-supervised style transfer through the mean teacher framework domain pseudo-label method. However, the design of related passive domain adaptive object detection frameworks often relies on traditional detection models and their specific components, such as Faster R-CNN and Region Proposal Network (RPN), but cannot be applied normally and efficiently in some novel and efficient frameworks, such as the DETR (Detection Transformer) detection model. Therefore, how to realize the application of the DETR detection model to domain adaptive object detection in passive scenarios is a technical problem to be solved by the present invention. Summary of the invention

[0005] Based on the above technical problems, the present invention provides a target detection method, device and equipment, aiming to realize the application of the DETR detection model to domain adaptive target detection in passive scenarios, and improve the detection accuracy of the DETR target detection model in passive domain adaptive scenarios.

[0006] A first aspect of the present invention provides a target detection method, the method comprising:

[0007] Performing weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and performing strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample;

[0008] Inputting the weakly enhanced sample into a pre-trained teacher model of the source domain, obtaining a teacher encoding feature output by an encoder of the teacher model and a teacher decoding feature output by a decoder of the teacher model;

[0009] Inputting the strongly enhanced sample into a student model to be trained in the target domain, obtaining a student encoding feature output by an encoder of the student model and a student decoding feature output by a decoder of the student model;

[0010] Based on the student category prediction results output by the student model, uncertainty estimation is performed to obtain uncertainty weights;

[0011] Obtaining a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight;

[0012] Obtaining contrastive learning loss based on pseudo labels, the student category prediction results, the student bounding box prediction results output by the student model, the student decoding features, and multiple categories of memory banks, wherein the pseudo labels are determined based on the teacher category prediction results and the teacher bounding box prediction results output by the teacher model;

[0013] Update the teacher model and the student model at least based on the feature distillation loss and the contrastive learning loss to obtain a trained student model;

[0014] Input the image to be detected in the target domain into the trained student model to obtain a target detection result.

[0015] A second aspect of the present invention provides a target detection device, the device comprising:

[0016] An image processing module, configured to perform weak augmentation processing on a target domain image sample to obtain a weakly augmented sample, and perform strong augmentation processing on the target domain image sample to obtain a strongly augmented sample;

[0017] A first input module, configured to input the weakly augmented sample into a pre-trained teacher model in the source domain to obtain a teacher encoded feature output by an encoder of the teacher model and a teacher decoded feature output by a decoder of the teacher model;

[0018] A second input module, configured to input the strongly augmented sample into a student model to be trained in the target domain to obtain a student encoded feature output by an encoder of the student model and a student decoded feature output by a decoder of the student model;

[0019] A first weight determination module, configured to perform uncertainty estimation based on a student class prediction result output by the student model to obtain an uncertainty weight;

[0020] A first loss determination module, configured to obtain a feature distillation loss based on the teacher encoded feature, the teacher decoded feature, the student encoded feature, and the uncertainty weight;

[0021] A second loss determination module, configured to obtain a contrastive learning loss based on a pseudo label, the student class prediction result, a student bounding box prediction result output by the student model, the student decoded feature, and a memory bank of multiple categories, where the pseudo label is determined according to a teacher class prediction result and a teacher bounding box prediction result output by the teacher model;

[0022] A model training module, configured to update the teacher model and the student model at least based on the feature distillation loss and the contrastive learning loss to obtain a trained student model;

[0023] A target detection module, configured to input the image to be detected in the target domain into the trained student model to obtain a target detection result.

[0024] In a third aspect of the embodiments of the present invention, an electronic device is provided. The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements the object detection method according to the first aspect of the embodiments of the present invention.

[0025] In the object detection method provided by the embodiments of the present invention, first, weak enhancement processing and strong enhancement processing are respectively performed on the target domain image samples to obtain weak enhancement samples and strong enhancement samples. Secondly, the weak enhancement samples are input into the pre-trained teacher model in the source domain to obtain the teacher encoding features output by the encoder of the teacher model and the teacher decoding features output by the decoder of the teacher model. And the strong enhancement samples are input into the student model to be trained in the target domain to obtain the student encoding features output by the encoder of the student model and the student decoding features output by the decoder of the student model. Then, uncertainty estimation is performed based on the student class prediction results output by the student model to obtain uncertainty weights. Based on the teacher encoding features, teacher decoding features, student encoding features, and uncertainty weights, a feature distillation loss is calculated. Then, based on the pseudo-labels determined by the teacher class prediction results and teacher bounding box prediction results output by the teacher model, the student class prediction results, the student bounding box prediction results output by the student model, the student decoding features, and a memory bank of multiple categories, a contrastive learning loss is calculated. Finally, at least based on the feature distillation loss and the contrastive learning loss, the teacher model and the student model are updated to obtain a trained student model. The trained student model is used to perform object detection on the image to be detected in the target domain to obtain an object detection result.

[0026] Through the object detection method of this embodiment, the teacher-student model framework is applied as the basic unsupervised domain adaptation object detection framework. During the process of domain adaptation using unsupervised target domain data (i.e., target domain image samples), the more robust teacher model generates pseudo-labels and provides them to the student model for training, and the student model is updated through gradients. Among them, considering the characteristic of the DETR object detection framework that it tries to assign only one feature query to the same object and the problem of class imbalance in the real-world environment, in this embodiment, under the basic contrast loss, a memory bank of multiple categories is used to obtain a large number of high-quality and as category-balanced as possible features for contrast learning. Thus, the contrast learning loss is calculated through pseudo-labels, the student class prediction results, the student bounding box prediction results, the student decoded features, and the memory bank of multiple categories, so as to learn higher-quality category features on the target domain. And considering the characteristics of the DETR object detection framework and the fact that the true labels are invisible in the unsupervised domain adaptation object detection task, this embodiment uses the prediction uncertainty to measure the prediction quality, obtains the uncertainty weight, and then calculates the feature distillation loss based on the teacher encoded features, the teacher decoded features, the student encoded features, and the uncertainty weight, so as to achieve more robust knowledge distillation between the teacher model and the student model. In this way, this embodiment updates the teacher model and the student model at least based on the contrast learning loss of the matching category memory bank and the feature distillation loss based on uncertainty weighting, obtains the student model trained on the target domain, and performs object detection on the images to be detected in the target domain, solving the problem that the DETR detection model cannot be applied to the domain adaptation object detection in the unsupervised scenario, improving the detection accuracy of the DETR object detection model in the unsupervised domain adaptation scenario, and enabling the DETR object detection model to have efficient and stable detection performance under various complex domain shifts in the real-world scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 It is a flowchart of the steps of an object detection method shown in an embodiment of the present invention;

[0029] Figure 2 It is a schematic diagram of an unsupervised domain adaptation object detection framework based on DETR shown in an embodiment of the present invention;

[0030] Figure 3 It is a structural block diagram of an object detection device provided by an embodiment of the present invention;

[0031] Figure 4 It is a schematic diagram of an electronic device shown in an embodiment of the present invention. Detailed implementation manners

[0032] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0033] Please refer to Figure 1 , Figure 1 It is a step flowchart of a target detection method shown in an embodiment of the present invention. As Figure 1 shown, the target detection method provided in this embodiment at least includes the following steps:

[0034] Step S11: Perform weak augmentation processing on the target domain image sample to obtain a weakly augmented sample, and perform strong augmentation processing on the target domain image sample to obtain a strongly augmented sample.

[0035] In this embodiment, weak augmentation processing and strong augmentation processing can be respectively performed on the same target domain image sample to obtain a weakly augmented sample and a strongly augmented sample. Among them, the target domain image sample is an image sample used for model training in the target domain, and the target domain is the domain to be migrated. Strong augmentation processing refers to performing a relatively large degree of transformation on the target domain image sample, usually changing some basic features of the target domain image sample. Strong augmentation processing at least includes one or more of the following: rotation, flipping, shearing, adding noise, mixing different data, so as to increase the diversity of training data without changing the image category, thereby improving the robustness of the model. Weak augmentation processing refers to performing a relatively small degree of transformation on the target domain image sample, usually not changing the basic features of the image. Weak augmentation processing at least includes one or more of the following: scaling, translation, changing the color space, so as to enable the model to learn more local features through small perturbations to the image, thereby improving the model performance.

[0036] Step S12: Input the weakly augmented sample into the pre-trained teacher model in the source domain to obtain the teacher encoding features output by the encoder of the teacher model and the teacher decoding features output by the decoder of the teacher model.

[0037] In this embodiment, the pre-trained teacher model in the source domain can be a trained object detection model in the source domain, such as a trained DETR object detection model in the source domain. The pre-trained teacher model in the source domain at least includes: an encoder and a decoder. After obtaining the weakly augmented samples, the weakly augmented samples can be input into the pre-trained teacher model in the source domain to obtain the teacher encoding features output by the encoder of the teacher model and the teacher decoding features output by the decoder of the teacher model.

[0038] In an alternative embodiment, after obtaining the teacher encoding features, the teacher encoding features are input into the decoder of the teacher model together with a first tensor to be trained to obtain the teacher decoding features output by the decoder of the teacher model. The first tensor to be trained is the initial object query in the teacher model, which can be understood as an initial feature tensor representation. The first tensor to be trained performs related operations with the decoder features in the transformer architecture of the decoder, guiding the decoder to output features according to the tensor dimension of the first tensor to be trained, that is, the teacher decoding features output by the decoder have the same dimension as the first tensor to be trained.

[0039] Step S13: Input the strongly augmented samples into the student model to be trained in the target domain to obtain the student encoding features output by the encoder of the student model and the student decoding features output by the decoder of the student model.

[0040] In this embodiment, the student model to be trained in the target domain can be a target detection model to be trained in the target domain, such as a DETR target detection model to be trained in the target domain. The student model to be trained in the target domain also at least includes: an encoder and a decoder. The student model to be trained in the target domain is initialized using the model parameters of the pre-trained teacher model in the source domain. After obtaining the strongly augmented samples, the strongly augmented samples can be input into the student model to be trained in the target domain to obtain the student encoding features output by the encoder of the student model and the student decoding features output by the decoder of the student model.

[0041] In an alternative embodiment, after obtaining the student encoding features, the student encoding features are input into the decoder of the student model together with a second tensor to be trained to obtain the student decoding features output by the decoder of the student model. The second tensor to be trained is the initial object query in the student model, which can be understood as an initial feature tensor representation. The second tensor to be trained performs related operations with the decoder features in the transformer architecture of the decoder, guiding the decoder to output features according to the tensor dimension of the second tensor to be trained, that is, the student decoding features output by the decoder have the same dimension as the second tensor to be trained.

[0042] Step S14: Based on the student class prediction result output by the student model, perform uncertainty estimation to obtain an uncertainty weight.

[0043] Considering that in the unsupervised domain adaptation object detection task, the true labels are invisible, in this embodiment, the predicted uncertainty is used to measure the prediction quality. Specifically, uncertainty estimation is performed based on the student class prediction result output by the student model to obtain an uncertainty weight. Among them, the student class prediction result output by the student model is the result of inputting the student decoded feature into the classification head for prediction.

[0044] Step S15: Based on the teacher encoded feature, the teacher decoded feature, the student encoded feature, and the uncertainty weight, obtain a feature distillation loss.

[0045] In this embodiment, after obtaining the uncertainty weight, based on the teacher encoded feature output by the encoder of the teacher model, the teacher decoded feature output by the decoder of the teacher model, the student encoded feature output by the encoder of the student model, and the uncertainty weight, a feature distillation loss is calculated to achieve more robust knowledge distillation between the teacher model and the student model.

[0046] Step S16: Based on the pseudo label, the student class prediction result, the student bounding box prediction result output by the student model, the student decoded feature, and a memory bank of multiple categories, obtain a contrastive learning loss, where the pseudo label is determined according to the teacher class prediction result and the teacher bounding box prediction result output by the teacher model.

[0047] In this embodiment, a contrastive learning loss can be calculated based on the pseudo label output by the teacher network, the student class prediction result, the student bounding box prediction result output by the student model, the student decoded feature, and a memory bank of multiple categories. Among them, the pseudo label is determined according to the teacher class prediction result and the teacher bounding box prediction result output by the teacher model. The teacher class prediction result output by the teacher model is the result of inputting the teacher decoded feature into the classification head for prediction. The teacher bounding box prediction result output by the teacher model is the result of inputting the teacher decoded feature into the regression head for prediction. The student bounding box prediction result output by the student model is the result of inputting the student decoded feature into the regression head for prediction. Also, the memory bank of multiple categories in this embodiment is a category feature bank obtained based on the student decoded feature during the model training process. In this embodiment, this memory bank of multiple categories is used to obtain a large number of high-quality and as category-balanced as possible features for contrastive learning.

[0048] Step S17: Update at least the teacher model and the student model based on the feature distillation loss and the contrastive learning loss to obtain a trained student model.

[0049] In this embodiment, after obtaining the feature distillation loss and the contrastive learning loss, the model parameters of the teacher model and the student model are updated based on at least the feature distillation loss and the contrastive learning loss to obtain a trained student model, thereby realizing the application of the DETR detection model to domain adaptive object detection in a passive scenario.

[0050] Step S18: Input the image to be detected in the target domain into the trained student model to obtain an object detection result.

[0051] In this embodiment, after obtaining the trained student model, the trained student model is used for object detection in the target domain. The image to be detected in the target domain can be input into the trained student model to obtain the object detection result output by the trained student model.

[0052] In this embodiment, the teacher-student model framework is applied as the basic passive domain adaptation object detection framework. In the process of domain adaptation through unsupervised target domain data (i.e., target domain image samples), the more robust teacher model generates pseudo-labels and provides them to the student model for training, and the student model is updated through gradients. Among them, considering the characteristic that the DETR object detection framework tries to assign only one feature query to the same object and the problem of class imbalance in the real-world environment, in this embodiment, under the basic contrast loss, a multi-class memory bank is used to obtain a large number of high-quality and as class-balanced as possible features for contrastive learning. Thus, the contrastive learning loss is calculated through pseudo-labels, the student class prediction results, the student bounding box prediction results, the student decoded features, and the multi-class memory bank to learn higher-quality class features in the target domain. And considering the characteristics of the DETR object detection framework and the fact that the true labels are invisible in the passive domain adaptation object detection task, this embodiment uses the prediction uncertainty to measure the prediction quality, obtains the uncertainty weight, and then calculates the feature distillation loss based on the teacher encoded features, the teacher decoded features, the student encoded features, and the uncertainty weight to achieve more robust knowledge distillation between the teacher model and the student model. In this way, this embodiment updates the teacher model and the student model based on at least the contrastive learning loss of the matching class memory bank and the feature distillation loss based on uncertainty weighting to obtain a trained student model in the target domain for object detection of the image to be detected in the target domain, solves the problem that the DETR detection model cannot be applied to domain adaptive object detection in a passive scenario, improves the detection accuracy of the DETR object detection model in the passive domain adaptation scenario, and enables the DETR object detection model to have efficient and stable detection performance under various complex domain shifts in the real-world scenario.

[0053] Combined with the above embodiments, in one implementation manner, the present invention further provides a target detection method. In this method, in addition to the above steps, it may further include step S21, and specifically, the above step S17 may include step S22:

[0054] Step S21: Based on the student encoding feature, the student decoding feature, the student category prediction result, and the student bounding box prediction result, obtain a reweighted classification loss.

[0055] Considering that the contrast learning loss can only mitigate the impact of class imbalance on contrast learning to a certain extent, in order to better solve the class imbalance problem, a reweighted classification loss is designed in this embodiment: Based on the student encoding feature, the student decoding feature, the student category prediction result, and the student bounding box prediction result, calculate to obtain a reweighted classification loss.

[0056] Step S22: Update the teacher model and the student model at least based on the feature distillation loss, the contrast learning loss, and the reweighted classification loss to obtain the trained student model.

[0057] In this embodiment, when updating the model parameters, the model parameters of the teacher model and the student model are updated at least based on the feature distillation loss, the contrast learning loss, and the reweighted classification loss to obtain the trained student model.

[0058] Combined with the above embodiments, the present invention further provides a target detection method. In this method, in addition to the above steps, it may further include step S31 and step S32, and specifically, the above step S22 may include step S33:

[0059] Step S31: Based on the bounding box pseudo-label corresponding to the teacher bounding box prediction result and the student bounding box prediction result, obtain a bounding box loss.

[0060] In this embodiment, a bounding box loss can be calculated based on the bounding box pseudo-label corresponding to the teacher bounding box prediction result and the student bounding box prediction result.

[0061] Step S32: Based on the reweighted classification loss and the bounding box loss, obtain a target detection loss.

[0062] In this embodiment, based on the reweighted classification loss and the bounding box loss, obtain a target detection loss That is to say, in this embodiment, the classification loss part in the detection loss (including classification detection loss and bounding box loss) of the traditional DETR model is replaced with a reweighted classification loss. Then, the object detection loss in this embodiment includes the reweighted classification loss and the bounding box loss. Among them, the classification detection loss of the traditional detection loss is the loss between the class pseudo-label corresponding to the teacher class prediction result and the student class prediction result.

[0063] Step S33: Based on the feature distillation loss, the contrastive learning loss, and the object detection loss, update the teacher model and the student model to obtain the trained student model.

[0064] In this embodiment, the total loss is calculated based on the feature distillation loss, the contrastive learning loss, and the object detection loss. Then, based on the total loss, the model parameters of the teacher model and the student model are updated to obtain the trained student model.

[0065] Among them, in this embodiment, the object detection loss, the contrastive learning loss, and the feature distillation loss are linearly weighted as the total loss. The total loss is:

[0066] ;

[0067] Among them, is the object detection loss, is the contrastive learning loss, is the feature distillation loss, and are the weights corresponding to the contrastive loss and the feature distillation loss, respectively.

[0068] Combining the above embodiments, in one implementation, the present invention also provides an object detection method. In this method, step S14 above may specifically include step S41 and step S42:

[0069] Step S41: Calculate the prediction entropy value E based on the student class prediction result output by the student model.

[0070] In this embodiment, uncertainty estimation (Uncertaint Estimation) is performed based on the student class prediction result output by the student model (the feature corresponding to each prediction instance, that is, the prediction score of each student decoded feature), and the prediction entropy value E is calculated.

[0071] Step S42: Obtain the uncertainty weight based on the prediction entropy value E.

[0072] In this embodiment, after obtaining the predicted entropy value E, an uncertainty weight is calculated based on the predicted entropy value E, and a higher weight is assigned to the prediction with a lower entropy value (higher prediction certainty). In an optional example, the uncertainty weight is as shown in the following formula:

[0073] ;

[0074] where (E) represents (E - min(E)) / (max(E) - min(E)); is a hyperparameter for smoothing, and E is the predicted entropy value.

[0075] Combining the above embodiments, in one implementation manner, the present invention further provides an object detection method. In this method, the above step S15 may specifically include steps S51 to S52:

[0076] Step S51: Obtain a pixel-level mask based on the last-layer teacher encoding feature in the teacher encoding features and the teacher decoding feature.

[0077] In this embodiment, a pixel-level mask can be obtained based on the last-layer teacher encoding feature in the teacher encoding features (i.e., the global feature output by the last encoder layer of the teacher model) and the teacher decoding feature. Specifically, the last-layer teacher encoding feature in the teacher encoding features and the teacher decoding feature can be fused to obtain a pixel-level mask that reflects the objectivity of the target.

[0078] In an optional example, the formula for the pixel-level mask is:

[0079] ;

[0080] where is the last-layer teacher encoding feature; is the teacher decoding feature, which is a tensor of [n q , C], n q is the number of single queries, C is the dimension of a single query, j represents the jth element, that is, the tensor with a shape of [C] in the jth row; T is the transpose.

[0081] Step S52: Obtain the feature distillation loss based on the last-layer teacher encoding feature, the last-layer student encoding feature in the student encoding features, the pixel-level mask, and the uncertainty weight.

[0082] In this embodiment, after obtaining the pixel-level mask, the pixel-level mask can be applied to the pixel-level feature distillation loss, and the above-mentioned uncertainty weights are assigned to the pixel-level masks corresponding to the decoded features of different teachers to obtain the feature distillation loss. Specifically, based on the last-layer teacher encoded feature, the last-layer student encoded feature in the student encoded features (i.e., the global feature output by the last-layer encoder of the student model), the pixel-level mask, and the uncertainty weight, the feature distillation loss can be calculated.

[0083] In an optional example, the feature distillation loss is calculated by the formula:

[0084] ;

[0085] where the superscript j represents the j-th element, j traverses from 0 to -1, is the pixel-level mask, is the Hadamard product, and represent the last-layer teacher encoded feature and the last-layer student encoded feature respectively, represents the uncertainty weight, H, W, and C respectively represent the height, width, and number of channels of the last-layer encoder feature (the last-layer teacher encoded feature or the last-layer student encoded feature), is the number of decoded features for a sample in the DETR object detection model when performing object detection (i.e., the number of single queries in the object query (the number of rows of the tensor)).

[0086] Different from the method of mimicking prediction scores commonly used in traditional knowledge distillation, this embodiment uses a novel feature distillation method, which has been proven to be effective in object detection tasks. Specifically, first, an additional forward propagation is performed, and the strongly augmented samples are forward propagated in the teacher model, and this additional forward propagation is only used for knowledge distillation. Next, the fused instance features are weighted according to the prediction quality, and a feature distillation loss is calculated for optimization. Thus, compared with the related technologies that often do not consider the different contribution degrees of different samples when using the feature distillation method, or the contribution-degree-based weighting method used cannot be directly applied to the source-free challenge, the feature distillation loss of this embodiment can be applied to source-free domain adaptation.

[0087] Combining the above embodiments, in one implementation, the present invention further provides an object detection method. In this method, the above step S21 may specifically include steps S61 to S64:

[0088] ​​​​Step S61: Fuse the multi-layer student encoding features and the student decoding features in the student encoding features to obtain a fused feature.

[0089] In this embodiment, the multi-layer student encoding features (global features output by the multi-layer encoder of the student model) and the student decoding features in the student encoding features can be fused to obtain a fused feature (the fused global feature) to calculate the similarity between the target represented by the target query and the global feature.

[0090] In an optional example, the fused feature has the following formula:

[0091] ;

[0092] where is the number of decoding features when the DETR object detection model performs object detection on a sample (i.e., the number of single queries in the object query (the number of rows of the tensor)); is the output feature of the i-th layer encoder of the student model, and the value range of i is {3, 4, 5}; is the transpose of the student decoding feature, j represents the j-th element, and j traverses from 0 to -1.

[0093] Step S62: Based on the fused feature and the student bounding box prediction result, obtain a region feature.

[0094] In the DETR framework, the model will predict class information and localization information (bounding box) for each object query. Therefore, in this embodiment, after obtaining the fused feature, a region feature can be obtained based on the fused feature and the student bounding box prediction result. Specifically, the RoIAlign algorithm can be used for the fused feature and the student bounding box prediction result to extract the local features within the bounding box to obtain the region feature, that is, the region attention.

[0095] In an optional embodiment, the region feature is as follows:

[0096] ;

[0097] where is the fused feature; bbox is the student bounding box prediction result, ; RoIAlign() is the RoIAlign algorithm.

[0098] Step S63: Based on the region feature, obtain a query weight.

[0099] In this embodiment, after obtaining the region features, the query weights can be obtained based on the region features. In a specific example, after obtaining the region features, at this time, all target queries have the attention of their corresponding objects (targets) within the region. Therefore, in this embodiment, the attention scores within the region can be summed to represent the objective prediction score of the target query having the corresponding target within the region.

[0100] To better utilize the multi-scale encoder features on the Transformer architecture, in an alternative embodiment, a set of objectness scores (an estimation metric for having valid objects within the region) can be calculated respectively on the output features of the 3rd, 4th, and 5th layer encoders of the student model according to the following method, and the same target query is summed on the features of encoders at different levels. The formula is as follows:

[0101] ;

[0102] where, are the height and width output by the RoIAlign algorithm respectively; j represents the j-th element, and j traverses from 0 to -1; is the region feature; S represents the set of target-specific attentions (region features) obtained from the multi-scale encoder features corresponding to each target query.

[0103] After obtaining S, a scaling process is performed to obtain the query weights :

[0104] ;

[0105] where, is a hyperparameter for smoothing, , is the i-th element in the set S.

[0106] Step S64: Obtain the reweighted classification loss based on the query weights and the student category prediction results.

[0107] In this embodiment, after obtaining the query weights, the reweighted classification loss can be calculated based on the query weights and the student category prediction results .

[0108] In an alternative embodiment, considering the imbalance between foreground and background samples, when re-weighting the samples in this embodiment, it is natural to hope to assign high weights to the foreground regions and difficult samples with low attention. The former (foreground regions) can be addressed to some extent by the weights of the Focal loss, while the latter (difficult samples with low attention) can be solved by assigning higher weights to samples with lower target scores. Specifically, in this embodiment, based on the traditional Focal loss, each sample is further weighted, and the formula is as follows:

[0109] ;

[0110] where, and are the balance parameters of the Focal loss; is the student class prediction result; is the query weight; is the number of decoded features when the DETR object detection model performs object detection on a sample (i.e., the number of single queries in objectquery (the number of rows of the tensor)).

[0111] Combined with the above embodiments, in one embodiment, the present invention also provides an object detection method. In this method, the multi-category memory bank includes: a background memory bank and multiple foreground memory banks of different categories; the above step S16 may specifically include steps S71 to S74:

[0112] Step S71: Fuse the student decoded features of multiple scales in the student decoded features to obtain multi-scale decoded features.

[0113] In this embodiment, considering the rich semantics of the multi-scale decoder output features in the Transformer module of the DETR object detection model, the student decoded features of multiple different scales in the student decoded features are accumulated and fused to obtain multi-scale decoded features. In this embodiment, the decoder features of different scales are accumulated and fused (multi-scale decoded features) as the key for contrastive learning.

[0114] Step S72: When the student class prediction result and the student bounding box prediction result match successfully with the pseudo-label, add the first multi-scale decoded feature to the foreground memory bank corresponding to the pseudo-label category.

[0115] In this embodiment, the predicted student category result and the predicted student bounding box result are matched with the pseudo-labels (bounding box pseudo-labels and category pseudo-labels) output by the teacher model. When the predicted student category result and the predicted student bounding box result are successfully matched with the pseudo-labels, the category corresponding to the successfully matched pseudo-label is determined, and the first multi-scale decoded feature is added to the foreground memory bank corresponding to the category of the pseudo-label. Among them, the first multi-scale decoded feature is the multi-scale decoded feature corresponding to the successfully matched predicted student category result and the predicted student bounding box result.

[0116] In an alternative embodiment, the Hungarian algorithm can be used for bipartite graph matching to construct the minimum matching cost between the query feature (i.e., the multi-scale decoded feature) and each category (including the foreground and background categories) to obtain the corresponding category information, and the feature memory bank is constructed based on the category information of the multi-scale decoded feature.

[0117] Step S73: When the predicted student category result and the predicted student bounding box result fail to match the pseudo-labels, add the second multi-scale decoded feature to the background memory bank.

[0118] In this embodiment, when the predicted student category result and the predicted student bounding box result fail to match the pseudo-labels, add the second multi-scale decoded feature to the background memory bank. Among them, the second multi-scale decoded feature is the multi-scale decoded feature corresponding to the predicted student category result and the predicted student bounding box result that fail to match.

[0119] In this embodiment, a memory bank is constructed for each category (including the background category), and all memory banks have the same maximum feature capacity and are updated in a first-in-first-out manner to maintain the timeliness of category features.

[0120] Step S74: Based on the features in the memory bank where the first multi-scale decoded feature is located and the features in the memory banks of the multiple categories, perform contrastive learning to obtain the contrastive learning loss.

[0121] In this embodiment, the contrastive learning loss can be calculated based on the memory banks of multiple categories. Specifically, based on the features in the memory bank where the first multi-scale decoded feature is located and the features in the memory banks of multiple categories, perform contrastive learning to obtain the contrastive learning loss. Among them, the memory bank where the first multi-scale decoded feature is located can be understood as the foreground memory bank corresponding to the successfully matched multi-scale decoded feature.

[0122] In an alternative embodiment, the contrastive learning loss is as follows:

[0123] ;

[0124] Among them, is the number of foreground class categories, is the current foreground class category serial number, is the foreground memory bank of the current foreground class, is the union set of the memory banks of all categories (including background and foreground), is the query vector in the current category memory bank (i.e., the multi-scale decoding features in the current category memory bank), is the key vector of the same category as the query vector in the current category memory bank, is the temperature coefficient, , are all the multi-scale decoding features in the memory banks of all categories, exp represents the exponential function, exponential function.

[0125] Combined with the above embodiments, in one implementation manner, the present invention further provides an object detection method. In this method, the "updating the teacher model and the student model" in the above step S17 may specifically include steps S81 to S83:

[0126] Step S81: Update the student model.

[0127] In this embodiment, after obtaining the total loss (such as the feature distillation loss and the contrast learning loss in the foregoing embodiments; the feature distillation loss, the contrast learning loss, and the reweighted classification loss; or, the feature distillation loss, the contrast learning loss, and the object detection loss), the model parameters of the student model can be updated based on the total loss. For example, the parameters of the student model are updated based on the stochastic gradient descent sgd algorithm, etc., to obtain the trained student model.

[0128] Step S82: Determine the exponential moving average update interval based on the base update interval and the current update round.

[0129] In this embodiment, after the student model is updated, the propagation of the gradient to the teacher model is cut off, and the exponential moving average of the student model parameters and the teacher model parameters is used to update the teacher model. Specifically, the dynamic exponential moving average update interval can be determined first based on the preset or default base update interval and the current update round speed (epoch) of the model.

[0130] In an alternative implementation manner, the formula for the exponential moving average update interval is as follows:

[0131] ;

[0132] Among them, is the calculated exponential moving average update interval, δ is the base update interval, e is the current update round (epoch) of the model, is a smoothing parameter.

[0133] Step S83: When the number of updates of the student model reaches the exponential moving average update interval, update the teacher model.

[0134] In this embodiment, the exponential moving average update interval can be understood as the concept of the number of iterations, that is, how many batches (the number of updates of the student model) the teacher model is updated once. That is to say, when the number of updates of the student model reaches the exponential moving average update interval, the teacher model is updated.

[0135] Different from most other methods that set the iteration interval of exponential moving average update to iteration number = 1, this dynamically increasing exponential moving average update interval in this embodiment is beneficial for the model to explore the parameter space more frequently in the initial stage of domain adaptation training and gradually improve the stability of parameter adjustment as the training progresses.

[0136] In one embodiment, as Figure 2 shown, Figure 2 is a schematic diagram of a DETR-based unsupervised domain adaptation object detection framework shown in an embodiment of the present invention. In Figure 2 it, the DETR-based unsupervised domain adaptation object detection framework is a feature reweighting and contrast learning network, at least including a contrast learning module based on a matching memory bank, a sample reweighting module based on object scores, and a query fusion feature distillation module based on uncertainty weighting.

[0137] Among them, the target domain image samples are weakly augmented and then input into the pre-trained teacher model of the source domain (that is, the Figure 2 teacher network), and the teacher encoded features output by the encoder of the teacher model are obtained. The teacher encoded features and the first tensor to be trained are input into the decoder of the teacher model, and the teacher decoded features output by the decoder are obtained. The teacher decoded features are respectively predicted by the regression head and the classification head to obtain pseudo-labels.

[0138] The target domain image samples are strongly augmented and then input into the student model to be trained in the target domain (that is, the Figure 2 student network), and the student encoded features output by the encoder of the student model are obtained. The student encoded features and the second tensor to be trained are input into the decoder of the student model, and the student decoded features output by the decoder are obtained. The student decoded features are respectively predicted by the regression head and the classification head to obtain the student bounding box prediction result and the student class prediction result.

[0139] Based on this, the student decoded features and the pseudo-labels are input into the contrast learning module based on the matching memory bank (that is, the Figure 2In the CMMB), in this module, the student decoding features at multiple scales in the student decoding features are accumulated and fused to obtain multi-scale decoding features (i.e., Figure 2 the multi-scale query features in), then the pseudo-labels are matched with the student class prediction results and the student bounding box prediction results to obtain the matching results. The multi-scale decoding features corresponding to the successfully matched student class prediction results and the student bounding box prediction results are added to the foreground memory bank corresponding to the pseudo-labels, and the multi-scale decoding features corresponding to the failed-matched student class prediction results and the student bounding box prediction results are added to the background memory bank. Finally, contrastive learning is performed based on the memory banks of multiple categories to obtain the contrastive learning loss .

[0140] The teacher encoding features, teacher decoding features, student encoding features, and student class prediction results are input into the query fusion feature distillation module based on uncertainty weighting (i.e., Figure 2 the UQFD in). In this module, first, based on the student class prediction results output by the student model, uncertainty estimation is performed to obtain the uncertainty weights; then, based on the last-layer teacher encoding features and teacher decoding features in the teacher encoding features, a pixel-level mask is obtained; finally, based on the last-layer teacher encoding features, the last-layer student encoding features in the student encoding features, the pixel-level mask, and the uncertainty weights, distillation learning is performed to obtain the feature distillation loss .

[0141] The student encoding features, student decoding features, student class prediction results, and student bounding box prediction results are input into the sample reweighting module based on the target score. In this module, first, the multi-layer student encoding features and student decoding features in the student encoding features are fused to obtain the fused features (i.e., Figure 2 the query fusion features in); then, based on the fused features and the student bounding box prediction results, RoI Align calculation is performed to obtain the region features (i.e., Figure 2 the region attention in); then, weighting and scaling are performed based on the region features to obtain the target query weights; finally, based on the target query weights and the student class prediction results (i.e., Figure 2 the class prediction in), the reweighted classification loss is calculated .

[0142] Finally, based on the contrastive learning loss , the feature distillation loss , and the reweighted classification loss the parameters of the student model to be trained in the target domain are updated, that is, the model and each component are learned and updated in the teacher-student framework. In this embodiment, a dynamic teacher update interval in the teacher-student framework is further proposed, that is, the teacher model is updated by the exponential moving average of the student model parameters (i.e.,Figure 2 The DTUI in [0]: Based on the base update interval and the current update round, determine the exponential moving average update interval. When the number of updates to the student model reaches the exponential moving average update interval, update the parameters of the teacher model. Moreover, the model designed in this embodiment can be deployed in the NVIDIA GeForce RTX4090 GPU environment and perform passive object detection under various domain shift conditions in the real world.

[0143] The purpose of this embodiment is to develop a passive domain adaptation object detection framework based on the DETR detection model and solve the corresponding challenges, enabling the advanced DETR object detector to efficiently handle the domain shift challenges and passive challenges brought by privacy protection issues in the real world environment. Specifically, based on the DETR object detector, this embodiment includes three components: a contrastive learning module based on a matching memory bank, a sample reweighting module based on object scores, and a query fusion feature distillation module based on uncertainty weighting, as well as a dynamic teacher update interval for model parameter update in the teacher-student framework. After testing, on a variety of internationally common passive cross-domain object detection benchmarks, after the model undergoes domain adaptation using the proposed algorithm and is compared with a variety of advanced baseline methods, it achieves the state-of-the-art detection performance and can effectively improve the generalization performance of the DETR object detector in the real world environment.

[0144] Among them, the contrastive learning module based on a matching memory bank in this embodiment, compared with the related technology that conducts contrastive learning based on instance features directly obtained from the feature extractor, proposes to use the category memory bank features of multi-level feature fusion and uses bipartite graph matching to match category labels for the memory bank features. The sample reweighting module based on object scores in this embodiment, compared with the related technology that reweights samples by predicting confidence quality or intersection over union quality, can be used under passive challenges. Moreover, the query fusion feature distillation module based on uncertainty weighting in this embodiment, compared with the related technology that does not consider the different contribution degrees of different samples when using the feature distillation method or uses a contribution degree-based weighting method, can be directly applied to passive challenges.

[0145] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequences because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0146] Based on the same inventive concept, an embodiment of the present invention provides an object detection device. Refer to Figure 3 , Figure 3 which is a structural block diagram of an object detection device provided by an embodiment of the present invention. As Figure 3 shown, the object detection device of this embodiment may include:

[0147] An image processing module, configured to perform weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and perform strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample;

[0148] A first input module, configured to input the weakly enhanced sample into a pre-trained teacher model in the source domain to obtain a teacher encoding feature output by the encoder of the teacher model and a teacher decoding feature output by the decoder of the teacher model;

[0149] A second input module, configured to input the strongly enhanced sample into a student model to be trained in the target domain to obtain a student encoding feature output by the encoder of the student model and a student decoding feature output by the decoder of the student model;

[0150] A first weight determination module, configured to perform uncertainty estimation based on the student class prediction result output by the student model to obtain an uncertainty weight;

[0151] A first loss determination module, configured to obtain a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight;

[0152] A second loss determination module, configured to obtain a contrastive learning loss based on the pseudo-label, the student class prediction result, the student bounding box prediction result output by the student model, the student decoding feature, and a memory bank of multiple categories, where the pseudo-label is determined according to the teacher class prediction result and the teacher bounding box prediction result output by the teacher model;

[0153] A model training module, configured to update at least the teacher model and the student model based on the feature distillation loss and the contrastive learning loss to obtain a trained student model;

[0154] An object detection module, configured to input an image to be detected in the target domain into the trained student model to obtain an object detection result.

[0155] Optionally, the device further includes:

[0156] A third loss determination module, configured to obtain a reweighted classification loss based on the student encoding feature, the student decoding feature, the student class prediction result, and the student bounding box prediction result;

[0157] The model training module includes:

[0158] The first training module is used to update the teacher model and the student model at least based on the feature distillation loss, the contrastive learning loss, and the reweighted classification loss, so as to obtain the trained student model.

[0159] Optionally, the device further includes:

[0160] The fourth loss determination module is used to obtain a bounding box loss based on the bounding box pseudo-label corresponding to the teacher bounding box prediction result and the student bounding box prediction result;

[0161] The fifth loss determination module is used to obtain an object detection loss based on the reweighted classification loss and the bounding box loss;

[0162] The first training module includes:

[0163] The second training module is used to update the teacher model and the student model based on the feature distillation loss, the contrastive learning loss, and the object detection loss, so as to obtain the trained student model.

[0164] Optionally, the first weight determination module includes:

[0165] The entropy value determination module is used to calculate a prediction entropy value E based on the student class prediction result output by the student model;

[0166] The second weight determination module is used to obtain the uncertainty weight based on the prediction entropy value E.

[0167] Optionally, the first loss determination module includes:

[0168] The first determination module is used to obtain a pixel-level mask based on the last-layer teacher encoded feature in the teacher encoded features and the teacher decoded feature;

[0169] The second determination module is used to obtain the feature distillation loss based on the last-layer teacher encoded feature, the last-layer student encoded feature in the student encoded features, the pixel-level mask, and the uncertainty weight.

[0170] Optionally, the third loss determination module includes:

[0171] The first fusion module is used to fuse multiple-layer student encoded features in the student encoded features and the student decoded feature to obtain a fused feature;

[0172] A third determination module, configured to obtain region features based on the fusion features and the student bounding box prediction results;

[0173] A fourth determination module, configured to obtain query weights based on the region features;

[0174] A fifth determination module, configured to obtain the reweighted classification loss based on the query weights and the student category prediction results.

[0175] Optionally, the memory bank of multiple categories includes: a background memory bank and multiple foreground memory banks of different categories; the second loss determination module includes:

[0176] A second fusion module, configured to fuse the student decoded features at multiple scales in the student decoded features to obtain multi-scale decoded features;

[0177] A first addition module, configured to add the first multi-scale decoded features to the foreground memory bank corresponding to the pseudo-label when the student category prediction results and the student bounding box prediction results match the pseudo-label successfully, where the first multi-scale decoded features are the multi-scale decoded features corresponding to the successfully matched student category prediction results and student bounding box prediction results;

[0178] A second addition module, configured to add the second multi-scale decoded features to the background memory bank when the student category prediction results and the student bounding box prediction results fail to match the pseudo-label, where the second multi-scale decoded features are the multi-scale decoded features corresponding to the failed-matched student category prediction results and student bounding box prediction results;

[0179] A sixth determination module, configured to perform contrastive learning based on the features in the memory bank where the first multi-scale decoded features are located and the features in the memory bank of multiple categories to obtain the contrastive learning loss.

[0180] Optionally, the model training module, the first training module or the second training module includes:

[0181] A first update module, configured to update the student model;

[0182] A seventh determination module, configured to determine an exponential moving average update interval based on a base update interval and a current update round;

[0183] A second update module, configured to update the teacher model when the number of times the student model is updated reaches the exponential moving average update interval.

[0184] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, as Figure 4 shown.Figure 4 It is a schematic diagram of an electronic device shown in an embodiment of the present invention. The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes, it implements the steps in the object detection method described in any one of the above embodiments of the present invention.

[0185] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment.

[0186] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, refer to each other.

[0187] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a device, or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0188] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0189] These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing terminal devices to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide for implementing the process Figure 1 in one process or multiple processes and / or blocks Figure 1 or steps for the functions specified in one block or multiple blocks.

[0191] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0192] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the element.

[0193] The above has introduced in detail a target detection method, device and equipment provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A target detection method, characterized in that: The method comprises: Performing weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and performing strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample; Inputting the weakly enhanced sample into a pre-trained teacher model of the source domain, obtaining a teacher encoding feature output by an encoder of the teacher model and a teacher decoding feature output by a decoder of the teacher model; Inputting the strongly enhanced sample into a student model to be trained in the target domain, obtaining a student encoding feature output by an encoder of the student model and a student decoding feature output by a decoder of the student model; Based on the student category prediction results output by the student model, uncertainty estimation is performed to obtain uncertainty weights; Obtaining a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight; Obtaining contrastive learning loss based on pseudo labels, the student category prediction results, the student bounding box prediction results output by the student model, the student decoding features, and multiple categories of memory banks, wherein the pseudo labels are determined based on the teacher category prediction results and the teacher bounding box prediction results output by the teacher model; At least based on the feature distillation loss and the contrastive learning loss, the teacher model and the student model are updated to obtain a trained student model; The image to be detected in the target domain is input into the trained student model to obtain the target detection result.

2. The target detection method according to claim 1, characterized in that: The method further comprises: Obtaining a reweighted classification loss based on the student encoding feature, the student decoding feature, the student category prediction result, and the student bounding box prediction result; At least based on the feature distillation loss and the contrastive learning loss, the teacher model and the student model are updated to obtain a trained student model, including: At least based on the feature distillation loss, the contrastive learning loss and the reweighted classification loss, the teacher model and the student model are updated to obtain the trained student model.

3. The target detection method according to claim 2, characterized in that: The method further comprises: Obtaining a bounding box loss based on the bounding box pseudo-label corresponding to the teacher bounding box prediction result and the student bounding box prediction result; Obtaining an object detection loss based on the reweighted classification loss and the bounding box loss; At least based on the feature distillation loss, the contrastive learning loss, and the reweighted classification loss, the teacher model and the student model are updated to obtain the trained student model, including: Based on the feature distillation loss, the contrastive learning loss and the target detection loss, the teacher model and the student model are updated to obtain the trained student model.

4. The target detection method according to claim 1, characterized in that: Based on the student category prediction results output by the student model, uncertainty estimation is performed to obtain uncertainty weights, including: Calculate the predicted entropy value E based on the student category prediction result output by the student model; Based on the predicted entropy value E, the uncertainty weight is obtained.

5. The target detection method according to claim 1, characterized in that: Based on the teacher encoding feature, the teacher decoding feature, the student encoding feature and the uncertainty weight, a feature distillation loss is obtained, including: Based on the last layer of teacher encoding features in the teacher encoding features and the teacher decoding features, a pixel-level mask is obtained; The feature distillation loss is obtained based on the last layer of teacher encoding features, the last layer of student encoding features in the student encoding features, the pixel level mask and the uncertainty weight.

6. The target detection method according to claim 2, characterized in that: Based on the student encoding feature, the student decoding feature, the student category prediction result, and the student bounding box prediction result, a reweighted classification loss is obtained, including: Fusing the multiple layers of student encoding features and the student decoding features in the student encoding features to obtain fused features; Obtaining regional features based on the fusion features and the student bounding box prediction results; Based on the regional features, a query weight is obtained; Based on the query weight and the student category prediction result, the reweighted classification loss is obtained.

7. The target detection method according to claim 1, characterized in that: The multiple categories of memory libraries include: a background memory library and multiple different categories of foreground memory libraries; based on the pseudo-label, the student category prediction result, the student bounding box prediction result output by the student model, the student decoding feature, and the multiple categories of memory libraries, a contrastive learning loss is obtained, including: fusing the student decoding features at multiple scales in the student decoding features to obtain multi-scale decoding features; When the student category prediction result and the student bounding box prediction result successfully match the pseudo label, adding a first multi-scale decoding feature to the foreground memory bank of the category corresponding to the pseudo label, wherein the first multi-scale decoding feature is a multi-scale decoding feature corresponding to the successfully matched student category prediction result and the student bounding box prediction result; In the case where the student category prediction result and the student bounding box prediction result fail to match the pseudo label, adding a second multi-scale decoding feature to the background memory library, where the second multi-scale decoding feature is a multi-scale decoding feature corresponding to the student category prediction result and the student bounding box prediction result that failed to match; Based on the features in the memory bank where the first multi-scale decoding features are located and the features in the memory banks of the multiple categories, contrastive learning is performed to obtain the contrastive learning loss.

8. The target detection method according to any one of claims 1 to 7, characterized in that: The teacher model and the student model are updated, including: updating the student model; Based on the basic update interval and the current update round number, determine the exponential moving average update interval; When the number of updates of the student model reaches the exponential moving average update interval, the teacher model is updated.

9. A target detection device, characterized in that: The device comprises: An image processing module, used for performing weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and performing strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample; A first input module is used to input the weakly enhanced sample into a pre-trained teacher model of the source domain to obtain a teacher encoding feature output by the encoder of the teacher model and a teacher decoding feature output by the decoder of the teacher model; A second input module is used to input the strongly enhanced sample into a student model to be trained in the target domain to obtain a student encoding feature output by an encoder of the student model and a student decoding feature output by a decoder of the student model; A first weight determination module, configured to perform uncertainty estimation based on the student category prediction result output by the student model to obtain an uncertainty weight; A first loss determination module, configured to obtain a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight; a second loss determination module, configured to obtain a contrastive learning loss based on a pseudo label, the student category prediction result, the student bounding box prediction result output by the student model, the student decoding feature, and a plurality of categories of memory banks, wherein the pseudo label is determined according to the teacher category prediction result and the teacher bounding box prediction result output by the teacher model; A model training module, used for updating the teacher model and the student model based on at least the feature distillation loss and the contrastive learning loss to obtain a trained student model; The target detection module is used to input the image to be detected in the target domain into the trained student model to obtain the target detection result.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the target detection method according to any one of claims 1 to 8 is implemented.

Images