Target detection method, device and equipment

Abstract

The invention provides a target detection method, device and equipment, and relates to the field of self-adaption. Comprising the steps of performing different processing on a target domain image sample, and then respectively inputting a pre-trained teacher model of a source domain and a to-be-trained student model of the target domain to obtain a teacher coding feature, a teacher decoding feature, a student coding feature and a student decoding feature; obtaining an uncertainty weight based on a student category prediction result; obtaining feature distillation loss based on teacher coding features, teacher decoding features, student coding features and uncertainty weights; obtaining comparison learning loss based on the pseudo tag, the student category prediction result, the student bounding box prediction result, the student decoding feature and the memory bank of multiple categories; and carrying out model updating on the basis of the feature distillation loss and the contrast learning loss to obtain a trained student model so as to carry out target detection on the to-be-detected image of the target domain, thereby realizing application of the DETR detection model to field adaptive target detection in a passive scene.

Description

Technical Field

[0001] The present invention relates to the field of domain adaptation, and in particular to a target detection method, device and equipment. Background Art

[0002] In recent years, with the rapid development of deep neural networks, computer vision technology has also experienced rapid development, and its application efficiency and performance in real-world environments have been significantly improved. Among them, object detection, as an important task in computer vision, has received widespread attention and research, and has produced efficient deep object detection models such as Faster R-CNN, YOLO, and DETR. However, although these deep object detectors have achieved very good results on some data sets and benchmarks, they often require large-scale training data sets with high annotation quality to achieve satisfactory performance. In real-world environments, the collection and annotation of these data often require a lot of manpower and financial resources, which hinders the widespread application of these object detectors in real-world environments. In addition, when applied in diverse real-world environments, these deep object detectors often experience performance degradation, and one of the important reasons is called domain shift. Domain shift refers to the fact that there are certain differences in the feature distribution of the source domain and the target domain, such as lighting, texture, and shape in natural images. These domain gaps will cause the model pre-trained on the source domain data to be unable to be efficiently applied to the target domain data to achieve satisfactory performance. To address this challenge, a lot of research has been conducted on domain adaptive object detection, and relatively successful progress has been made in this field: through methods and techniques such as domain difference terms, adversarial discriminators, contrastive feature learning, and robust learning, domain difference adaptive object detection between domains can effectively bridge the gap between the source domain and the target domain, thereby achieving the effect of migrating the model from the source domain to the target domain application.

[0003] Although domain adaptive object detection can alleviate the performance degradation caused by domain shift to a certain extent, the application and effectiveness of these traditional domain adaptive object detection methods are still limited in more real-world application scenarios. For example, most current methods use adversarial alignment or intermediate domain generation methods for adaptation, but these methods rely on labeled source domain data. Obviously, when the source domain data cannot be directly obtained due to privacy protection and other reasons, these methods cannot properly exert their migration performance. This scenario is called passive domain adaptive object detection, that is, during the domain adaptation process, the source domain data and the distribution itself are invisible, and only the pre-trained model containing the source domain information participates in domain adaptation.

[0004] Fortunately, in response to the challenges of domain adaptive object detection in the above-mentioned restricted scenarios, some efficient solutions have been developed on specific architectures, such as semi-supervised style transfer through the mean teacher framework domain pseudo-label method. However, the design of related passive domain adaptive object detection frameworks often relies on traditional detection models and their specific components, such as Faster R-CNN and Region Proposal Network (RPN), but cannot be applied normally and efficiently in some novel and efficient frameworks, such as the DETR (Detection Transformer) detection model. Therefore, how to realize the application of the DETR detection model to domain adaptive object detection in passive scenarios is a technical problem to be solved by the present invention. Summary of the invention

[0005] Based on the above technical problems, the present invention provides a target detection method, device and equipment, aiming to realize the application of the DETR detection model to domain adaptive target detection in passive scenarios, and improve the detection accuracy of the DETR target detection model in passive domain adaptive scenarios.

[0006] A first aspect of the present invention provides a target detection method, the method comprising: Performing weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and performing strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample; Inputting the weakly enhanced sample into a pre-trained teacher model of the source domain, obtaining a teacher encoding feature output by an encoder of the teacher model and a teacher decoding feature output by a decoder of the teacher model; Inputting the strongly enhanced sample into a student model to be trained in the target domain, obtaining a student encoding feature output by an encoder of the student model and a student decoding feature output by a decoder of the student model; Based on the student category prediction results output by the student model, uncertainty estimation is performed to obtain uncertainty weights; Obtaining a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight; Obtaining contrastive learning loss based on pseudo labels, the student category prediction results, the student bounding box prediction results output by the student model, the student decoding features, and multiple categories of memory banks, wherein the pseudo labels are determined based on the teacher category prediction results and the teacher bounding box prediction results output by the teacher model; At least based on the feature distillation loss and the contrastive learning loss, the teacher model and the student model are updated to obtain a trained student model; The image to be detected in the target domain is input into the trained student model to obtain the target detection result.

[0007] A second aspect of the present invention provides a target detection device, the device comprising: An image processing module, used for performing weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and performing strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample; A first input module is used to input the weakly enhanced sample into a pre-trained teacher model of the source domain to obtain a teacher encoding feature output by the encoder of the teacher model and a teacher decoding feature output by the decoder of the teacher model; A second input module is used to input the strongly enhanced sample into a student model to be trained in the target domain to obtain a student encoding feature output by an encoder of the student model and a student decoding feature output by a decoder of the student model; A first weight determination module, configured to perform uncertainty estimation based on the student category prediction result output by the student model to obtain an uncertainty weight; A first loss determination module, configured to obtain a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight; a second loss determination module, configured to obtain a contrastive learning loss based on a pseudo label, the student category prediction result, the student bounding box prediction result output by the student model, the student decoding feature, and a plurality of categories of memory banks, wherein the pseudo label is determined according to the teacher category prediction result and the teacher bounding box prediction result output by the teacher model; A model training module, used for updating the teacher model and the student model based on at least the feature distillation loss and the contrastive learning loss to obtain a trained student model; The target detection module is used to input the image to be detected in the target domain into the trained student model to obtain the target detection result.

[0008] A third aspect of an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the target detection method of the first aspect of the embodiment of the present invention is implemented.

[0009] In the target detection method provided by the embodiment of the present invention, first, weak enhancement processing and strong enhancement processing are performed on the target domain image samples respectively to obtain weak enhancement samples and strong enhancement samples; secondly, the weak enhancement samples are input into the pre-trained teacher model of the source domain to obtain the teacher encoding features output by the encoder of the teacher model and the teacher decoding features output by the decoder of the teacher model, and the strongly enhanced samples are input into the student model to be trained in the target domain to obtain the student encoding features output by the encoder of the student model and the student decoding features output by the decoder of the student model; then, uncertainty estimation is performed based on the student category prediction result output by the student model to obtain the uncertainty weights. heavy; then based on the teacher encoding features, the teacher decoding features, the student encoding features and the uncertainty weights, the feature distillation loss is calculated; then, based on the teacher category prediction results output by the teacher model and the pseudo labels determined by the teacher bounding box prediction results, the student category prediction results, the student bounding box prediction results output by the student model, the student decoding features, and the memory banks of multiple categories, the contrastive learning loss is calculated; finally, based on at least the feature distillation loss and the contrastive learning loss, the teacher model and the student model are updated to obtain a trained student model; the trained student model is used to perform target detection on the image to be detected in the target domain to obtain a target detection result.

[0010] Through the target detection method of this embodiment, a passive domain adaptation target detection framework is applied as the basis of the teacher-student model framework. In the process of domain adaptation through unsupervised target domain data (i.e., target domain image samples), a more robust teacher model generates pseudo labels and provides them to the student model for training, and the student model is updated through gradients. Considering the characteristics of the DETR target detection framework that only one feature query is assigned to the same target as much as possible and the problem of category imbalance in the real-world environment, this embodiment uses a variety of categories of memory libraries under basic contrast loss to obtain a large number of high-quality features that are as balanced as possible for contrastive learning, thereby calculating the contrastive learning loss through pseudo labels, student category prediction results, student bounding box prediction results, student decoding features, and a variety of categories of memory libraries to learn higher quality category features in the target domain. Considering the characteristics of the DETR target detection framework and the problem of category imbalance in the real-world environment, In the source domain adaptation target detection task, the true label is invisible. This embodiment uses the uncertainty of the prediction to measure the prediction quality, obtains the uncertainty weight, and then calculates the feature distillation loss based on the teacher encoding features, the teacher decoding features, the student encoding features and the uncertainty weight to achieve more robust knowledge distillation between the teacher model and the student model; in this way, this embodiment updates the teacher model and the student model at least based on the contrastive learning loss of the matching category memory library and the feature distillation loss based on uncertainty weighting to obtain a trained student model of the target domain to perform target detection on the image to be detected in the target domain, which solves the problem that the DETR detection model cannot be applied to domain adaptive target detection in passive scenarios, improves the detection accuracy of the DETR target detection model in passive domain adaptive scenarios, and enables the DETR target detection model to have efficient and stable detection performance under various complex domain offsets in real-world scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0012] Figure 1 is a flowchart of a target detection method according to an embodiment of the present invention; Figure 2 is a schematic diagram of a passive domain adaptive target detection framework based on DETR according to an embodiment of the present invention; Figure 3 is a structural block diagram of a target detection device provided by an embodiment of the present invention; Figure 4It is a schematic diagram of an electronic device shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] Please refer to Figure 1 , Figure 1 FIG. 1 is a flowchart of a target detection method according to an embodiment of the present invention. Figure 1 As shown, the target detection method provided in this embodiment includes at least the following steps: Step S11: performing weak enhancement processing on the target domain image samples to obtain weak enhancement samples, and performing strong enhancement processing on the target domain image samples to obtain strong enhancement samples.

[0015] In this embodiment, weak enhancement processing and strong enhancement processing can be performed on the same target domain image sample, respectively, to obtain weak enhancement samples and strong enhancement samples. Among them, the target domain image sample is an image sample in the target domain used for model training, and the target domain is the domain to be migrated. Strong enhancement processing refers to a large degree of transformation of the target domain image sample, which usually changes some basic features of the target domain image sample. Strong enhancement processing includes at least one or more of the following: rotation, flipping, shearing, adding noise, mixing different data, so as to increase the diversity of training data without changing the image category, thereby improving the robustness of the model. Weak enhancement processing refers to a small degree of transformation of the target domain image sample, which usually does not change the basic features of the image. Weak enhancement processing includes at least one or more of the following: scaling, translation, changing the color space, so as to make smaller perturbations to the image so that the model can learn more local features, thereby improving model performance.

[0016] Step S12: input the weakly enhanced sample into a pre-trained teacher model of the source domain to obtain teacher encoding features output by the encoder of the teacher model and teacher decoding features output by the decoder of the teacher model.

[0017] In this embodiment, the pre-trained teacher model of the source domain can be a trained target detection model of the source domain, such as a trained DETR target detection model of the source domain, and the pre-trained teacher model of the source domain includes at least: an encoder and a decoder. After obtaining the weakly enhanced sample, the weakly enhanced sample can be input into the pre-trained teacher model of the source domain to obtain the teacher encoding features output by the encoder of the teacher model and the teacher decoding features output by the decoder of the teacher model.

[0018] In an optional implementation, after obtaining the teacher coding features, the teacher coding features are input into the decoder of the teacher model together with the first tensor to be trained to obtain the teacher decoding features output by the decoder of the teacher model. The first tensor to be trained is the initial object query in the teacher model, which can be understood as the initial feature tensor representation. The first tensor to be trained performs related operations with the decoder features in the transformer architecture of the decoder to guide the decoder to output features according to the tensor dimension of the first tensor to be trained, that is, the teacher decoding features output by the decoder have the same dimension as the first tensor to be trained.

[0019] Step S13: input the strongly enhanced sample into the student model to be trained in the target domain to obtain the student encoding features output by the encoder of the student model and the student decoding features output by the decoder of the student model.

[0020] In this embodiment, the student model to be trained in the target domain can be a target detection model to be trained in the target domain, such as a DETR target detection model to be trained in the target domain, and the student model to be trained in the target domain also includes at least: an encoder and a decoder, and the student model to be trained in the target domain is initialized using the model parameters of the pre-trained teacher model in the source domain. After obtaining the strongly enhanced sample, the strongly enhanced sample can be input into the student model to be trained in the target domain to obtain the student encoding features output by the encoder of the student model and the student decoding features output by the decoder of the student model.

[0021] In an optional implementation, after obtaining the student coding features, the student coding features are input into the decoder of the student model together with the second tensor to be trained to obtain the student decoding features output by the decoder of the student model. The second tensor to be trained is the initial object query in the student model, which can be understood as the initial feature tensor representation. The second tensor to be trained performs related operations with the decoder features in the transformer architecture of the decoder to guide the decoder to output features according to the tensor dimension of the second tensor to be trained, that is, the student decoding features output by the decoder are the same as the dimension of the second tensor to be trained.

[0022] Step S14: Based on the student category prediction result output by the student model, uncertainty estimation is performed to obtain uncertainty weights.

[0023] Considering that the true label is invisible in the passive domain adaptation object detection task, this embodiment uses the uncertainty of prediction to measure the prediction quality. Specifically, the uncertainty is estimated based on the student category prediction result output by the student model to obtain the uncertainty weight. The student category prediction result output by the student model is the result of inputting the student decoding feature into the classification head for prediction.

[0024] Step S15: Obtain feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature and the uncertainty weight.

[0025] In this embodiment, after obtaining the uncertainty weight, the feature distillation loss is calculated based on the teacher encoding features output by the encoder of the teacher model, the teacher decoding features output by the decoder of the teacher model, the student encoding features output by the encoder of the student model, and the uncertainty weight to achieve more robust knowledge distillation between the teacher model and the student model.

[0026] Step S16: Based on the pseudo-label, the student category prediction result, the student bounding box prediction result output by the student model, the student decoding feature, and a memory library of multiple categories, a contrastive learning loss is obtained, wherein the pseudo-label is determined according to the teacher category prediction result and the teacher bounding box prediction result output by the teacher model.

[0027] In this embodiment, the contrastive learning loss can be calculated based on the pseudo-labels output by the teacher network, the student category prediction results, the student bounding box prediction results output by the student model, the student decoding features, and the memory libraries of various categories. Among them, the pseudo-labels are determined based on the teacher category prediction results and the teacher bounding box prediction results output by the teacher model. The teacher category prediction results output by the teacher model are the result of inputting the teacher decoding features into the classification head for prediction, the teacher bounding box prediction results output by the teacher model are the result of inputting the teacher decoding features into the regression head for prediction, and the student bounding box prediction results output by the student model are the result of inputting the student decoding features into the regression head for prediction. In addition, the memory libraries of various categories in this embodiment are category feature libraries obtained based on the student decoding features during the model training process. This embodiment uses the memory libraries of various categories to obtain a large number of high-quality features that are as balanced as possible for contrastive learning.

[0028] Step S17: updating the teacher model and the student model based at least on the feature distillation loss and the contrastive learning loss to obtain a trained student model.

[0029] In this embodiment, after obtaining the feature distillation loss and the contrastive learning loss, the model parameters of the teacher model and the student model are updated at least based on the feature distillation loss and the contrastive learning loss to obtain a trained student model, thereby realizing the application of the DETR detection model to domain adaptive target detection in passive scenarios.

[0030] Step S18: input the image to be detected in the target domain into the trained student model to obtain the target detection result.

[0031] In this embodiment, after obtaining the trained student model, the trained student model is used to perform target detection in the target domain. The image to be detected in the target domain can be input into the trained student model to obtain the target detection result output by the trained student model.

[0032] In this embodiment, a passive domain adaptation target detection framework is based on a teacher-student model framework. In the process of domain adaptation through unsupervised target domain data (i.e., target domain image samples), a more robust teacher model generates pseudo labels and provides them to the student model for training, and the student model is updated through gradients. Considering the characteristics of the DETR target detection framework that only one feature query is assigned to the same target as much as possible and the problem of category imbalance in the real world environment, this embodiment uses a variety of categories of memory libraries under the basic contrast loss to obtain a large number of high-quality features that are as balanced as possible for contrastive learning, thereby calculating the contrastive learning loss through pseudo labels, student category prediction results, student bounding box prediction results, student decoding features, and a variety of categories of memory libraries to learn higher quality category features in the target domain. Considering the characteristics of the DETR target detection framework and the passive domain adaptation In the target detection task, the true label is invisible. This embodiment uses the uncertainty of the prediction to measure the prediction quality, obtains the uncertainty weight, and then calculates the feature distillation loss based on the teacher encoding features, the teacher decoding features, the student encoding features and the uncertainty weight to achieve more robust knowledge distillation between the teacher model and the student model; in this way, this embodiment updates the teacher model and the student model at least based on the contrastive learning loss of the matching category memory library and the feature distillation loss based on uncertainty weighting to obtain a trained student model of the target domain to perform target detection on the image to be detected in the target domain, thereby solving the problem that the DETR detection model cannot be applied to domain adaptive target detection in passive scenarios, improving the detection accuracy of the DETR target detection model in passive domain adaptive scenarios, and enabling the DETR target detection model to have efficient and stable detection performance under various complex domain offsets in real-world scenarios.

[0033] In combination with the above embodiments, in one implementation, the present invention further provides a method based on target detection. In the method, in addition to the above steps, step S21 may be further included, and the above step S17 may specifically include step S22: Step S21: obtaining a reweighted classification loss based on the student encoding feature, the student decoding feature, the student category prediction result and the student bounding box prediction result.

[0034] Considering that contrastive learning loss can only alleviate the impact of category imbalance on contrastive learning to a certain extent, in order to better solve the problem of category imbalance, a reweighted classification loss is designed in this embodiment: based on student encoding features, student decoding features, student category prediction results and student bounding box prediction results, the reweighted classification loss is calculated.

[0035] Step S22: updating the teacher model and the student model based at least on the feature distillation loss, the contrastive learning loss, and the reweighted classification loss to obtain the trained student model.

[0036] In this embodiment, when updating the model parameters, the model parameters of the teacher model and the student model are updated based on at least the feature distillation loss, the contrastive learning loss and the reweighted classification loss to obtain a trained student model.

[0037] In combination with the above embodiments, the present invention further provides a target detection method, in which, in addition to the above steps, the method may further include step S31 and step S32, and the above step S22 may specifically include step S33: Step S31: Obtain a bounding box loss based on the bounding box pseudo-label corresponding to the teacher bounding box prediction result and the student bounding box prediction result.

[0038] In this embodiment, the bounding box loss can be calculated based on the bounding box pseudo-label corresponding to the teacher's bounding box prediction result and the student's bounding box prediction result.

[0039] Step S32: Obtaining the object detection loss based on the reweighted classification loss and the bounding box loss.

[0040] In this embodiment, the target detection loss is obtained based on the reweighted classification loss and bounding box loss. That is to say, this embodiment replaces the classification loss part of the detection loss (including classification detection loss and bounding box loss) of the traditional DETR model with the reweighted classification loss, and the target detection loss of this embodiment includes the reweighted classification loss and the bounding box loss. Among them, the classification detection loss of the traditional detection loss is the loss between the category pseudo label corresponding to the teacher category prediction result and the student category prediction result.

[0041] Step S33: Based on the feature distillation loss, the contrastive learning loss and the target detection loss, the teacher model and the student model are updated to obtain the trained student model.

[0042] In this embodiment, the total loss is calculated based on the feature distillation loss, the contrastive learning loss, and the target detection loss, and then the model parameters of the teacher model and the student model are updated based on the total loss to obtain a trained student model.

[0043] Among them, this embodiment uses the target detection loss, contrastive learning loss and feature distillation loss for linear weighting as the total loss. The total loss for: ; in, is the target detection loss, For contrastive learning loss, is the characteristic distillation loss, and are the weights corresponding to the contrast loss and feature distillation loss, respectively.

[0044] In combination with the above embodiments, in one implementation, the present invention further provides a target detection method. In this method, the above step S14 may specifically include step S41 and step S42: Step S41: Calculate the predicted entropy value E based on the student category prediction result output by the student model.

[0045] In this embodiment, uncertainty estimation is performed based on the student category prediction results output by the student model (the features corresponding to each prediction instance, that is, the prediction scores of each student decoding features), and the prediction entropy value E is calculated.

[0046] Step S42: Based on the predicted entropy value E, obtain the uncertainty weight.

[0047] In this embodiment, after obtaining the predicted entropy value E, an uncertainty weight is calculated based on the predicted entropy value E, and a higher weight is given to the prediction with a lower entropy value (higher prediction certainty). As shown in the following formula: ; in, (E) means (E-min(E)) / (max(E) - min(E)); is a hyperparameter for smoothing, and E is the predicted entropy value.

[0048] In combination with the above embodiments, in one implementation, the present invention further provides a target detection method. In this method, the above step S15 may specifically include steps S51 to S52: Step S51: Obtain a pixel-level mask based on the last layer of teacher encoding features in the teacher encoding features and the teacher decoding features.

[0049] In this embodiment, a pixel-level mask can be obtained based on the last layer of teacher coding features in the teacher coding features (i.e., the global features output by the last layer of encoder of the teacher model) and the teacher decoding features. Specifically, the last layer of teacher coding features and teacher decoding features in the teacher coding features can be fused to obtain a pixel-level mask that reflects the objectivity of the target.

[0050] In an alternative example, the pixel level mask The calculation formula is: ; in, Encode features for the final layer of teachers; Decode the features for the teacher, [n q , C] tensor, n q is the number of single queries, C is the dimension of a single query, j represents the j-th element, that is, the j-th row is a tensor with a shape of [C]; T is the transpose.

[0051] Step S52: Obtain the feature distillation loss based on the last layer of teacher coding features, the last layer of student coding features in the student coding features, the pixel-level mask and the uncertainty weight.

[0052] In this embodiment, after obtaining the pixel-level mask, the pixel-level mask can be applied to the pixel-level feature distillation loss, and the above-mentioned uncertainty weights are assigned to the pixel-level masks corresponding to different teacher decoding features to obtain the feature distillation loss. Specifically, the feature distillation loss can be calculated based on the last layer of teacher encoding features, the last layer of student encoding features in the student encoding features (i.e., the global features output by the last layer of encoder of the student model), the pixel-level mask, and the uncertainty weights.

[0053] In an alternative example, the feature distillation loss The calculation formula is: ; Among them, the superscript j represents the jth element, and j traverses from 0 to -1, is a pixel-level mask, is the Hadamard product, and Respectively represent the last layer of teacher coding features and the last layer of student coding features, represents the uncertainty weight, H, W, and C represent the height, width, and number of channels of the last layer of encoder features (the last layer of teacher encoding features or the last layer of student encoding features), respectively. The number of decoded features when the DETR target detection model performs target detection on a sample (that is, the number of single queries in the object query (the number of rows in the tensor)).

[0054] Unlike the prediction score imitation method commonly used in traditional knowledge distillation, this embodiment uses a novel feature distillation method, which has been proven to be effective in target detection tasks. Specifically, an additional forward propagation is first performed to forward propagate the strongly enhanced samples in the teacher model, and this additional forward propagation is only used for knowledge distillation. Next, the fused instance features are weighted according to the prediction quality, and a feature distillation loss is calculated for optimization. In this way, compared to the use of feature distillation methods in related technologies, which often do not consider the different contributions of different samples, or the contribution-based weighted method used cannot be directly applied to passive challenges, the feature distillation loss of this embodiment can be applied to passive domain adaptation.

[0055] In combination with the above embodiments, in one implementation, the present invention further provides a target detection method. In this method, the above step S21 may specifically include steps S61 to S64: Step S61: fusing the multiple layers of student encoding features and the student decoding features in the student encoding features to obtain fused features.

[0056] In this embodiment, the multi-layer student encoding features (global features output by the multi-layer encoder of the student model) and the student decoding features in the student encoding features can be fused to obtain fused features (global features after fusion) to calculate the similarity between the target represented by the target query and the global features.

[0057] In an optional example, the fusion feature The formula is: ; in, The number of decoded features when the DETR target detection model performs target detection on a sample (that is, the number of single queries in the object query (the number of rows in the tensor)); is the output feature of the i-th layer encoder of the student model, and the value range of i is {3,4,5}; The transpose of the student decoding feature, j represents the jth element, and j traverses from 0 to -1.

[0058] Step S62: Obtaining regional features based on the fusion features and the student bounding box prediction results.

[0059] In the DETR framework, the model predicts category information and location information (bounding box) for each target query. Therefore, after obtaining the fusion feature, this embodiment can obtain the regional feature based on the fusion feature and the student bounding box prediction result. Specifically, the RoIAlign algorithm can be used to extract the local features in the bounding box for the fusion feature and the student bounding box prediction result to obtain the regional feature, that is, the regional attention.

[0060] In an alternative embodiment, the regional features As shown below: ; in, is the fusion feature; bbox is the student bounding box prediction result, ;RoIAlign() is the RoIAlign algorithm.

[0061] Step S63: Obtaining a query weight based on the regional features.

[0062] In this embodiment, after obtaining the regional features, the query weight can be obtained based on the regional features. In a specific example, after obtaining the regional features, all target queries have the attention of their corresponding objects (targets) in the region, so this embodiment can sum the attention scores in the region to represent the objectivity prediction score of the target query having the corresponding target in the region.

[0063] In order to better utilize the multi-scale encoder features on the Transformer architecture, in an optional embodiment, a set of target scores (objectness score, an estimated indicator of valid objects in the region) can be calculated on the output features of the 3rd, 4th, and 5th encoders of the student model respectively according to the following method and the same target query can be summed on the features of encoders at different levels. The formula is as follows: ; in, are the height and width output by the RoIAlign algorithm; j represents the jth element, and j traverses from 0 to -1; is the regional feature; S represents the set of target-specific attention (regional features) obtained from the multi-scale encoder features corresponding to each target query.

[0064] After obtaining S, we scale it to get the query weight. : ; in, is a hyperparameter for smoothing, , is the i-th element in the set S.

[0065] Step S64: Based on the query weight and the student category prediction result, the reweighted classification loss is obtained.

[0066] In this embodiment, after obtaining the query weight, the reweighted classification loss can be calculated based on the query weight and the student category prediction result. .

[0067] In an optional implementation, considering the imbalance of foreground-background samples, this embodiment naturally hopes to assign high weights to foreground areas and difficult samples with low attention when re-weighting samples. The former (foreground area) can be solved to a certain extent by the weight of Focal loss, and the latter (difficult samples with low attention) can be solved by assigning higher weights to samples with lower target scores. Specifically, in this embodiment, on the basis of traditional Focal loss, each sample is further weighted, and the formula is as follows: ; in, and is the balance parameter of Focal loss; Predict outcomes for student categories; is the query weight; The number of decoded features when the DETR target detection model performs target detection on a sample (that is, the number of single queries in objectquery (the number of rows in the tensor)).

[0068] In combination with the above embodiments, in one implementation, the present invention further provides a target detection method. In this method, the multiple categories of memory libraries include: a background memory library and multiple different categories of foreground memory libraries; the above step S16 may specifically include steps S71 to S74: Step S71: fusing the student decoding features of multiple scales in the student decoding features to obtain multi-scale decoding features.

[0069] In this embodiment, considering the rich semantics of the multi-scale decoder output features in the Transformer module of the DETR target detection model, the student decoding features of multiple different scales in the student decoding features are accumulated and fused to obtain multi-scale decoding features. In this embodiment, the decoder features of different scales are accumulated and fused (multi-scale decoding features) as the key for comparative learning.

[0070] Step S72: When the student category prediction result and the student bounding box prediction result successfully match the pseudo label, the first multi-scale decoding feature is added to the foreground memory library of the category corresponding to the pseudo label.

[0071] In this embodiment, the student category prediction result and the student bounding box prediction result are matched with the pseudo-label (bounding box pseudo-label and category pseudo-label) output by the teacher model. When the student category prediction result and the student bounding box prediction result are successfully matched with the pseudo-label, the category corresponding to the successfully matched pseudo-label is determined, and the first multi-scale decoding feature is added to the foreground memory bank of the category corresponding to the pseudo-label. The first multi-scale decoding feature is a multi-scale decoding feature corresponding to the successfully matched student category prediction result and the student bounding box prediction result.

[0072] In an optional implementation, the Hungarian algorithm can be used for bipartite graph matching to construct the minimum matching cost between the query feature (i.e., the multi-scale decoding feature) and each category (including foreground and background categories) to obtain the corresponding category information, so as to construct a feature memory library based on the category information of the multi-scale decoding feature.

[0073] Step S73: When the student category prediction result and the student bounding box prediction result fail to match the pseudo label, adding the second multi-scale decoding feature to the background memory library.

[0074] In this embodiment, when the student category prediction result and the student bounding box prediction result fail to match the pseudo label, the second multi-scale decoding feature is added to the background memory library. The second multi-scale decoding feature is the multi-scale decoding feature corresponding to the student category prediction result and the student bounding box prediction result that failed to match.

[0075] This embodiment constructs a memory bank for each category (including the background category), all memory banks have the same maximum feature capacity, and are updated in a first-in-first-out manner to maintain the timeliness of category features.

[0076] Step S74: performing contrastive learning based on the features in the memory bank where the first multi-scale decoding features are located and the features in the memory banks of the multiple categories to obtain the contrastive learning loss.

[0077] In this embodiment, the contrastive learning loss can be calculated based on multiple categories of memory banks. Specifically, the contrastive learning can be performed based on the features in the memory bank where the first multi-scale decoding feature is located and the features in the memory banks of multiple categories to obtain the contrastive learning loss. The memory bank where the first multi-scale decoding feature is located can be understood as the foreground memory bank corresponding to the successfully matched multi-scale decoding feature.

[0078] In an alternative embodiment, contrastive learning loss as follows: ; in, is the number of foreground categories, is the current foreground category number, is the foreground memory of the current foreground category, is the union of the memory banks of all categories (including background and foreground), is the query vector in the current category memory bank (i.e., the multi-scale decoding features in the current category memory bank), is the key vector of the same category as the query vector in the current category memory, is the temperature coefficient, , is all the multi-scale decoding features in the memory bank of all categories, exp represents the exponential function, exponential function.

[0079] In combination with the above embodiments, in one implementation, the present invention further provides a target detection method. In this method, the "updating the teacher model and the student model" in the above step S17 may specifically include steps S81 to S83: Step S81: Update the student model.

[0080] In this embodiment, after obtaining the total loss (such as the feature distillation loss and contrastive learning loss in the aforementioned embodiments; the feature distillation loss, contrastive learning loss and reweighted classification loss; or, the feature distillation loss, contrastive learning loss and target detection loss), the model parameters of the student model can be updated based on the total loss, for example, based on the stochastic gradient descent sgd algorithm, etc., to obtain a trained student model.

[0081] Step S82: Determine the exponential moving average update interval based on the basic update interval and the current update round number.

[0082] In this embodiment, after the student model is updated, the gradient propagation to the teacher model is cut off, and the teacher model is updated using the exponential moving average of the student model parameters and the teacher model parameters. Specifically, the dynamic exponential moving average update interval can be determined based on a preset or default basic update interval and the current update epoch.

[0083] In an optional implementation, the formula for the exponential moving average update interval is as follows: ; in, is the calculated exponential moving average update interval, δ is the basic update interval, e is the current update round number (epoch) of the model, is a smoothing parameter.

[0084] Step S83: When the number of updates of the student model reaches the exponential moving average update interval, the teacher model is updated.

[0085] In this embodiment, the exponential moving average update interval can be understood as the concept of the number of iterations, that is, the interval between batches (the number of times the student model is updated) at which the teacher model is updated. In other words, when the number of student model updates reaches the exponential moving average update interval, the teacher model is updated.

[0086] Different from most other methods that set the iteration interval of the exponential moving average update to iteration number = 1, the dynamically increased exponential moving average update interval in this embodiment is conducive to the model exploring the parameter space more frequently in the early stage of domain adaptation training, and gradually improving the stability of parameter adjustment as the training progresses.

[0087] In one embodiment, if Figure 2 As shown, Figure 2 FIG. 1 is a schematic diagram of a passive domain adaptive target detection framework based on DETR according to an embodiment of the present invention. Figure 2 In the paper, the passive domain adaptation object detection framework based on DETR is a feature reweighting and contrastive learning network, which at least includes a contrastive learning module based on matching memory, a sample reweighting module based on target score, and a query fusion feature distillation module based on uncertainty weighting.

[0088] Among them, the target domain image samples are weakly enhanced and then input into the pre-trained teacher model of the source domain (i.e. Figure 2 The teacher network of the teacher model is obtained by obtaining the teacher encoding features output by the encoder of the teacher model. The teacher encoding features and the first tensor to be trained are input into the decoder of the teacher model to obtain the teacher decoding features output by the decoder. The teacher decoding features are predicted by the regression head and the classification head to obtain the pseudo labels.

[0089] The target domain image samples are strongly enhanced and then input into the student model to be trained in the target domain (i.e. Figure 2 The student network of the student model is constructed by the student encoding head, and the student encoding features and the second tensor to be trained are input into the decoder of the student model to obtain the student decoding features output by the decoder. The student decoding features are predicted by the regression head and the classification head respectively to obtain the student bounding box prediction results and the student category prediction results.

[0090] Based on this, the student decoded features and pseudo labels are input into the contrastive learning module based on the matching memory bank (i.e. Figure 2 CMMB in the module), in which the student decoding features of multiple scales in the student decoding features are accumulated and fused to obtain the multi-scale decoding features (i.e. Figure 2 The multi-scale query features in the pseudo-label are then matched with the student category prediction results and the student bounding box prediction results to obtain the matching results. The multi-scale decoding features corresponding to the successfully matched student category prediction results and the student bounding box prediction results are added to the foreground memory library of the category corresponding to the pseudo-label. The multi-scale decoding features corresponding to the failed matching student category prediction results and the student bounding box prediction results are added to the background memory library. Finally, comparative learning is performed based on the memory libraries of multiple categories to obtain the comparative learning loss. .

[0091] The teacher encoding features, teacher decoding features, student encoding features, and student category prediction results are input into the uncertainty-weighted query fusion feature distillation module (i.e. Figure 2 In this module, uncertainty estimation is first performed based on the student category prediction results output by the student model to obtain uncertainty weights; then pixel-level masks are obtained based on the last layer of teacher encoding features and teacher decoding features in the teacher encoding features; finally, distillation learning is performed based on the last layer of teacher encoding features, the last layer of student encoding features in the student encoding features, pixel-level masks, and uncertainty weights to obtain feature distillation loss .

[0092] The student encoding features, student decoding features, student category prediction results, and student bounding box prediction results are input into the sample reweighting module based on the target score. In this module, the multi-layer student encoding features and student decoding features in the student encoding features are first fused to obtain the fused features (i.e. Figure 2 Then, based on the fusion features and the student bounding box prediction results, RoI Align calculation is performed to obtain the regional features (i.e. Figure 2Then, the target query weight is obtained by weighting and scaling based on the regional features; finally, the target query weight and the student category prediction result (i.e. Figure 2 ), calculate the reweighted classification loss .

[0093] Finally, based on contrastive learning loss , feature distillation loss and reweighted classification loss The parameters of the student model to be trained in the target domain are updated, that is, the model and each component are learned and updated under the teacher-student framework. This embodiment further proposes a dynamic teacher update interval under the teacher-student framework, that is, the teacher model is updated by the exponential moving average of the student model parameters (i.e. Figure 2 DTUI in): Based on the basic update interval and the current update round number, the exponential moving average update interval is determined, and when the number of student model updates reaches the exponential moving average update interval, the teacher model parameters are updated. In addition, the model designed in this embodiment can be deployed in the NVIDIA GeForce RTX4090 GPU environment and perform passive target detection under various domain offset conditions in the real world.

[0094] The purpose of this embodiment is to develop a passive domain adaptive target detection framework based on the DETR detection model and solve the corresponding challenges, so that the advanced DETR target detector can effectively cope with the passive challenges brought by the domain shift challenge and privacy protection problem in the real world environment. Specifically, based on the DETR target detector, this embodiment includes three components: a contrastive learning module based on a matching memory library, a sample reweighting module based on a target score, and a query fusion feature distillation module based on uncertainty weighting, as well as a dynamic teacher update interval for model parameter updates under the teacher-student framework. After testing, on a variety of internationally used passive cross-domain target detection benchmarks, the model has achieved the most advanced detection performance after the proposed algorithm is domain adapted, compared with a variety of advanced baseline methods, and can effectively improve the generalization performance of the DETR target detector in the real world environment.

[0095] Among them, the contrastive learning module based on matching memory of this embodiment, compared with the related art that performs contrastive learning based on instance features directly obtained from the feature extractor, proposes a category memory feature using multi-level feature fusion, and uses bipartite graph matching to match category labels for memory features. The sample reweighting module based on target score of this embodiment, compared with the related art that reweights samples by predicting confidence quality or intersection quality, can be used under passive challenges. And, the query fusion feature distillation module based on uncertainty weighting of this embodiment, compared with the related art that does not consider the different contributions of different samples when using feature distillation methods, or uses contribution-based weighting methods, can be directly applied to passive challenges.

[0096] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0097] Based on the same inventive concept, an embodiment of the present invention provides a target detection device. Figure 3 , Figure 3 FIG. 1 is a block diagram of a target detection device provided by an embodiment of the present invention. Figure 3 As shown, the target detection device of this embodiment may include: An image processing module, used for performing weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and performing strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample; A first input module is used to input the weakly enhanced sample into a pre-trained teacher model of the source domain to obtain a teacher encoding feature output by the encoder of the teacher model and a teacher decoding feature output by the decoder of the teacher model; A second input module is used to input the strongly enhanced sample into a student model to be trained in the target domain to obtain a student encoding feature output by an encoder of the student model and a student decoding feature output by a decoder of the student model; A first weight determination module, configured to perform uncertainty estimation based on the student category prediction result output by the student model to obtain an uncertainty weight; A first loss determination module, configured to obtain a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight; a second loss determination module, configured to obtain a contrastive learning loss based on a pseudo label, the student category prediction result, the student bounding box prediction result output by the student model, the student decoding feature, and a plurality of categories of memory banks, wherein the pseudo label is determined according to the teacher category prediction result and the teacher bounding box prediction result output by the teacher model; A model training module, used for updating the teacher model and the student model based on at least the feature distillation loss and the contrastive learning loss to obtain a trained student model; The target detection module is used to input the image to be detected in the target domain into the trained student model to obtain the target detection result.

[0098] Optionally, the device further comprises: A third loss determination module, configured to obtain a reweighted classification loss based on the student encoding feature, the student decoding feature, the student category prediction result, and the student bounding box prediction result; Model training module, including: The first training module is used to update the teacher model and the student model based on at least the feature distillation loss, the contrastive learning loss and the reweighted classification loss to obtain the trained student model.

[0099] Optionally, the device further comprises: a fourth loss determination module, configured to obtain a bounding box loss based on a bounding box pseudo label corresponding to the teacher bounding box prediction result and the student bounding box prediction result; a fifth loss determination module, configured to obtain an object detection loss based on the reweighted classification loss and the bounding box loss; The first training module includes: The second training module is used to update the teacher model and the student model based on the feature distillation loss, the contrastive learning loss and the target detection loss to obtain the trained student model.

[0100] Optionally, the first weight determination module includes: An entropy value determination module, used to calculate a predicted entropy value E based on the student category prediction result output by the student model; The second weight determination module is used to obtain the uncertainty weight based on the predicted entropy value E.

[0101] Optionally, the first loss determination module includes: A first determination module is used to obtain a pixel-level mask based on the last layer of teacher encoding features in the teacher encoding features and the teacher decoding features; The second determination module is used to obtain the feature distillation loss based on the last layer of teacher encoding features, the last layer of student encoding features in the student encoding features, the pixel level mask and the uncertainty weight.

[0102] Optionally, the third loss determination module includes: A first fusion module is used to fuse the multiple layers of student encoding features and the student decoding features in the student encoding features to obtain a fusion feature; A third determination module is used to obtain a regional feature based on the fusion feature and the student bounding box prediction result; A fourth determination module, configured to obtain a query weight based on the regional features; The fifth determination module is used to obtain the reweighted classification loss based on the query weight and the student category prediction result.

[0103] Optionally, the multiple types of memory libraries include: a background memory library and multiple different types of foreground memory libraries; the second loss determination module includes: A second fusion module is used to fuse the student decoding features of multiple scales in the student decoding features to obtain multi-scale decoding features; A first adding module is used to add a first multi-scale decoding feature to a foreground memory bank of a category corresponding to the pseudo-label when the student category prediction result and the student bounding box prediction result successfully match the pseudo-label, wherein the first multi-scale decoding feature is a multi-scale decoding feature corresponding to the successfully matched student category prediction result and the student bounding box prediction result; A second adding module is used to add a second multi-scale decoding feature to the background memory library when the student category prediction result and the student bounding box prediction result fail to match the pseudo label, wherein the second multi-scale decoding feature is a multi-scale decoding feature corresponding to the student category prediction result and the student bounding box prediction result that failed to match; The sixth determination module is used to perform contrastive learning based on the features in the memory bank where the first multi-scale decoding features are located and the features in the memory banks of the multiple categories to obtain the contrastive learning loss.

[0104] Optionally, the model training module, the first training module or the second training module includes: A first updating module, used for updating the student model; A seventh determination module, configured to determine an exponential moving average update interval based on a basic update interval and a current update round number; The second updating module is used to update the teacher model when the number of updates of the student model reaches the exponential moving average update interval.

[0105] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, such as Figure 4 shown. Figure 4 1 is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when executed by the processor, the steps of the target detection method according to any of the above embodiments of the present invention are implemented.

[0106] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0107] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0108] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, embodiments of the present invention may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware. Furthermore, embodiments of the present invention may take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0109] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0113] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0114] The target detection method, device and equipment provided by the present invention are introduced in detail above. The principles and implementation methods of the present invention are explained in this article using specific examples. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A target detection method, characterized in that: The method comprises: Performing weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and performing strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample; Inputting the weakly enhanced sample into a pre-trained teacher model of the source domain, obtaining a teacher encoding feature output by an encoder of the teacher model and a teacher decoding feature output by a decoder of the teacher model; Inputting the strongly enhanced sample into a student model to be trained in the target domain, obtaining a student encoding feature output by an encoder of the student model and a student decoding feature output by a decoder of the student model; Based on the student category prediction results output by the student model, uncertainty estimation is performed to obtain uncertainty weights; Obtaining a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight; Obtaining contrastive learning loss based on pseudo labels, the student category prediction results, the student bounding box prediction results output by the student model, the student decoding features, and multiple categories of memory banks, wherein the pseudo labels are determined based on the teacher category prediction results and the teacher bounding box prediction results output by the teacher model; At least based on the feature distillation loss and the contrastive learning loss, the teacher model and the student model are updated to obtain a trained student model; The image to be detected in the target domain is input into the trained student model to obtain the target detection result.

2. The target detection method according to claim 1, characterized in that: The method further comprises: Obtaining a reweighted classification loss based on the student encoding feature, the student decoding feature, the student category prediction result, and the student bounding box prediction result; At least based on the feature distillation loss and the contrastive learning loss, the teacher model and the student model are updated to obtain a trained student model, including: At least based on the feature distillation loss, the contrastive learning loss and the reweighted classification loss, the teacher model and the student model are updated to obtain the trained student model.

3. The target detection method according to claim 2, characterized in that: The method further comprises: Obtaining a bounding box loss based on the bounding box pseudo-label corresponding to the teacher bounding box prediction result and the student bounding box prediction result; Obtaining an object detection loss based on the reweighted classification loss and the bounding box loss; At least based on the feature distillation loss, the contrastive learning loss, and the reweighted classification loss, the teacher model and the student model are updated to obtain the trained student model, including: Based on the feature distillation loss, the contrastive learning loss and the target detection loss, the teacher model and the student model are updated to obtain the trained student model.

4. The target detection method according to claim 1, characterized in that: Based on the student category prediction results output by the student model, uncertainty estimation is performed to obtain uncertainty weights, including: Calculate the predicted entropy value E based on the student category prediction result output by the student model; Based on the predicted entropy value E, the uncertainty weight is obtained.

5. The target detection method according to claim 1, characterized in that: Based on the teacher encoding feature, the teacher decoding feature, the student encoding feature and the uncertainty weight, a feature distillation loss is obtained, including: Based on the last layer of teacher encoding features in the teacher encoding features and the teacher decoding features, a pixel-level mask is obtained; The feature distillation loss is obtained based on the last layer of teacher encoding features, the last layer of student encoding features in the student encoding features, the pixel level mask and the uncertainty weight.

6. The target detection method according to claim 2, characterized in that: Based on the student encoding feature, the student decoding feature, the student category prediction result, and the student bounding box prediction result, a reweighted classification loss is obtained, including: Fusing the multiple layers of student encoding features and the student decoding features in the student encoding features to obtain fused features; Obtaining regional features based on the fusion features and the student bounding box prediction results; Based on the regional features, a query weight is obtained; Based on the query weight and the student category prediction result, the reweighted classification loss is obtained.

7. The target detection method according to claim 1, characterized in that: The multiple categories of memory libraries include: a background memory library and multiple different categories of foreground memory libraries; based on the pseudo-label, the student category prediction result, the student bounding box prediction result output by the student model, the student decoding feature, and the multiple categories of memory libraries, a contrastive learning loss is obtained, including: fusing the student decoding features at multiple scales in the student decoding features to obtain multi-scale decoding features; When the student category prediction result and the student bounding box prediction result successfully match the pseudo label, adding a first multi-scale decoding feature to the foreground memory bank of the category corresponding to the pseudo label, wherein the first multi-scale decoding feature is a multi-scale decoding feature corresponding to the successfully matched student category prediction result and the student bounding box prediction result; In the case where the student category prediction result and the student bounding box prediction result fail to match the pseudo label, adding a second multi-scale decoding feature to the background memory library, where the second multi-scale decoding feature is a multi-scale decoding feature corresponding to the student category prediction result and the student bounding box prediction result that failed to match; Based on the features in the memory bank where the first multi-scale decoding features are located and the features in the memory banks of the multiple categories, contrastive learning is performed to obtain the contrastive learning loss.

8. The target detection method according to any one of claims 1 to 7, characterized in that: The teacher model and the student model are updated, including: updating the student model; Based on the basic update interval and the current update round number, determine the exponential moving average update interval; When the number of updates of the student model reaches the exponential moving average update interval, the teacher model is updated.

9. A target detection device, characterized in that: The device comprises: An image processing module, used for performing weak enhancement processing on the target domain image sample to obtain a weakly enhanced sample, and performing strong enhancement processing on the target domain image sample to obtain a strongly enhanced sample; A first input module is used to input the weakly enhanced sample into a pre-trained teacher model of the source domain to obtain a teacher encoding feature output by the encoder of the teacher model and a teacher decoding feature output by the decoder of the teacher model; A second input module is used to input the strongly enhanced sample into a student model to be trained in the target domain to obtain a student encoding feature output by an encoder of the student model and a student decoding feature output by a decoder of the student model; A first weight determination module, configured to perform uncertainty estimation based on the student category prediction result output by the student model to obtain an uncertainty weight; A first loss determination module, configured to obtain a feature distillation loss based on the teacher encoding feature, the teacher decoding feature, the student encoding feature, and the uncertainty weight; a second loss determination module, configured to obtain a contrastive learning loss based on a pseudo label, the student category prediction result, the student bounding box prediction result output by the student model, the student decoding feature, and a plurality of categories of memory banks, wherein the pseudo label is determined according to the teacher category prediction result and the teacher bounding box prediction result output by the teacher model; A model training module, used for updating the teacher model and the student model based on at least the feature distillation loss and the contrastive learning loss to obtain a trained student model; The target detection module is used to input the image to be detected in the target domain into the trained student model to obtain the target detection result.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the target detection method according to any one of claims 1 to 8 is implemented.

Images