Target detection method, device, computer equipment, and storage medium

Abstract

The present application relates to a method, apparatus, computer equipment, and storage medium for target detection. The method comprises: obtaining source domain data including annotated data, target domain data including unannotated data, and a student model; training the student model based on the source domain data to obtain the structure and weight of the trained student model; constructing a teacher model based on the structure and weight of the trained student model; inputting the target domain data into the teacher model to obtain a pseudo label of the target domain data; inputting the pseudo label and the target domain data into the trained student model for training to obtain an optimized student model, updating the model parameters of the teacher model based on the model parameters of the student model, and performing target detection based on the updated teacher model.

Description

Technical Field

[0001] The present application relates to the field of target detection, and in particular to a target detection method, apparatus, computer equipment, and storage medium. Background Art

[0002] For object detection tasks, precise location coordinates and category information must be annotated for each instance in each image, making annotating a complete dataset time-consuming and labor-intensive. Generalizing models pre-trained on annotated datasets (source domain datasets) to unlabeled datasets (target domain datasets) is a promising approach. However, because the images in the source and target domain datasets are collected under different devices, weather conditions, angles, and lighting conditions, object detection models often have lower accuracy in the target domain dataset. Summary of the Invention

[0003] Based on this, it is necessary to provide a target detection method, device, computer equipment, and storage medium to address the above technical problems.

[0004] In a first aspect, the present application provides a target detection method. The method comprises:

[0005] Obtain annotated source domain data, unannotated target domain data, and student models;

[0006] Training a student model based on the source domain data to obtain a structure and weight of the trained student model;

[0007] Constructing a teacher model based on the structure and weights of the trained student model;

[0008] Inputting the target domain data into the teacher model to obtain pseudo labels for the target domain data;

[0009] The pseudo labels and target domain data are input into the trained student model for training to obtain an optimized student model, the model parameters of the teacher model are updated based on the model parameters of the optimized student model, and target detection is performed based on the updated teacher model.

[0010] In one embodiment, training the student model based on the source domain data includes:

[0011] extracting instance features from the source domain data based on the student model;

[0012] For any multiple instance features in the same category, calculate the distance between the instance features;

[0013] In the case where the distance between the instance features is less than a preset distance, the feature instances whose distance between the instance features is less than the preset distance are input into the same clustering set to form a category multi-prototype structure.

[0014] In one embodiment, the method further comprises:

[0015] The pseudo labels are modified based on the category multi-prototype structure and virtual instance features are generated.

[0016] In one embodiment, the pseudo-label is corrected in the following manner:

[0017] Extract instance features of each target instance in the image;

[0018] For each category, the spatial distance between all instance features of each category and all prototypes in each category is calculated;

[0019] The spatial distance is calculated according to the following formula:

[0020] = I k j ∙φ[dis( I k j , P k i )]< δ d ;

[0021] in represents the j-th prototype in category k, represents the j-th instance feature in category k, φ[∙] Represents the mapping function, when [ ∙ ] When the content is true, φ[∙]=1 ,otherwise φ[∙]=0 , Represents the distance threshold between instance features and category multi-prototypes;

[0022] The instance features that meet the distance threshold are used to predict pseudo labels.

[0023] In one embodiment, the method further comprises:

[0024] Initialize the parameters of the feature point classification branch newly added to the student model based on the source domain data to obtain the initialization loss function L init = ∑ k ∑ n H[ h cls stu ( f k n ), y s ] ;

[0025] in, Represents the nth feature point in category k in a batch of features, represents the feature point classifier output of the student model, represents the true classification label;

[0026] Generate virtual instance features based on the category multi-prototype structure, and obtain the feature consistency loss function based on the output results of the student model, the output results of the teacher model, and the virtual instance features. L consist = ∑ k ∑ n H[ h cls stu ( f * k n ), h cls tea ( f k n )] + ∑ k ∑ n H[ h cls stu ( f k n ), h cls tea ( f k n )] , ;

[0027] in, Represents the nth feature point in category k in the virtual instance feature, represents a Gaussian distribution, represents the mean of the i-th prototype within the class k, Represents the variance between feature points in a batch, represents the output of the student model, represents the output of the teacher model, represents the cross entropy loss function.

[0028] In one embodiment, updating the model parameters of the teacher model based on the optimized model parameters of the student model includes:

[0029] The model parameters of the optimized student model are obtained, and the model parameters of the teacher model are updated based on the exponential moving average algorithm.

[0030] In a second aspect, the present application further provides a target detection device, comprising:

[0031] An acquisition module, used to acquire source domain data with annotations, target domain data without annotations, and a student model;

[0032] A training module, configured to train a student model based on the source domain data to obtain a structure and weight of the trained student model;

[0033] Constructing a model for constructing a teacher model based on the structure and weights of the trained student model;

[0034] a generation module, configured to input the target domain data into the teacher model to obtain pseudo labels for the target domain data;

[0035] The detection module is used to input the pseudo-label into the trained student model for training to obtain an optimized student model, update the model parameters of the teacher model based on the model parameters of the student model, and perform target detection based on the updated teacher model.

[0036] In one embodiment, training the student model based on the source domain data includes:

[0037] extracting instance features from the source domain data based on the student model;

[0038] For any multiple instance features in the same category, calculate the distance between the instance features;

[0039] In the case where the distance between the instance features is less than a preset distance, the feature instances whose distance between the instance features is less than the preset distance are input into the same clustering set to form a category multi-prototype structure.

[0040] In one embodiment, the apparatus further comprises:

[0041] The pseudo labels are modified based on the category multi-prototype structure and virtual instance features are generated.

[0042] In one embodiment, the pseudo-label is corrected in the following manner:

[0043] Extract instance features of each target instance in the image;

[0044] For each category, the spatial distance between all instance features of each category and all prototypes in each category is calculated;

[0045] The spatial distance is calculated according to the following formula:

[0046] = I k j ∙φ[dis( I k j , P k i )]< δ d ;

[0047] in represents the j-th prototype in category k, represents the j-th instance feature in category k, φ[∙] Represents the mapping function, when [ ∙ ] When the content is true, φ[∙]=1 ,otherwise φ[∙]=0 , Represents the distance threshold between instance features and category multi-prototypes;

[0048] The instance features that meet the distance threshold are used to predict pseudo labels.

[0049] In one embodiment, the apparatus further comprises:

[0050] Initialize the parameters of the feature point classification branch newly added to the student model based on the source domain data to obtain the initialization loss function L init = ∑ k ∑ n H[ h cls stu ( f k n ), y s ] ;

[0051] in, Represents the nth feature point in category k in a batch of features, represents the feature point classifier output of the student model, represents the true classification label;

[0052] Generate virtual instance features based on the category multi-prototype structure, and obtain the feature consistency loss function based on the output results of the student model, the output results of the teacher model, and the virtual instance features. L consist = ∑ k ∑ n H[ h cls stu ( f * k n ), h cls tea ( f k n )] + ∑ k ∑ n H[ h cls stu ( f k n ), h cls tea ( f k n )] , ;

[0053] in, Represents the nth feature point in category k in the virtual instance feature, represents a Gaussian distribution, represents the mean of the i-th prototype within the class k, Represents the variance between feature points in a batch, represents the output of the student model, represents the output of the teacher model, represents the cross entropy loss function.

[0054] In one embodiment, updating the model parameters of the teacher model based on the optimized model parameters of the student model includes:

[0055] The model parameters of the optimized student model are obtained, and the model parameters of the teacher model are updated based on the exponential moving average algorithm.

[0056] In a third aspect, the present disclosure further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the target detection method when executing the computer program.

[0057] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the target detection method when executed by a processor.

[0058] In a fifth aspect, the present disclosure further provides a computer program product, comprising a computer program that implements the steps of the target detection method when executed by a processor.

[0059] The above target detection method has at least the following beneficial effects:

[0060] The embodiment scheme provided by the present disclosure uses source domain data to train a student model in the pre-training stage, and constructs a teacher model based on the student model. In the mutual learning stage, target domain data is input into the teacher model to obtain pseudo labels, and then the student model is trained with the pseudo labels to improve the model's target detection capability.

[0061] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the traditional technology, the following briefly introduces the drawings required for use in the embodiments or the description of the traditional technology. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] FIG1 is a diagram illustrating an application environment of a target detection method according to an embodiment;

[0064] Figure 2 1 is a flow chart of a target detection method according to an embodiment;

[0065] Figure 3 A schematic diagram of generating a multi-prototype structure of a category in one embodiment;

[0066] Figure 4 is a schematic diagram of pseudo-label correction in one embodiment;

[0067] Figure 5 Schematic diagram of a feature consistency loss process in one embodiment;

[0068] Figure 6This is an architectural diagram of the student model and teacher model in one embodiment;

[0069] Figure 7 is a structural block diagram of a target detection device in one embodiment;

[0070] FIG8 is a diagram showing the internal structure of a computer device according to one embodiment;

[0071] FIG9 is a diagram showing the internal structure of a server according to an embodiment. DETAILED DESCRIPTION

[0072] In order to enable ordinary people in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0073] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the accompanying drawings are used to distinguish similar items and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims. The terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, product, or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, product, or apparatus. Without further limitation, this does not preclude the presence of additional identical or equivalent elements in the process, method, product, or apparatus comprising the elements. For example, the use of terms such as "first," "second," and the like are intended to indicate names and do not imply any specific order.

[0074] The embodiment of the present disclosure provides a target detection method, which can be applied in the application environment shown in Figure 1. The terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0075] In some embodiments of the present disclosure, Figure 2 As shown, a target detection method is provided, and the method is described by taking the method as an example of the server in Figure 1 processing an image. It is understood that the method can be applied to the server, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In a specific embodiment, the method can include the following steps:

[0076] S202: Obtain labeled source domain data, unlabeled target domain data, and a student model.

[0077] The source domain data includes a large number of annotated images, which are usually related to the target domain data but not identical to them. The target domain data includes unannotated images.

[0078] S204: Train a student model based on the source domain data to obtain the structure and weight of the trained student model.

[0079] S206: Constructing a teacher model based on the structure and weights of the trained student model.

[0080] An initial student model is trained using source domain data to obtain the structure and weights of the trained student model. A teacher model is constructed based on the structure and weights of the trained student model. In some embodiments, the weights of the teacher model are further updated using an exponential moving average algorithm to increase its stability.

[0081] S208: Input the target domain data into the teacher model to obtain a pseudo label of the target domain data.

[0082] S210: Input the pseudo labels and target domain data into the trained student model for training to obtain an optimized student model, update the model parameters of the teacher model based on the model parameters of the optimized student model, and perform target detection based on the updated teacher model.

[0083] The target domain data is fed into the teacher model, which generates predicted labels for the data, also known as pseudo-labels. The target domain data and its corresponding pseudo-labels are fed back into the trained student model for a new round of training, allowing the student model to learn from more data. The teacher model's parameters are updated based on the optimized student model's parameters. This can be done by simply copying the student model's weights to the teacher model, or by more complex strategies, such as weighted averaging based on performance metrics.

[0084] In the initial stage, the student model is trained using labeled source domain data, and the teacher model with the same architecture is used to generate pseudo labels for the unlabeled target domain data. The student model is further trained with pseudo labels and target domain data to improve its performance on the target domain. The optimized student model is used to update the teacher model to maintain the performance of the teacher model, and target detection is performed based on the updated teacher model.

[0085] In the above target detection method, in the pre-training stage, the source domain data is used to train the student model, and a teacher model is constructed based on the student model. In the mutual learning stage, the target domain data is input into the teacher model to obtain pseudo labels, and then the student model is trained with the pseudo labels to improve the model's target detection capability.

[0086] In some embodiments of the present disclosure, training the student model based on the source domain data includes:

[0087] extracting instance features from the source domain data based on the student model;

[0088] For any multiple instance features in the same category, calculate the distance between the instance features;

[0089] In the case where the distance between the instance features is less than a preset distance, the feature instances whose distance between the instance features is less than the preset distance are input into the same clustering set to form a category multi-prototype structure.

[0090] Figure 3 A schematic diagram of generating a multi-prototype structure of a category in one embodiment.

[0091] Using labeled source domain data, we extract instance features from the image through the backbone network and detector. To preserve fine-grained differences within the same category, facilitate subsequent accurate correction of pseudo-labels, and semantic control of the generated virtual instance features, we use the following formula during initialization to cluster instance categories based on the distance between instance features:

[0092] ;

[0093] in represents the j-th instance feature in category k, represents the mth instance feature in category k, represents the i-th clustering set of all instance features that meet the screening conditions in the k-th category, Represents the mapping function, when When the content is true, ,otherwise , Indicates the distance threshold between features.

[0094] Secondly, to ensure the timeliness and stability of the prototype, we use the exponential moving average algorithm to continuously update the category multi-prototype structure:

[0095] ;

[0096] in represents the j-th prototype in category k, represents the j-th instance feature in category k, Represents the mapping function, when When the content is true, ,otherwise , Represents the distance threshold between features and .

[0097] In some embodiments of the present disclosure, the method further includes:

[0098] The pseudo labels are modified based on the category multi-prototype structure and virtual instance features are generated.

[0099] A multi-prototype structure means that each category has multiple prototypes. These prototypes can represent subgroups within the category, such as similar objects of different colors or paintings of different styles. This helps the model better understand and distinguish the diversity within the category.

[0100] The predicted labels generated by the model on the unlabeled target domain data are called pseudo-labels. Due to the imperfections of the model during the training process, the generated pseudo-labels may be wrong. Through the category multi-prototype structure, the pseudo-labels can be accurately corrected.

[0101] For each unlabeled target domain data, find the prototype that is most similar to it and correct the pseudo-label based on the similarity. When an unlabeled target domain data is very close to a prototype, it can be considered that the unlabeled target domain data belongs to the subcategory represented by the prototype, and the pseudo-label is corrected.

[0102] In addition to correcting pseudo-labels, the multi-class prototype architecture can also be used to generate virtual instance features. Based on each class's prototype, these features can be generated through interpolation, combining existing prototype features, adding noise, and using Gaussian distribution. Adding these generated virtual instance features to an existing dataset expands the dataset size and improves the model's generalization capabilities.

[0103] In some embodiments of the present disclosure, the pseudo-label is corrected in the following manner:

[0104] Extract instance features of each target instance in the image;

[0105] For each category, the spatial distance between all instance features of each category and all prototypes in each category is calculated;

[0106] The spatial distance is calculated according to the following formula:

[0107] ;

[0108] in represents the j-th prototype in category k, represents the j-th instance feature in category k, Represents the mapping function, when When the content is true, ,otherwise , Represents the distance threshold between instance features and category multi-prototypes;

[0109] The instance features that meet the distance threshold are used to predict pseudo labels.

[0110] Figure 4 Schematic diagram of pseudo-label correction in one embodiment.

[0111] Extracting the feature vector of the instance features can be done by using a pre-trained convolutional neural network such as ResNet or other feature extractors.

[0112] For each category K, calculate the spatial distance between all instance features of the category and all prototypes in the category. Spatial distance The calculation formula is as follows:

[0113] ,in, Represents the mapping function, when When the content is true, ,otherwise , Represents the distance threshold between features and .

[0114] For each category K, the distance between each prototype and the instance feature is calculated.

[0115] Each category prototype obtains multiple calculation results. If there is a spatial distance close to the category prototype, the instance feature is retained, otherwise the instance feature is deleted, that is, , retain the instance features, otherwise delete the instance features. The retained instance features will be fed into the detector for pseudo label prediction. Since the retained instance features are close enough to the category prototype, the possibility of correct classification is higher, thus generating high-quality pseudo labels.

[0116] By comparing the distance between instance features and category multi-prototypes, we can determine which instance features should be retained and further used for pseudo-label prediction, which can reduce the impact of noisy data and improve the quality of pseudo-labels.

[0117] In some embodiments of the present disclosure, the method further includes:

[0118] Initialize the parameters of the feature point classification branch newly added to the student model based on the source domain data to obtain the initialization loss function ;

[0119] in, Represents the nth feature point in category k in a batch of features, represents the feature point classifier output of the student model, represents the true classification label;

[0120] Generate virtual instance features based on the category multi-prototype structure, and obtain the feature consistency loss function based on the output results of the student model, the output results of the teacher model, and the virtual instance features. , ,

[0121] in, Represents the nth feature point in category k in the virtual instance feature, represents a Gaussian distribution, represents the mean of the i-th prototype within the class k, Represents the variance between feature points in a batch, represents the output of the student model, represents the output of the teacher model, represents the cross entropy loss function.

[0122] Figure 5 Schematic diagram of the feature consistency loss process in one embodiment.

[0123] Object detection models, such as Faster R-CNN, are typically used to identify multiple objects and their locations in an image. While a typical object detection model may include multiple components such as a backbone network, a feature extractor, a classifier, and a bounding box regressor, it is possible to add an additional feature to the original object detection model: a feature point classification branch to identify key feature points within an object. In the field of object detection, knowledge distillation can be used to train a smaller student model to achieve performance close to that of the teacher model while reducing the computational resources required. For example, a large, complex Faster R-CNN model can be used as the teacher model, and a smaller, more efficient Faster R-CNN model can be trained as the student model.

[0124] In addition to the original classification loss and bounding box regression loss, a new loss function needs to be defined for learning the feature point classification branch, namely the initialization loss function, which is based on the prediction error of the key feature point position. ,in Represents the nth feature point in category k in a batch of features, represents the feature point classifier output of the student detection model, represents the true classification label.

[0125] Generate virtual instance features based on multiple category prototypes to ensure the semantic consistency and diversity of generated instance features. ,in Represents the nth feature point in category k in the virtual instance feature, represents a Gaussian distribution, represents the mean of the i-th prototype within the class k, Represents the variance between feature points in a batch.

[0126] In order to ensure the consistency of features, feature consistency loss can be introduced , represents the feature point classifier output of the student detection model, represents the feature point classifier output of the teacher detection model, represents the cross entropy loss function.

[0127] During the initialization phase, the feature point classification branch is initialized using source domain data, optimizing the initialization loss function. During the training phase, virtual instance features are generated using multiple category prototypes. The feature consistency loss is optimized by comparing the differences between the outputs of the student and teacher models and the virtual instance features. This enhances the model's adaptability to data from different domains and improves the quality of pseudo-labels, as the feature consistency loss ensures that the features learned by the model are not only valid in the source domain but also consistent and diverse in the target domain.

[0128] In some embodiments of the present disclosure, updating the model parameters of the teacher model based on the optimized model parameters of the student model includes:

[0129] The model parameters of the optimized student model are obtained, and the model parameters of the teacher model are updated based on the exponential moving average algorithm.

[0130] Update the teacher model's weights using the exponential moving average algorithm (EMA):

[0131] ,

[0132] in represents the teacher model weight parameter, represents the student model weight parameter, It is a hyperparameter that controls the update speed of the teacher model weights.

[0133] The overall loss of the model can be expressed as:

[0134] ;

[0135] in, and They are the losses of the RPN (region generation network) and ROI (region of interest) branches in Faster RCNN (target detection model). is a hyperparameter to balance the total loss, and the source domain dataset is , Represent the source domain image, category, and coordinate information respectively. Let the target domain dataset be , Represents the target domain image. The target domain has no annotation information, so it does not contain category and coordinate information. The detection box generated by the teacher model during training is recorded as a pseudo label , Represents classification and coordinate information respectively.

[0136] Figure 6 This is an architectural diagram of the student model and teacher model in one embodiment.

[0137] In one embodiment of the present disclosure, the trained multi-prototype domain adaptive detection model is used for domain adaptive target detection. This specific embodiment demonstrates the migration results of the present invention on Cityscapes to Cityscapes Foggy (semantic city landscape segmentation dataset). The source domain image is Cityscapes, which includes eight common categories such as people and cars, and is collected during the day and in good weather conditions. The target domain image is Cityscapes Foggy, which includes eight common categories such as people and cars, and is collected in foggy and bad weather conditions. The experiment uses Mean Average Precision (MAP evaluation index) to represent the average accuracy of the model detection results. It can be seen from Table 1 that the present invention has different degrees of improvement under different backbone networks, among which the performance improvement using the VGG16 backbone network is 3.8%, and the performance improvement using the ResNet50 backbone network is 3.3%. The recognition effect after the model improvement is shown in Table 1:

[0138] Table 1 Recognition effect after model improvement

[0139]

[0140] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0141] Based on the same inventive concept, the presently disclosed embodiments further provide a target detection device for implementing the target detection method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations in the target detection device embodiments provided below can be found in the limitations of the target detection method described above and will not be further elaborated here.

[0142] The device may include a system (including a distributed system), software (application), module, component, server, client, etc. that uses the method described in the embodiments of this specification and is combined with the necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided by the embodiments of the present disclosure is as described in the following embodiments. Since the implementation scheme of the device to solve the problem is similar to the method, the implementation of the specific device in the embodiments of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements the predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0143] In one embodiment, Figure 7 As shown, a target detection device 700 is provided. The device may be the aforementioned server, or a module, component, device, unit, etc. integrated in the server. The device 700 may include:

[0144] An acquisition module 702 is configured to acquire source domain data with annotations, target domain data without annotations, and a student model;

[0145] A training module 704 is configured to train a student model based on the source domain data to obtain a structure and weight of the trained student model;

[0146] Building a model 706, for building a teacher model based on the structure and weights of the trained student model;

[0147] A generating module 708 is configured to input the target domain data into the teacher model to obtain pseudo labels for the target domain data;

[0148] The detection module 710 is used to input the pseudo label into the trained student model for training to obtain an optimized student model, update the model parameters of the teacher model based on the model parameters of the student model, and perform target detection based on the updated teacher model.

[0149] In one embodiment, training the student model based on the source domain data includes:

[0150] The training of the student model based on the source domain data includes:

[0151] extracting instance features from the source domain data based on the student model;

[0152] For any multiple instance features in the same category, calculate the distance between the instance features;

[0153] In the case where the distance between the instance features is less than a preset distance, the feature instances whose distance between the instance features is less than the preset distance are input into the same clustering set to form a category multi-prototype structure.

[0154] In one embodiment, the apparatus further comprises:

[0155] The pseudo labels are modified based on the category multi-prototype structure and virtual instance features are generated.

[0156] In one embodiment, the pseudo-label is corrected in the following manner:

[0157] Extract instance features of each target instance in the image;

[0158] For each category, the spatial distance between all instance features of each category and all prototypes in each category is calculated;

[0159] The spatial distance is calculated according to the following formula:

[0160] ;

[0161] in represents the j-th prototype in category k, represents the j-th instance feature in category k, Represents the mapping function, when When the content is true, ,otherwise , Represents the distance threshold between instance features and category multi-prototypes;

[0162] The instance features that meet the distance threshold are used to predict pseudo labels.

[0163] In one embodiment, the apparatus further comprises:

[0164] Initialize the parameters of the feature point classification branch newly added to the student model based on the source domain data to obtain the initialization loss function ;

[0165] in, Represents the nth feature point in category k in a batch of features, represents the feature point classifier output of the student model, represents the true classification label;

[0166] Generate virtual instance features based on the category multi-prototype structure, and obtain the feature consistency loss function based on the output results of the student model, the output results of the teacher model, and the virtual instance features. , ,

[0167] in, Represents the nth feature point in category k in the virtual instance feature, represents a Gaussian distribution, represents the mean of the i-th prototype within the class k, Represents the variance between feature points in a batch, represents the output of the student model, represents the output of the teacher model, represents the cross entropy loss function.

[0168] In one embodiment, after obtaining the optimized student model, the apparatus further comprises:

[0169] The weight of the optimized student model is obtained, and the weight of the teacher model is updated based on the exponential moving average algorithm.

[0170] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0171] Each module in the target detection device described above can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0172] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in Figure 8. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store images. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a target detection method is implemented.

[0173] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure may be as shown in Figure 9. The computer device includes a processor, memory, a communication interface, a display screen, and an input device connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal via wired or wireless communication. The wireless communication may be achieved via Wi-Fi, a mobile cellular network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a target detection method. The display screen of the computer device may be a liquid crystal display or an electronic ink display. The input device of the computer device may be a touch screen layer covering the display screen, keys, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse.

[0174] Those skilled in the art will understand that the structures shown in Figures 8 and 9 are merely block diagrams of partial structures related to the scheme of the present disclosure, and do not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

[0175] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any embodiment of the present disclosure is implemented.

[0176] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method described in any embodiment of the present disclosure is implemented.

[0177] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, database, or other media used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, and the like.

[0178] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The above-described embodiments merely represent several implementation methods of the present disclosure. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present disclosure. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present disclosure, all of which fall within the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be determined by the appended claims.

Claims

1. A target detection method, characterized in that: The method comprises: Obtain annotated source domain data, unannotated target domain data, and student models; Training the student model based on the source domain data includes: Extracting instance features in the source domain data based on the student model; For any multiple instance features in the same category, calculate the distance between the instance features; ， ； in represents the jth instance feature in category k, represents the mth instance feature in category k, represents the i-th clustering set of all instance features in the k-th category that meet the screening conditions, Represents the mapping function, when When the content is true, ,otherwise , Indicates the distance threshold between features; In the case where the distance between the instance features is less than a preset distance, the feature instances whose distance between the instance features is less than the preset distance are input into the same clustering set to form a category multi-prototype structure; Training a student model based on the source domain data to obtain a structure and a weight of the trained student model; Building a teacher model based on the structure and weights of the trained student model; Inputting the target domain data into the teacher model to obtain a pseudo label of the target domain data; Modifying the pseudo-label and generating virtual instance features based on the category multi-prototype structure; The pseudo-labels are corrected as follows: Extract instance features of each target instance in the image; For each category, the spatial distance between all instance features of each category and all prototypes in each category is calculated; The spatial distance is calculated according to the following formula: = ； in represents the i-th prototype in category k, represents the jth instance feature in category k, Represents the mapping function, when When the content is true, ,otherwise , Represents the distance threshold between instance features and category multi-prototypes; Predicting pseudo labels for instance features that meet the distance threshold; The pseudo labels and target domain data are input into the trained student model for training to obtain an optimized student model, the model parameters of the teacher model are updated based on the model parameters of the optimized student model, and target detection is performed based on the updated teacher model.

2. The method according to claim 1, characterized in that The method further comprises: Based on the source domain data, the parameters of the feature point classification branch newly added to the student model are initialized to obtain the initialization loss function ; in, Represents the nth feature point in category k in a batch of features. represents the feature point classifier output of the student model, Represents the true classification label; Generate virtual instance features based on the category multi-prototype structure, and obtain the feature consistency loss function based on the output results of the student model, the output results of the teacher model, and the virtual instance features. , ; in, represents the nth feature point in category k in the virtual instance feature, represents a Gaussian distribution, represents the mean of the prototype within the i-th class of the prototype of category k, Represents the variance between feature points in a batch, represents the output of the student model, represents the output of the teacher model, represents the cross entropy loss function.

3. The method according to claim 1, characterized in that: Updating the model parameters of the teacher model based on the optimized model parameters of the student model includes: The model parameters of the optimized student model are obtained, and the model parameters of the teacher model are updated based on the exponential moving average algorithm.

4. A target detection device, characterized in that: The device comprises: The acquisition module is used to acquire source domain data with annotations, target domain data without annotations, and a student model; training the student model based on the source domain data includes: Extracting instance features in the source domain data based on the student model; For any multiple instance features in the same category, calculate the distance between the instance features; ， ； in represents the jth instance feature in category k, represents the mth instance feature in category k, represents the i-th clustering set of all instance features in the k-th category that meet the screening conditions, Represents the mapping function, when When the content is true, ,otherwise , Indicates the distance threshold between features; In the case where the distance between the instance features is less than a preset distance, the feature instances whose distance between the instance features is less than the preset distance are input into the same clustering set to form a category multi-prototype structure; A training module, used for training a student model based on the source domain data to obtain a structure and a weight of the trained student model; Constructing a model for constructing a teacher model based on the structure and weights of the trained student model; A generation module, used for inputting the target domain data into the teacher model to obtain a pseudo label of the target domain data; Modifying the pseudo-label and generating virtual instance features based on the category multi-prototype structure; The pseudo-labels are corrected as follows: Extract instance features of each target instance in the image; For each category, the spatial distance between all instance features of each category and all prototypes in each category is calculated; The spatial distance is calculated according to the following formula: = ； in represents the i-th prototype in category k, represents the jth instance feature in category k, Represents the mapping function, when When the content is true, ,otherwise , Represents the distance threshold between instance features and category multi-prototypes; Predicting pseudo labels for instance features that meet the distance threshold; The detection module is used to input the pseudo-label into the trained student model for training to obtain an optimized student model, update the model parameters of the teacher model based on the model parameters of the student model, and perform target detection based on the updated teacher model.

5. The device according to claim 4, characterized in that The device also includes: Based on the source domain data, the parameters of the feature point classification branch newly added to the student model are initialized to obtain the initialization loss function ; in, Represents the nth feature point in category k in a batch of features. represents the feature point classifier output of the student model, Represents the true classification label; Generate virtual instance features based on the category multi-prototype structure, and obtain the feature consistency loss function based on the output results of the student model, the output results of the teacher model, and the virtual instance features. , ; in, represents the nth feature point in category k in the virtual instance feature, represents a Gaussian distribution, represents the mean of the prototype within the i-th class of the prototype of category k, Represents the variance between feature points in a batch, represents the output of the student model, represents the output of the teacher model, represents the cross entropy loss function.

6. The device according to claim 4, characterized in that Updating the model parameters of the teacher model based on the optimized model parameters of the student model includes: The model parameters of the optimized student model are obtained, and the model parameters of the teacher model are updated based on the exponential moving average algorithm.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images