Training method and device of target detector, target detection method, device and equipment

Abstract

The application provides a target detector training method, a target detection method, a device and equipment. The training method comprises: inputting a target image in a target data set in a plurality of data sets into a first feature extraction network to obtain a first feature image; inputting the first feature image into a candidate detection head to obtain a target score of a candidate target and a candidate box; determining a target score loss according to the target score and determining a candidate box loss according to the candidate box; inputting the candidate box and the first feature image into a second feature extraction network to obtain a second feature image corresponding to the candidate box; inputting the second feature image into a detection head to obtain a unified class score of a detection target and a plurality of detection boxes; selecting a target detection box corresponding to a class in a label space of the target data set from the plurality of detection boxes; determining a class score loss according to the unified class score; determining a detection box loss according to the target detection box; and optimizing the target detector according to the losses.

Description

Technical Field

[0001] This invention relates to the field of computer vision target detection technology, and in particular to a training method for a target detector, a target detection method, an apparatus, and a device. Background Technology

[0002] Traditional object detectors are trained and tested on a single dataset, limiting their ability to detect objects of the same labeled category within that dataset, which has significant limitations in practical applications. In multi-dataset fusion object detection, multiple datasets are needed to train the detector, enabling it to detect objects of labeled categories across multiple datasets. However, multi-dataset fusion training suffers from missing labeling issues; each dataset may only label a portion of the object categories, leaving unlabeled categories. During training, the detector may mistakenly treat these unlabeled categories as background noise, thus compromising the performance of the trained detector. Summary of the Invention

[0003] This invention provides a training method, object detection method, apparatus, and device for an object detector, which addresses the problem that during the training of an object detector using a multi-dataset fusion dataset, errors in unlabeled objects are easily mistaken for background, thus failing to guarantee the performance of the trained object detector.

[0004] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0005] In a first aspect, embodiments of the present invention provide a method for training a target detector, the target detector comprising a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the method comprising:

[0006] Obtain multiple datasets;

[0007] The target image from the target dataset in the multiple datasets is input into the first feature extraction network to obtain the first feature image;

[0008] The first feature image is input into the candidate detection head to obtain the target score and candidate bounding box of the candidate target; the target score loss of the candidate detection head is determined based on the target score, and the candidate bounding box loss of the candidate detection head is determined based on the candidate bounding box.

[0009] The candidate box and the first feature image are input into the second feature extraction network to obtain the second feature image corresponding to the candidate box;

[0010] The second feature image is input into the detection head to obtain a unified category score of the detected target and multiple detection boxes; from the multiple detection boxes, the target detection box corresponding to the category in the label space of the target dataset is selected; the detection head category score loss is determined based on the unified category score; the detection head detection box loss is determined based on the target detection box.

[0011] The target detector is optimized based on the candidate target score loss, the candidate bounding box loss, the target category score loss, and the target bounding box loss to obtain the optimized target detector.

[0012] Optionally, each dataset includes multiple images and corresponding labels for the images, wherein the labels include the category of the labeled target and a detection box.

[0013] Optionally, the candidate detection head outputs multiple target scores for the candidate targets, the number of which is the same as the number of datasets. Each target score corresponds to a dataset, representing the probability that the candidate target is a labeled target in the corresponding dataset.

[0014] The candidate detection head target score loss is determined based on the target score, including:

[0015] Obtain the target score corresponding to the target dataset from the target scores of the candidate targets;

[0016] The target score loss of the candidate detection head is determined based on the target score of the corresponding target dataset of the candidate target.

[0017] Optionally, the unified category score includes multiple category scores, each category score corresponding to a category in a unified label space, the unified label space recording the categories of all labeled targets in the dataset.

[0018] Optionally, determining the detection head category score loss based on the unified category score includes:

[0019] The unified category score is mapped to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset;

[0020] The detection head category score loss is determined based on the category scores of the dataset.

[0021] Optionally, for each candidate target, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0022] Optionally, for each candidate target, the detection output consists of K detection boxes, each of which corresponds to one or more categories in the unified label space.

[0023] Optionally, the method further includes:

[0024] An initial target detector is obtained, and the detection head of the initial target detector outputs C detection boxes for each candidate target, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0025] Cluster the features corresponding to the C categories of the last fully connected layer of the detection head to obtain features of K categories;

[0026] In this case, categories within the same cluster share a single detection box;

[0027] The target detector obtained after clustering outputs K detection boxes for each candidate target; the target detector obtained after clustering is used as the target detector to be trained.

[0028] Optionally, the detection head includes, in sequence, a first fully connected layer, a ReLU layer, a second fully connected layer, a ReLU layer, and a third fully connected layer.

[0029] Secondly, embodiments of the present invention provide a method for training a target detector, the target detector comprising a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the method comprising:

[0030] Obtain multiple datasets;

[0031] The target detector to be trained is copied to obtain the student detector and the teacher detector;

[0032] The target images from the target datasets in the multiple datasets are input into the teacher detector to obtain the first detection target. Pseudo-labeling is performed on the first detection target that has not been manually labeled.

[0033] The target image is input into the first feature extraction network of the student detector to obtain a first feature image; the first feature image is input into the candidate detection head of the student detector to obtain the target score and candidate bounding box of the candidate target; the target score loss of the candidate detection head is determined based on the target score, and the candidate bounding box loss of the candidate detection head is determined based on the candidate bounding box; the candidate bounding box and the first feature image are input into the second feature extraction network of the student detector to obtain a second feature image corresponding to the candidate bounding box; the second feature image is input into the detection head of the student detector to obtain the unified category score of the detected target and multiple detection boxes; from the multiple detection boxes, the target detection box corresponding to the category in the label space of the target dataset is selected; the detection head category score loss and pseudo-labeling loss are determined based on the unified category score; the detection head detection box loss is determined based on the target detection box;

[0034] The target detector is optimized based on the candidate head target score loss, the candidate head candidate box loss, the detection head category score loss, the detection head detection box loss, and the pseudo-labeling loss to obtain the optimized target detector.

[0035] Optionally, each dataset includes multiple images and corresponding labels for the images, wherein the labels include the category of the labeled target and a detection box.

[0036] Optionally, the candidate detection head outputs multiple target scores for the candidate targets, the number of which is the same as the number of datasets. Each target score corresponds to a dataset, representing the probability that the candidate target is a labeled target in the corresponding dataset.

[0037] The candidate detection head target score loss is determined based on the target score, including:

[0038] Obtain the target score corresponding to the target dataset from the target scores of the candidate targets;

[0039] The target score loss of the candidate detection head is determined based on the target score of the corresponding target dataset of the candidate target.

[0040] Optionally, the unified category score includes multiple category scores, each category score corresponding to a category in a unified label space, the unified label space recording the categories of all labeled targets in the dataset.

[0041] Optionally, determining the detection head category score loss and pseudo-labeling loss based on the unified category score includes:

[0042] The unified category score is mapped to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset;

[0043] Determine the detection head category score loss based on the category scores of the dataset;

[0044] The pseudo-label loss is determined based on the pseudo-class scores in the unified category scores, excluding the dataset category scores.

[0045] Optionally, for each candidate target, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0046] Optionally, for each candidate target, the detection output consists of K detection boxes, each of which corresponds to one or more categories in the unified label space.

[0047] Optionally, the method further includes:

[0048] The teacher detector is periodically updated using the student detector.

[0049] Optionally, the periodic updating of the teacher detector using the student detector includes:

[0050] Once the learning rate of the target detector meets a preset condition, the teacher detector is updated using the student detector.

[0051] Optionally, the preset condition is that the learning rate of the optimized target detector reaches the minimum point of the Cosine function.

[0052] Thirdly, embodiments of the present invention provide a target detection method applied to a target detector, the target detector comprising a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the method comprising:

[0053] The image to be detected is input into the first feature extraction network to obtain the first feature image;

[0054] The first feature image is input into the candidate detection head to obtain the target score and candidate bounding box of the candidate target.

[0055] The candidate box and the first feature image are input into the second feature extraction network to obtain the second feature image corresponding to the candidate box;

[0056] The second feature image is input into the detection head to obtain the unified category score and detection box of the detected target. The unified category score includes multiple category scores, each category score corresponds to a category in a unified label space, and the unified label space records the categories of targets labeled in multiple datasets.

[0057] Optionally, the step of inputting the first feature image into the candidate detection head to obtain the target score and candidate bounding box of the candidate target includes:

[0058] The candidate detection head outputs multiple target scores for candidate targets, and the target score with the highest score is selected as the target score of the candidate target.

[0059] Fourthly, embodiments of the present invention provide a training apparatus for an object detector, the object detector comprising a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the training apparatus for the object detector comprising:

[0060] The first acquisition module is used to acquire multiple datasets;

[0061] The first feature extraction module is used to input the target image from the target dataset in the multiple datasets into the first feature extraction network to obtain the first feature image;

[0062] The candidate target detection module is used to input the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target; determine the target score loss of the candidate detection head based on the target score; and determine the candidate box loss of the candidate detection head based on the candidate box.

[0063] The second feature extraction module is used to input the candidate box and the first feature image into the second feature extraction network to obtain the second feature image corresponding to the candidate box;

[0064] The target detection module is used to input the second feature image into the detection head to obtain a unified category score of the detected target and multiple detection boxes; select the target detection box corresponding to the category in the label space of the target dataset from the multiple detection boxes; determine the detection head category score loss based on the unified category score; and determine the detection head detection box loss based on the target detection box.

[0065] The first optimization module is used to optimize the target detector based on the target score loss of the candidate detection head, the candidate bounding box loss of the candidate detection head, the category score loss of the detection head, and the detection bounding box loss of the detection head, so as to obtain the optimized target detector.

[0066] Fifthly, embodiments of the present invention provide a training apparatus for an object detector, the object detector comprising a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the training apparatus for the object detector comprising:

[0067] The first acquisition module is used to acquire multiple datasets;

[0068] The copy module is used to copy the target detector to be trained, resulting in a student detector and a teacher detector.

[0069] The pseudo-annotation module is used to input the target images from the target datasets in the multiple datasets into the teacher detector to obtain the first detection target, and to perform pseudo-annotation on the first detection target that has not been manually annotated.

[0070] The training module is used to input the target image into the first feature extraction network of the student detector to obtain a first feature image; input the first feature image into the candidate detection head of the student detector to obtain the target score and candidate bounding box of the candidate target; determine the target score loss of the candidate detection head based on the target score, and determine the candidate bounding box loss of the candidate detection head based on the candidate bounding box; input the candidate bounding box and the first feature image into the second feature extraction network of the student detector to obtain the second feature image corresponding to the candidate bounding box; input the second feature image into the detection head of the student detector to obtain the unified category score of the detected target and multiple detection boxes; select the target detection box corresponding to the category in the label space of the target dataset from the multiple detection boxes; determine the detection head category score loss and pseudo-labeling loss based on the unified category score; determine the detection head detection box loss based on the target detection box; and optimize the target detector based on the target score loss of the candidate detection head, the candidate bounding box loss of the candidate detection head, the category score loss of the detection head, the detection head detection box loss, and the pseudo-labeling loss to obtain an optimized target detector.

[0071] Sixthly, embodiments of the present invention provide a target detection device that uses a target detector to detect targets. The target detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head. The target detection device includes:

[0072] The first feature extraction module is used to input the image to be detected into the first feature extraction network of the target detector to obtain the first feature image;

[0073] The candidate target detection module is used to input the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target;

[0074] The second feature extraction module is used to input the candidate box and the first feature image into the second feature extraction network to obtain the second feature image corresponding to the candidate box;

[0075] The target detection module is used to input the second feature image into the detection head to obtain the unified category score and detection box of the detected target. The unified category score includes multiple category scores, each category score corresponds to a category in a unified label space, and the unified label space records the categories of targets labeled in multiple datasets.

[0076] In a seventh aspect, embodiments of the present invention provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the training method for the target detector as described in the first or second aspect above, or, when the program is executed by the processor, it implements the steps of the target detection method as described in the third aspect above.

[0077] Eighthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, wherein when executed by a processor, the computer program implements the steps of the training method for the target detector as described in the first or second aspect above; or, when executed by a processor, the computer program implements the steps of the target detection method as described in the third aspect above.

[0078] In this embodiment of the invention, candidate boxes are first coarsely detected using a candidate detection head, and then the detection head is used to finely detect the detection boxes of the target and the unified category score. That is, a two-stage target detection is adopted, which can improve the detection accuracy of the trained target detector. In addition, the number of categories in multi-dataset fusion training is usually large, and it is difficult to achieve accurate localization of multi-category detection boxes using a single detection box regression module. In this embodiment of the invention, by designing a multi-detection box detection head, accurate localization of multi-category detection boxes can be achieved. Attached Figure Description

[0079] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0080] Figure 1 This is a schematic flowchart of a target detector training method according to an embodiment of the present invention;

[0081] Figure 2 This is a schematic diagram of a two-stage target detection method according to an embodiment of the present invention;

[0082] Figure 3 This is a schematic diagram of the target detector architecture according to an embodiment of the present invention;

[0083] Figure 4 This is a schematic diagram of the detection head according to an embodiment of the present invention;

[0084] Figure 5 This is a flowchart illustrating a training method for a target detector according to another embodiment of the present invention;

[0085] Figure 6 This is a schematic diagram of the training method for the online pseudo-labeled target detector according to an embodiment of the present invention;

[0086] Figure 7 This is a schematic diagram of the target detection method according to an embodiment of the present invention;

[0087] Figure 8 This is a schematic diagram of the structure of a training device for a target detector according to an embodiment of the present invention;

[0088] Figure 9 This is a schematic diagram of the structure of a training device for a target detector according to another embodiment of the present invention;

[0089] Figure 10 This is a schematic diagram of the target detection device according to an embodiment of the present invention;

[0090] Figure 11 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0091] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0092] Please refer to Figure 1 , Figure 2 and Figure 3 This invention provides a method for training a target detector, the target detector comprising a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the method comprising:

[0093] Step 11: Obtain multiple datasets.

[0094] In this embodiment of the invention, each dataset includes multiple images and corresponding labels for the images, wherein the labels include the category of the labeled target and a detection box.

[0095] In this embodiment of the invention, the categories of labeled targets in different datasets can be completely different or partially overlapping. For example, the categories of labeled targets in dataset D1 include: people, cats, dogs, trees, and houses, etc., while the categories of labeled targets in dataset D2 include: people, trees, flowers, birds, etc. Datasets D1 and D2 have overlapping categories: people and trees.

[0096] The detection box in the label is data representing the position and / or size of the detection box, which may include the center coordinates (x, y) of the detection box, as well as the width w and height h of the detection box. In some embodiments, it may also include the coordinate positions of the four endpoints of the detection box, etc.

[0097] Figure 2 and Figure 3 In the example shown, there are 3 datasets, but the number of datasets is not limited to this.

[0098] Step 12: Input the target image from the target dataset in the multiple datasets into the first feature extraction network to obtain the first feature image;

[0099] Optionally, the target dataset can be any of the plurality of datasets. The target detector can be trained sequentially using images from the plurality of datasets.

[0100] The first feature extraction network can be a backbone network for feature extraction, such as a CNN backbone network like ResNet or MobileNet, or a Transformer backbone network like SWIN. The backbone network extracts general features of the image.

[0101] Step 13: Input the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target; determine the target score loss of the candidate detection head based on the target score, and determine the candidate box loss of the candidate detection head based on the candidate box;

[0102] In this embodiment of the invention, the candidate detection head is used to detect the candidate bounding boxes of candidate targets that may be real targets, and output the target score of the candidate target and the position information of the candidate bounding box, without needing to determine the category of the candidate target.

[0103] The target score represents the probability that the candidate target is the real target.

[0104] In this embodiment of the invention, the loss of the candidate detection head includes: the target score loss of the candidate detection head and the candidate box loss of the candidate detection head.

[0105] Among them, the candidate detection head target score loss is determined according to the target score, specifically: the candidate detection head target score loss can be calculated according to the target score of the candidate target and the labeled target in the label corresponding to the target image.

[0106] Among them, the candidate detection head candidate box loss is determined according to the candidate box, specifically: the candidate detection head candidate box loss can be calculated according to the candidate box of the candidate target and the true detection box in the label corresponding to the target image.

[0107] When calculating the loss of the candidate detection head, corresponding weights can be assigned to the candidate detection head target score loss and the candidate detection head candidate box loss respectively, and then the sum of the two after weighting is calculated. For example, the weight q(0 < q < 1) is assigned to the candidate detection head target score loss, and the weight 1 - q is assigned to the candidate detection head candidate box loss. At this time, the loss of the candidate detection head can be: q * candidate detection head target score loss+(1 - q) * candidate detection head candidate box loss.

[0108] Step 14: Input the candidate box and the first feature image into the second feature extraction network to obtain the second feature image corresponding to the candidate box;

[0109] In the embodiment of the present invention, the second feature extraction network (which can also be called ROIPooling) is used to extract the features of the candidate target. Specifically, it can be to intercept the screenshot corresponding to the candidate box in the first feature image by using the candidate box to obtain the second feature image. Optionally, the second feature extraction network can use the ROIPooling module of FasterRCNN to extract the features of the candidate target.

[0110] Step 15: Input the second feature image into the detection head to obtain the unified class score of the detection target and multiple detection boxes; from the multiple detection boxes, select the target detection box corresponding to the class in the label space of the target dataset; determine the detection head class score loss according to the unified class score; determine the detection head detection box loss according to the target detection box;

[0111] In the embodiment of the present invention, the unified class score includes multiple class scores, each class score corresponding to a class in the unified label space, and the unified label space records all the classes of the labeled targets in the dataset.

[0112] In the embodiment of the present invention, assume that the number of datasets for training the target detector is N, and the N datasets are respectively denoted as D1, D2, …, D N , and the label spaces corresponding to the N datasets are respectively denoted as L1, L2, …, L N, the label space is used to record the labels of the data sets. The label spaces of N data sets are merged to obtain a unified label space. For the categories in the labels, the merging can be to merge the same categories in the data sets and retain the different categories. The unified label space can be denoted as L, and the label space L of each data set i is a subset of L, and the label space L of each data set i has a mapping correspondence with the categories in the unified label space L, and L i always maps a certain category to the corresponding category in L. For example, suppose there are three data sets. The label space of data set D1 includes 5 categories, the label space of data set D2 includes 6 categories, and the label space of data set D3 includes 7 categories. The categories in the unified label space can be 15 (the same categories in the 3 data sets are merged). For example, the mapping correspondence between the unified label space L and the label space L i can be, for example: the 1st, 2nd, 4th, 6th, and 10th categories in the unified label space L correspond one-to-one with the 5 categories in data set D1.

[0113] In an embodiment of the present invention, the detection head can be obtained by improving on the basis of the Faster RCNN detection head.

[0114] In an embodiment of the present invention, the losses of the detection head include: the detection head category score loss and the detection head detection box loss.

[0115] In an embodiment of the present invention, optionally, determining the detection head category score loss according to the unified category score may include: calculating the detection head category score loss according to the unified category score of the detection target and the true category of the labeled target in the target image.

[0116] In an embodiment of the present invention, optionally, determining the detection head detection box loss according to the target detection box may include: calculating the detection head detection box loss according to the target detection box and the detection box of the labeled target in the target image.

[0117] When calculating the loss of the detection head, corresponding weights can be assigned to the detection head category score loss and the detection head detection box loss respectively, and then the sum of the two after weighting is calculated. For example, a weight p (0 < p < 1) is assigned to the detection head category score loss, and a weight 1 - p is assigned to the detection head detection box loss. At this time, the loss of the detection head can be: p * detection head category score loss + (1 - p) * detection head detection box loss.

[0118] In an embodiment of the present invention, when calculating the candidate detection head detection box loss, the Soft L1 loss function can be used. Of course, it is not limited to this, and other types of loss functions can also be used.

[0119] Step 16: Optimize the target detector based on the candidate target score loss, the candidate bounding box loss, the target category score loss, and the target bounding box loss to obtain the optimized target detector.

[0120] In this embodiment of the invention, the candidate detection head target score loss and candidate detection head candidate box loss can also be referred to as candidate detection head loss, and the detection head category score loss and detection head detection box loss can also be referred to as detection head loss. The total loss of the target detector can be the sum of the candidate detection head loss and the detection head loss, or a weighted sum of the two.

[0121] In this embodiment of the invention, images from multiple datasets can be sequentially input into the target detector to train the target detector. For each input training image, steps 12-16 are executed until training ends. The conditions for training to end can be that all images in the datasets have participated in the training, or that the trained target detector meets the set conditions, etc.

[0122] In this embodiment of the invention, candidate boxes are first coarsely detected using a candidate detection head, and then the detection head is used to finely detect the detection boxes of the target and the unified category score. That is, a two-stage target detection is adopted, which can improve the detection accuracy of the trained target detector. In addition, the number of categories in multi-dataset fusion training is usually large, and it is difficult to achieve accurate localization of multi-category detection boxes using a single detection box regression module. In this embodiment of the invention, by designing a multi-detection box detection head, accurate localization of multi-category detection boxes can be achieved.

[0123] Optionally, in this embodiment of the invention, please refer to... Figure 3 The candidate detection head outputs multiple target scores for the candidate targets, the number of which is the same as the number of data sets. Each target score corresponds to a data set, representing the probability that the candidate target is a labeled target in the corresponding data set. In step 13 above, determining the target score loss of the candidate detection head based on the target scores includes:

[0124] Step 131: Obtain the target score corresponding to the target dataset from the target scores of the candidate targets;

[0125] Step 132: Determine the target score loss of the candidate detection head based on the target score of the corresponding target dataset of the candidate target.

[0126] Specifically, the target score loss of the candidate detection head can be determined based on the target score of the candidate target in the target dataset and the labeled targets in the target image.

[0127] by Figure 3 Taking an example, the candidate detection head outputs three target scores, O1, O2, and O3, corresponding to datasets D1, D2, and D3, respectively. Assuming the target image input to the target detector belongs to dataset D1, if the candidate detection head target score loss is calculated by comparing the scores of targets not corresponding to dataset D1 with the labeled targets of the target image in dataset D1, it will inevitably be inaccurate, mistaking unlabeled targets in dataset D1 as background. In this embodiment of the invention, the candidate detection head target score loss is calculated based on the target score O1 corresponding to dataset D1 and the labeled targets of the target image in dataset D1, thus avoiding the misclassification of unlabeled targets in the target dataset as background.

[0128] In this embodiment of the invention, optionally, determining the detection head category score loss based on the unified category score includes:

[0129] Step 151: Map the unified category score to the dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset;

[0130] Step 152: Determine the detection head category score loss based on the category scores of the dataset.

[0131] The following example illustrates how to map the unified category score to the dataset category score. Assuming there are 15 categories in the unified label space L, the detection head can output 15 unified category scores for a detected target, each corresponding to one of the 15 categories in the unified label space L. In other words, a score is assigned to each category in the unified label space L. Since the target dataset containing the target image currently input to the target detector only labels a portion of the categories—for example, if the target image belongs to dataset D1, and dataset D1 only labels targets in 5 categories—it would be inaccurate to calculate the loss of the detection head's category score using the scores of all 15 categories and the labels in dataset D1. This would result in mistaking unlabeled targets in dataset D1 as background. To overcome this problem, in this embodiment of the invention, the unified category score is mapped to the dataset score corresponding to dataset D1. The mapping method is as follows: assuming that the 1st, 2nd, 4th, 6th, and 10th categories in the unified label space L correspond one-to-one with the 5 categories in dataset D1, the 1st, 2nd, 4th, 6th, and 10th scores in the unified category score are extracted as the dataset category score of dataset D1. The detection head category score loss is calculated based on the dataset category score of dataset D1 and the labels in dataset D1 to avoid mistaking unlabeled targets in the target dataset as background.

[0132] Since the unified category score includes the scores of all categories in the dataset, while the target dataset containing the current training image only labels some categories, it is inaccurate to use the unified category score to calculate the detection head category score loss. This can easily lead to mistaking unlabeled targets in the target dataset as background. In this invention, the unified category score is mapped to the dataset score of the target dataset containing the current training image. The detection head category score loss is calculated based on the dataset score, which can avoid mistaking unlabeled targets in the target dataset as background, thereby improving the detection accuracy of the trained object detector and enhancing its performance.

[0133] In some embodiments of the present invention, optionally, for each candidate target, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space. That is, each category in the unified label space has a dedicated detection box output.

[0134] In some embodiments of the present invention, optionally, for each candidate target, the detection output includes K detection boxes, each corresponding to one or more categories in the unified label space. That is, not every category in the unified label space corresponds to a single detection box output; some different categories can share a single detection box, thereby reducing the number of detection boxes output by the detection head. For example, categories such as cows, horses, dogs, and cats have similar image features and similar detection box boundaries, and can therefore correspond to the same detection box. Conversely, categories with large differences in detection box boundaries are not suitable for sharing a single detection box.

[0135] The following section will explain how to determine the correspondence between detection boxes and categories, that is, how to determine which categories can share a detection box.

[0136] Optionally, in embodiments of the present invention, the training method for the target detector further includes:

[0137] Step 01: Obtain the initial target detector. The detection head of the initial target detector outputs C detection boxes for each candidate target, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0138] Step 02: Cluster the features corresponding to the C categories of the last fully connected layer of the detection head to obtain features of K categories, wherein categories in the same cluster share a detection box; the detection head of the target detector obtained after clustering outputs K detection boxes for each candidate target; the target detector obtained after clustering is used as the target detector to be trained.

[0139] Optionally, a feature clustering algorithm such as KMeans can be used to cluster the features corresponding to the C categories of the last fully connected layer of the detection head.

[0140] Please refer to Figure 4 In one embodiment of the present invention, the detection head includes, in sequence, a first fully connected layer (FC1), a ReLU layer, a second fully connected layer (FC2), a ReLU layer, and a third fully connected layer (FC3). The above clustering process can be as follows: the input to the last fully connected layer FC3 of the detection head is N. in =1024-dimensional vector, the output is N out = 4C-dimensional vector. The weight matrix W of FC3 has a dimension of 1024×4C, and the offset matrix B of FC3 has a dimension of 1024×C. Combining them yields a coefficient matrix of 1024×5C, which is then reorganized to obtain a parameter matrix P of 5120×C. This matrix can be regarded as a 5120-dimensional feature corresponding to each category. Using a feature clustering algorithm (such as KMeans), C categories can be clustered into K categories, corresponding to K detection boxes. Categories in the same cluster share a detection box.

[0141] Please refer to Figure 5 The present invention also provides a method for training an object detector, the object detector comprising a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the method comprising:

[0142] Step 51: Obtain multiple datasets;

[0143] Optionally, each dataset includes multiple images and corresponding labels for the images, wherein the labels include the category of the labeled target and a detection box.

[0144] Step 52: Duplicate the target detector to be trained to obtain the student detector and the teacher detector;

[0145] Step 53: Input the target images from the target datasets in the multiple datasets into the teacher detector to obtain the first detection target, and perform pseudo-annotation on the first detection target that has not been manually annotated;

[0146] Step 54: Input the target image into the first feature extraction network of the student detector to obtain a first feature image; input the first feature image into the candidate detection head of the student detector to obtain the target score and candidate bounding box of the candidate target; determine the target score loss of the candidate detection head based on the target score, and determine the candidate bounding box loss of the candidate detection head based on the candidate bounding box; input the candidate bounding box and the first feature image into the second feature extraction network of the student detector to obtain a second feature image corresponding to the candidate bounding box; input the second feature image into the detection head of the student detector to obtain the unified category score of the detected target and multiple detection boxes; select the target detection box corresponding to the category in the label space of the target dataset from the multiple detection boxes; determine the detection head category score loss and pseudo-labeling loss based on the unified category score; determine the detection head detection box loss based on the target detection box.

[0147] Step 55: Optimize the target detector based on the candidate target score loss, the candidate bounding box loss, the target category score loss, the target bounding box loss, and the pseudo-labeling loss to obtain the optimized target detector.

[0148] In multi-dataset training, each dataset is only labeled with its corresponding label space L. i The dataset contains targets, but not all targets in the unified label space L are labeled, indicating some targets are missing. To maximize the use of the images in the dataset and provide as many labeled targets as possible for training, this embodiment employs a teacher detector to detect images in the dataset and automatically label the missing labels. To distinguish these labels from the manually labeled real labels in the dataset, the labels labeled by the teacher detector are called pseudo-labels. The pseudo-label loss is calculated using the pseudo-labels and added to the manually labeled loss to obtain the total loss. This total loss is used to train the student detector, further enriching the cross-scene labeled data and thus improving the detection performance of the trained target detector.

[0149] In this embodiment of the invention, optionally, the candidate detection head outputs multiple target scores for the candidate targets, the number of which is the same as the number of datasets, and each target score corresponds to a dataset, indicating the probability that the candidate target is a labeled target in the corresponding dataset;

[0150] The candidate detection head target score loss is determined based on the target score, including:

[0151] Obtain the target score corresponding to the target dataset from the target scores of the candidate targets;

[0152] The target score loss of the candidate detection head is determined based on the target score of the corresponding target dataset of the candidate target.

[0153] In this embodiment of the invention, optionally, the unified category score includes multiple category scores, each category score corresponding to a category in a unified label space, and the unified label space records the categories of all labeled targets in the dataset.

[0154] In this embodiment of the invention, optionally, determining the detection head category score loss and pseudo-labeling loss based on the unified category score includes:

[0155] The unified category score is mapped to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset;

[0156] Determine the detection head category score loss based on the category scores of the dataset;

[0157] The pseudo-label loss is determined based on the pseudo-class scores in the unified category scores, excluding the dataset category scores.

[0158] by Figure 6 Taking the student detector as an example, the unified category score C of the detected target is output by the student detector. u Assuming the target image is an image from dataset D1, the unified category score C will be... u Mapping to dataset D1 yields dataset class scores C1 and unified class scores C. u The category score C1 in the dataset other than the category score C1 mentioned above. ‘ .

[0159] In this embodiment of the invention, optionally, for each candidate target, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0160] In this embodiment of the invention, optionally, for each candidate target, the detection output consists of K detection boxes, each of which corresponds to one or more categories in the unified label space.

[0161] Optionally, in this embodiment of the invention, the training method for the target detector further includes:

[0162] The teacher detector is periodically updated using the student detector.

[0163] In this embodiment of the invention, optionally, the periodic updating of the teacher detector using the student detector includes:

[0164] Once the learning rate of the target detector meets a preset condition, the teacher detector is updated using the student detector.

[0165] In this embodiment of the invention, optionally, the preset condition is that the learning rate of the optimized target detector reaches the minimum point of the Cosine function.

[0166] In this embodiment of the invention, both the teacher detector and the student detector employ a two-stage target detector with the same structure. During model training, the accuracy of the student detector gradually improves. Therefore, the teacher detector is periodically updated using the student detector, thereby gradually improving the quality of the pseudo-labels annotated by the teacher detector. Optionally, the student detector can be trained using a periodically changing Cosine learning rate. Whenever the learning rate reaches the minimum point of the Cosine function, the teacher detector is updated using the student detector. Practice shows that this periodic updating of the teacher detector achieves higher accuracy than real-time updating.

[0167] The advantages of the target detector training method in this embodiment of the invention will be explained below in conjunction with specific experimental results.

[0168] Experimental Results Comparison: See Table 1. Using the COCO-SPLIT5 dataset to train the object detector, the mAP (average accuracy across all classes) was 39.29 without pseudo-labels, 41.81 with offline pseudo-labels, 41.47 with the traditional semi-supervised learning EMA real-time update method, and the highest mAP (42.57) was achieved using the semi-supervised learning periodic update method proposed in this invention. Experiments verify that the training method of this invention improves accuracy by more than 1 percentage point compared to traditional semi-supervised learning methods, demonstrating a significant improvement in accuracy.

[0169] Table 1

[0170]

[0171] Please refer to Figure 7 This invention also provides a target detection method applied to a target detector. The target detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head. The target detector is trained using the target detector training method described in any of the above embodiments. The target detection method includes:

[0172] Step 71: Input the image to be detected into the first feature extraction network to obtain the first feature image;

[0173] Step 72: Input the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target;

[0174] Step 73: Input the candidate box and the first feature image into the second feature extraction network to obtain the second feature image corresponding to the candidate box;

[0175] Step 74: Input the second feature image into the detection head to obtain the unified category score and detection box of the detected target. The unified category score includes multiple category scores, each category score corresponds to a category in the unified label space, and the unified label space records the categories of targets labeled in multiple datasets.

[0176] In this embodiment of the invention, optionally, the step of inputting the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target includes: the candidate detection head outputs multiple target scores of the candidate target, and selects the target score with the highest score as the target score of the candidate target.

[0177] In this embodiment of the invention, optionally, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0178] In this embodiment of the invention, optionally, the detection output consists of K detection boxes, each of which corresponds to one or more categories in the unified label space.

[0179] In this embodiment of the invention, by designing a multi-frame detection head, accurate positioning of multiple types of detection frames can be achieved.

[0180] The training method and target detection method of the target detector in this invention can be applied to a variety of fields.

[0181] For example, in one embodiment, it can be applied to the field of autonomous driving. In the field of autonomous driving, the target detector used for autonomous driving is required to detect multiple types of targets, such as pedestrians, cars, buses, bicycles, electric vehicles, traffic lights, signs, dogs on the road, obstacles on the road, etc. When training the target detector, since there are many types of targets to be identified, it is difficult for a single dataset to cover all categories. Therefore, it is necessary to merge the label spaces of multiple datasets to obtain a unified label space, and use multiple datasets to train the target detector so that the trained target detector can identify more categories.

[0182] In another embodiment, this can be applied to the field of video surveillance, such as video surveillance in school cafeterias. The target detector used for video surveillance is required to detect multiple types of targets, such as people, pots and pans, mice, vegetables, and seasonings. When training the target detector, because there are many types of targets to be identified, a single dataset cannot cover all categories. Therefore, it is necessary to merge the label spaces of multiple datasets to obtain a unified label space, and use multiple datasets to train the target detector, so that the trained target detector can recognize more categories. Please refer to [reference needed]. Figure 8 This invention also provides a training device 80 for a target detector, the target detector including a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the training device for the target detector including:

[0183] The first acquisition module 81 is used to acquire multiple datasets;

[0184] The first feature extraction module 82 is used to input the target image from the target dataset in the multiple datasets into the first feature extraction network to obtain the first feature image;

[0185] The candidate target detection module 83 is used to input the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target; determine the target score loss of the candidate detection head according to the target score; and determine the candidate box loss of the candidate detection head according to the candidate box.

[0186] The second feature extraction module 84 is used to input the candidate box and the first feature image into the second feature extraction network to obtain the second feature image corresponding to the candidate box;

[0187] The target detection module 85 is used to input the second feature image into the detection head to obtain a unified category score of the detected target and multiple detection boxes; select the target detection box corresponding to the category in the label space of the target dataset from the multiple detection boxes; determine the detection head category score loss based on the unified category score; and determine the detection head detection box loss based on the target detection box.

[0188] The first optimization module 86 is used to optimize the target detector based on the candidate detection head target score loss, the candidate detection head candidate box loss, the detection head category score loss, and the detection head detection box loss, so as to obtain an optimized target detector.

[0189] Optionally, each dataset includes multiple images and corresponding labels for the images, wherein the labels include the category of the labeled target and a detection box.

[0190] Optionally, the candidate detection head outputs multiple target scores for the candidate targets, the number of which is the same as the number of datasets. Each target score corresponds to a dataset, representing the probability that the candidate target is a labeled target in the corresponding dataset.

[0191] The candidate target detection module 83 is used to obtain the target score of the candidate target corresponding to the target dataset from the target scores of the candidate target; and to determine the target score loss of the candidate detection head based on the target score of the candidate target corresponding to the target dataset.

[0192] Optionally, the unified category score includes multiple category scores, each category score corresponding to a category in a unified label space, the unified label space recording the categories of all labeled targets in the dataset.

[0193] Optionally, the target detection module 85 is configured to map the unified category score to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset; and to determine the detection head category score loss based on the dataset category score.

[0194] Optionally, for each candidate target, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0195] Optionally, for each candidate target, the detection output consists of K detection boxes, each of which corresponds to one or more categories in the unified label space.

[0196] Optionally, the training device 80 for the target detector further includes:

[0197] The second acquisition module is used to acquire an initial target detector. The detection head of the initial target detector outputs C detection boxes for each candidate target, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0198] The clustering module is used to cluster the features corresponding to the C categories of the last fully connected layer of the detection head to obtain features of K categories;

[0199] In this case, categories within the same cluster share a single detection box;

[0200] The target detector obtained after clustering outputs K detection boxes for each candidate target; the target detector obtained after clustering is used as the target detector to be trained.

[0201] Optionally, the detection head includes, in sequence, a first fully connected layer, a ReLU layer, a second fully connected layer, a ReLU layer, and a third fully connected layer.

[0202] Please refer to Figure 9 This invention also provides a training device 90 for a target detector, the target detector including a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head, the training device for the target detector including:

[0203] The first acquisition module 91 is used to acquire multiple datasets;

[0204] The copying module 92 is used to copy the target detector to be trained, resulting in a student detector and a teacher detector;

[0205] The pseudo-annotation module 93 is used to input the target image in the target dataset from the multiple datasets into the teacher detector to obtain the first detection target, and to perform pseudo-annotation on the first detection target that has not been manually annotated.

[0206] Training module 94 is used to input the target image into the first feature extraction network of the student detector to obtain a first feature image; input the first feature image into the candidate detection head of the student detector to obtain the target score and candidate bounding box of the candidate target; determine the target score loss of the candidate detection head based on the target score, and determine the candidate bounding box loss of the candidate detection head based on the candidate bounding box; input the candidate bounding box and the first feature image into the second feature extraction network of the student detector to obtain a second feature image corresponding to the candidate bounding box; input the second feature image into the detection head of the student detector to obtain the unified category score of the detected target and multiple detection boxes; select the target detection box corresponding to the category in the label space of the target dataset from the multiple detection boxes; determine the detection head category score loss and pseudo-labeling loss based on the unified category score; determine the detection head detection box loss based on the target detection box; and optimize the target detector based on the target score loss of the candidate detection head, the candidate bounding box loss of the candidate detection head, the category score loss of the detection head, the detection head detection box loss, and the pseudo-labeling loss to obtain an optimized target detector.

[0207] Optionally, each dataset includes multiple images and corresponding labels for the images, wherein the labels include the category of the labeled target and a detection box.

[0208] Optionally, the candidate detection head outputs multiple target scores for the candidate targets, the number of which is the same as the number of datasets. Each target score corresponds to a dataset, representing the probability that the candidate target is a labeled target in the corresponding dataset.

[0209] The training module 94 is used to obtain the target score of the candidate target corresponding to the target dataset from the target scores of the candidate target; and to determine the target score loss of the candidate detection head based on the target score of the candidate target corresponding to the target dataset.

[0210] Optionally, the unified category score includes multiple category scores, each category score corresponding to a category in a unified label space, the unified label space recording the categories of all labeled targets in the dataset.

[0211] Optionally, the training module 94 is configured to map the unified category score to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset; determine the detection head category score loss based on the dataset category score; and determine the pseudo-labeling loss based on the pseudo-category scores other than the dataset category scores in the unified category score.

[0212] Optionally, for each candidate target, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0213] Optionally, for each candidate target, the detection output consists of K detection boxes, each of which corresponds to one or more categories in the unified label space.

[0214] Optionally, the training device 90 for the target detector further includes:

[0215] An update module is used to periodically update the teacher detector using the student detector.

[0216] Optionally, the update module is used to update the teacher detector using the student detector after the learning rate of the target detector meets a preset condition.

[0217] Optionally, the preset condition is that the learning rate of the optimized target detector reaches the minimum point of the Cosine function.

[0218] Please refer to Figure 10 This invention also provides a target detection device 100, which uses a target detector to detect targets. The target detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head. The target detection device 100 includes:

[0219] The first feature extraction module 101 is used to input the image to be detected into the first feature extraction network of the target detector to obtain the first feature image;

[0220] The candidate target detection module 102 is used to input the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target;

[0221] The second feature extraction module 103 is used to input the candidate box and the first feature image into the second feature extraction network to obtain the second feature image corresponding to the candidate box;

[0222] The target detection module 104 is used to input the second feature image into the detection head to obtain the unified category score and detection box of the detected target. The unified category score includes multiple category scores, each category score corresponds to a category in a unified label space, and the unified label space records the categories of targets labeled in multiple datasets.

[0223] Optionally, the candidate detection head outputs multiple target scores for candidate targets, and the candidate target detection module 102 is used to select the target score with the highest score as the target score of the candidate target.

[0224] Optionally, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space.

[0225] Optionally, the detection output consists of K detection boxes, each corresponding to one or more categories in the unified label space.

[0226] Please refer to Figure 11 The present invention also provides an electronic device 110, including a processor 111, a memory 112, and a computer program stored in the memory 112 and executable on the processor 111. When the computer program is executed by the processor 111, it implements the various processes of the above-described target detector training method or target detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0227] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described target detector training method or target detection method embodiment, achieving the same technical effect. To avoid repetition, further details are omitted here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0228] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0229] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0230] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A training method for an object detector, characterized in that, The target detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head; the method includes: Obtain multiple datasets; The target image from the target dataset in the multiple datasets is input into the first feature extraction network to obtain the first feature image; The first feature image is input into the candidate detection head to obtain the target score and candidate bounding box of the candidate target; the target score loss of the candidate detection head is determined based on the target score, and the candidate bounding box loss of the candidate detection head is determined based on the candidate bounding box. The candidate box and the first feature image are input into the second feature extraction network to obtain the second feature image corresponding to the candidate box; The second feature image is input into the detection head to obtain a unified category score of the detected target and multiple detection boxes; from the multiple detection boxes, the target detection box corresponding to the category in the label space of the target dataset is selected; the detection head category score loss is determined based on the unified category score; the detection head detection box loss is determined based on the target detection box. Based on the candidate detection head target score loss, the candidate detection head candidate box loss, the detection head category score loss, and the detection head detection box loss, the target detector is optimized to obtain the optimized target detector. The step of determining the candidate detection head target score loss based on the target score includes: Obtain the target score corresponding to the target dataset from the target scores of the candidate targets; The candidate detection head target score loss is determined based on the target score of the candidate target in the target dataset. The step of determining the detection head category score loss based on the unified category score includes: The unified category score is mapped to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset; The detection head category score loss is determined based on the category scores of the dataset.

2. The method according to claim 1, characterized in that, Each dataset includes multiple images and corresponding labels for the images, the labels including the category of the labeled target and the detection box.

3. The method according to claim 1, characterized in that, The candidate detection head outputs multiple target scores for the candidate targets, the number of which is the same as the number of datasets. Each target score corresponds to a dataset, indicating the probability that the candidate target is a labeled target in the corresponding dataset.

4. The method according to claim 1, characterized in that, The unified category score includes multiple category scores, each category score corresponding to a category in the unified label space, which records the categories of all labeled targets in the dataset.

5. The method according to claim 4, characterized in that, For each candidate target, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to one category in the unified label space.

6. The method according to claim 4, characterized in that, For each candidate target, the detection head outputs K detection boxes, and each detection box corresponds to one or more categories in the unified label space.

7. The method according to claim 6, characterized in that, Also includes: An initial target detector is obtained, and the detection head of the initial target detector outputs C detection boxes for each candidate target, where C is the number of categories in the unified label space, and each detection box corresponds to a category in the unified label space. Cluster the features corresponding to the C categories of the last fully connected layer of the detection head to obtain features of K categories; In this case, categories within the same cluster share a single detection box; The target detector obtained after clustering outputs K detection boxes for each candidate target; the target detector obtained after clustering is used as the target detector to be trained.

8. The method according to claim 1 or 7, characterized in that, The detection head comprises, in sequence: a first fully connected layer, a ReLU layer, a second fully connected layer, a ReLU layer, and a third fully connected layer.

9. A training method for an object detector, characterized in that, The target detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head; the method includes: Obtain multiple datasets; The target detector to be trained is copied to obtain the student detector and the teacher detector; The target images from the target datasets in the multiple datasets are input into the teacher detector to obtain the first detection target. Pseudo-labeling is performed on the first detection target that has not been manually labeled. The target image is input into the first feature extraction network of the student detector to obtain a first feature image; the first feature image is input into the candidate detection head of the student detector to obtain the target score and candidate bounding box of the candidate target; the target score loss of the candidate detection head is determined based on the target score, and the candidate bounding box loss of the candidate detection head is determined based on the candidate bounding box; the candidate bounding box and the first feature image are input into the second feature extraction network of the student detector to obtain a second feature image corresponding to the candidate bounding box; the second feature image is input into the detection head of the student detector to obtain the unified category score of the detected target and multiple detection boxes; from the multiple detection boxes, the target detection box corresponding to the category in the label space of the target dataset is selected; the detection head category score loss and pseudo-labeling loss are determined based on the unified category score; the detection head detection box loss is determined based on the target detection box; Based on the candidate detection head target score loss, the candidate detection head candidate box loss, the detection head category score loss, the detection head detection box loss, and the pseudo-labeling loss, the target detector is optimized to obtain the optimized target detector. The step of determining the candidate detection head target score loss based on the target score includes: Obtain the target score corresponding to the target dataset from the target scores of the candidate targets; The candidate detection head target score loss is determined based on the target score of the candidate target in the target dataset. The step of determining the detection head category score loss and pseudo-labeling loss based on the unified category score includes: The unified category score is mapped to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset; The detection head category score loss is determined based on the category scores of the dataset. The pseudo-label loss is determined based on the pseudo-class scores in the unified category scores, excluding the dataset category scores.

10. The method according to claim 9, characterized in that, Each dataset includes multiple images and corresponding labels for the images, the labels including the category of the labeled target and the detection box.

11. The method according to claim 9, characterized in that, The candidate detection head outputs multiple target scores for the candidate targets, the number of which is the same as the number of datasets. Each target score corresponds to a dataset, indicating the probability that the candidate target is a labeled target in the corresponding dataset.

12. The method according to claim 9, characterized in that, The unified category score includes multiple category scores, each category score corresponding to a category in the unified label space, which records the categories of all labeled targets in the dataset.

13. The method according to claim 12, characterized in that, For each candidate target, the detection head outputs C detection boxes, where C is the number of categories in the unified label space, and each detection box corresponds to one category in the unified label space.

14. The method according to claim 13, characterized in that, For each candidate target, the detection head outputs K detection boxes, and each detection box corresponds to one or more categories in the unified label space.

15. The method according to claim 9, characterized in that, Also includes: The teacher detector is periodically updated using the student detector.

16. The method according to claim 15, characterized in that, The periodic updating of the teacher detector using the student detector includes: Once the learning rate of the target detector meets a preset condition, the teacher detector is updated using the student detector.

17. The method according to claim 16, characterized in that, The preset condition is that the learning rate of the optimized target detector reaches the minimum point of the Cosine function.

18. A target detection method, characterized in that, The method is applied to an object detector, which is trained using the method described in any one of claims 1-8, or using the method described in any one of claims 9-17. The object detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head. The method includes: The image to be detected is input into the first feature extraction network to obtain the first feature image; The first feature image is input into the candidate detection head to obtain the target score and candidate bounding box of the candidate target. The candidate box and the first feature image are input into the second feature extraction network to obtain the second feature image corresponding to the candidate box; The second feature image is input into the detection head to obtain the unified category score and detection box of the detected target. The unified category score includes multiple category scores, each category score corresponds to a category in a unified label space, and the unified label space records the categories of targets labeled in multiple datasets.

19. The method according to claim 18, characterized in that, The step of inputting the first feature image into the candidate detection head to obtain the target score and candidate bounding box of the candidate target includes: The candidate detection head outputs multiple target scores for candidate targets, and the target score with the highest score is selected as the target score of the candidate target.

20. A training device for a target detector, characterized in that, The target detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head. The training device for the target detector includes: The first acquisition module is used to acquire multiple datasets; The first feature extraction module is used to input the target image from the target dataset in the multiple datasets into the first feature extraction network to obtain the first feature image; The candidate target detection module is used to input the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target; determine the target score loss of the candidate detection head based on the target score; and determine the candidate box loss of the candidate detection head based on the candidate box. The second feature extraction module is used to input the candidate box and the first feature image into the second feature extraction network to obtain the second feature image corresponding to the candidate box; The target detection module is used to input the second feature image into the detection head to obtain a unified category score of the detected target and multiple detection boxes; select the target detection box corresponding to the category in the label space of the target dataset from the multiple detection boxes; determine the detection head category score loss based on the unified category score; and determine the detection head detection box loss based on the target detection box. The first optimization module is used to optimize the target detector based on the target score loss of the candidate detection head, the candidate bounding box loss of the candidate detection head, the category score loss of the detection head, and the detection bounding box loss of the detection head, so as to obtain the optimized target detector. The candidate target detection module is used to obtain the target score of the candidate target corresponding to the target dataset from the target scores of the candidate target; and to determine the target score loss of the candidate detection head based on the target score of the candidate target corresponding to the target dataset. The target detection module is configured to map the unified category score to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset; and to determine the detection head category score loss based on the dataset category score.

21. A training device for a target detector, characterized in that, The target detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head. The training device for the target detector includes: The first acquisition module is used to acquire multiple datasets; The copy module is used to copy the target detector to be trained, resulting in a student detector and a teacher detector. The pseudo-annotation module is used to input the target images from the target datasets in the multiple datasets into the teacher detector to obtain the first detection target, and to perform pseudo-annotation on the first detection target that has not been manually annotated. The training module is used to: input the target image into the first feature extraction network of the student detector to obtain a first feature image; input the first feature image into the candidate detection head of the student detector to obtain the target score and candidate bounding box of the candidate target; determine the target score loss of the candidate detection head based on the target score, and determine the candidate bounding box loss of the candidate detection head based on the candidate bounding box; input the candidate bounding box and the first feature image into the second feature extraction network of the student detector to obtain a second feature image corresponding to the candidate bounding box; input the second feature image into the detection head of the student detector to obtain the unified category score of the detected target and multiple detection boxes; select the target detection box corresponding to the category in the label space of the target dataset from the multiple detection boxes; determine the detection head category score loss and pseudo-labeling loss based on the unified category score; determine the detection head detection box loss based on the target detection box; and optimize the target detector based on the target score loss of the candidate detection head, the candidate bounding box loss of the candidate detection head, the category score loss of the detection head, the detection head detection box loss, and the pseudo-labeling loss to obtain an optimized target detector. The training module is used to obtain the target scores of the candidate targets that correspond to the target dataset; and to determine the target score loss of the candidate detection head based on the target scores of the candidate targets that correspond to the target dataset. The training module is configured to map the unified category score to a dataset category score, wherein the dataset category score is one or more, and each dataset category score corresponds to a category in the label space of the target dataset; determine the detection head category score loss based on the dataset category score; and determine the pseudo-labeling loss based on the pseudo-category scores other than the dataset category scores in the unified category score.

22. A target detection device, characterized in that, The target detection device employs a target detector, which is trained using the method described in any one of claims 1-8, or trained using the method described in any one of claims 9-17. The target detector includes a first feature extraction network, a candidate detection head, a second feature extraction network, and a detection head. The target detection device comprises: The first feature extraction module is used to input the image to be detected into the first feature extraction network of the target detector to obtain the first feature image; The candidate target detection module is used to input the first feature image into the candidate detection head to obtain the target score and candidate box of the candidate target; The second feature extraction module is used to input the candidate box and the first feature image into the second feature extraction network to obtain the second feature image corresponding to the candidate box; The target detection module is used to input the second feature image into the detection head to obtain the unified category score and detection box of the detected target. The unified category score includes multiple category scores, each category score corresponds to a category in a unified label space, and the unified label space records the categories of targets labeled in multiple datasets.

23. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when executed by the processor, the program implements the steps of the training method for the target detector as described in any one of claims 1 to 8; or, when executed by the processor, the program implements the steps of the training method for the target detector as described in any one of claims 9 to 17; or, when executed by the processor, the program implements the steps of the target detection method as described in claim 18 or 19.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the target detector training method as described in any one of claims 1 to 8; or, when executed by a processor, the computer program implements the steps of the target detector training method as described in any one of claims 9 to 17; or, when executed by a processor, the computer program implements the steps of the target detection method as described in claim 18 or 19.

Images