Pseudo label generation method, model training method, target detection method and device

Abstract

This invention provides a pseudo-label generation method, a model training method, an object detection method, and an apparatus. The method includes: training a constructed object detection model using a first labeled training image to obtain a first object detection model; using the first object detection model to obtain target features corresponding to labeled targets in the first training image; using the first object detection model and the target features corresponding to labeled targets to generate pseudo-labels for an unlabeled second training image; based on this, training a final object detection model using the labeled training image and the training image with pseudo-labels; and then using the trained object detection model to perform object detection on the image to be detected. This invention can generate accurate pseudo-labels for unlabeled training images, and based on this, a high-performance object detection model can be trained. Furthermore, using this model to perform object detection on the image to be detected can achieve better detection results.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for generating pseudo-labels, a method for training models, a method for detecting objects, and a device. Background Technology

[0002] Object detection refers to identifying objects of interest in an image and determining their category and location. In the many technical fields of computer vision, object detection is a fundamental task; tasks such as image segmentation, object tracking, and keypoint detection typically rely on it.

[0003] Object detection is generally achieved through a trained object detection model. That is, the object detection model is first trained using training images, and then the trained object detection model is used to detect objects in the image to be detected. In order to train the object detection model, it is usually necessary to annotate the training images, and then use the annotated training images to train the object detection model.

[0004] The annotation process for training images is generally done manually, that is, manually labeling the location and category of targets in the training images. Understandably, in order to train a high-performance object detection model, a large number of labeled training images are often needed. However, in practical applications, the high cost of manpower often leads to a lack of labeled training images, which imposes certain limitations on model training. Summary of the Invention

[0005] In view of this, the present invention provides a pseudo-label generation method, a model training method, an object detection method, and an apparatus to solve the problem that the lack of labeled training images due to high labor costs limits model training. The technical solution is as follows:

[0006] A method for generating pseudo-tags, comprising:

[0007] The first target detection model is trained using the labeled first training image to obtain the first target detection model;

[0008] Using the first target detection model, target features corresponding to the labeled targets in the first training image are obtained, and target features corresponding to several known targets are obtained respectively;

[0009] Using the first target detection model and supplemented by the target features corresponding to the known targets, pseudo-labels for the unlabeled second training images are generated.

[0010] Optionally, the step of using the first target detection model to obtain the target features corresponding to the labeled targets in the first training image includes:

[0011] Using the first target detection model, N candidate target features of the first training image are obtained, wherein each candidate target feature is a target feature corresponding to a candidate target, and N is an integer greater than 1;

[0012] Using the first target detection model, each candidate target feature of the first training image is converted into a candidate target detection result, resulting in N candidate target detection results for the first training image;

[0013] Based on the detection results of N candidate targets in the first training image, the annotation information of the first training image, and the N candidate target features of the first training image, the target features corresponding to the labeled targets in the first training image are obtained.

[0014] Optionally, the step of obtaining the target features corresponding to the labeled targets in the first training image based on the detection results of N candidate targets in the first training image, the annotation information of the first training image, and the N candidate target features of the first training image includes:

[0015] From the N candidate target detection results of the first training image, select the candidate target detection results that match the annotation information of the first training image;

[0016] Based on the selected candidate target detection results and the N candidate target features of the first training image, the target features corresponding to the labeled targets in the first training image are obtained.

[0017] Optionally, the step of generating pseudo-labels for the unlabeled second training image using the first target detection model, supplemented by the target features corresponding to the known targets, includes:

[0018] The target features corresponding to each labeled target of the same type are grouped together to obtain several target feature groups;

[0019] Using the first target detection model and supplemented by the several target feature groups, target detection is performed on the second training image to obtain several candidate target detection results of the second training image;

[0020] Based on the detection results of several candidate targets in the second training image, the pseudo-label of the second training image is determined.

[0021] Optionally, the step of using the first object detection model, supplemented by the plurality of object feature groups, to perform object detection on the second training image to obtain a plurality of candidate object detection results for the second training image includes:

[0022] Using the first target detection model, the image features of the second training image are obtained;

[0023] Using the first target detection model and supplemented by the several target feature groups, the image features of the second training image are decoded to obtain several candidate target features of the second training image;

[0024] Using the first object detection model, each candidate object feature of the second training image is converted into a candidate object detection result, resulting in several candidate object detection results for the second training image.

[0025] Optionally, the step of using the first target detection model, supplemented by the plurality of target feature groups, to decode the image features of the second training image to obtain a plurality of candidate target features of the second training image includes:

[0026] Traverse the aforementioned target feature groups:

[0027] Based on the first target detection model and supplemented by the target feature group currently being traversed, the second feature of the second training image is decoded to obtain N candidate target features of the second training image.

[0028] Optionally, each candidate target detection result in the second training image corresponds to a confidence level;

[0029] The step of determining the pseudo-label of the second training image based on the detection results of several candidate targets in the second training image includes:

[0030] Filter out candidate target detection results in the second training image whose confidence level is less than a preset confidence threshold.

[0031] The remaining candidate target detection results are deduplicated for multiple candidate target detection results for the same target, and the remaining candidate target detection results are used as pseudo-labels for the second training image.

[0032] Optionally, the first target detection model includes: a feature extractor, an encoder, a decoder, and a mapper;

[0033] The step of using the first target detection model, supplemented by the plurality of target feature groups, to perform target detection on the second training image, and obtaining a plurality of candidate target detection results for the second training image, includes:

[0034] The feature extractor of the first target detection model is used to extract features from the second training image to obtain the first feature of the second training image;

[0035] The encoder of the first target detection model is used to encode the first feature of the second training image to obtain the second feature of the second training image;

[0036] Using the decoder of the first target detection model, supplemented by the several target feature groups, the second feature of the second training image is decoded to obtain several candidate target features of the second training image;

[0037] Using the mapper of the first object detection model, each candidate object feature of the second training image is mapped to a candidate object detection result, thereby obtaining several candidate object detection results of the second training image.

[0038] Optionally, the decoder includes M cascaded decoding modules, each with the same structure, where M is an integer greater than 1;

[0039] Each target feature in each target feature group is a feature sequence, and the i-th feature in each feature sequence is the feature output by the i-th decoding module, 1<=i<=M;

[0040] When using the decoder of the first object detection model, supplemented by the several target feature groups, to decode the second feature of the second training image, the input of each decoding module includes target query features, reference features and the second feature of the second training image, and the output of each decoding module includes N candidate target features. Each decoding module uses the target query features and supplemented by the reference features to extract features related to the object detection task from the second feature of the second training image.

[0041] Wherein, the target query feature input to the first decoding module is the N features created, the reference feature input to the first decoding module is several features created, the target query feature input to the j-th decoding module is the N candidate target features output by the (j-1)-th decoding module, and the reference feature input to the j-th decoding module is the feature output by the (j-1)-th decoding module in each feature sequence of the several target feature groups, 2<=j<=M.

[0042] A method for training an object detection model, comprising:

[0043] Obtain the labeled first training image and the unlabeled second training image;

[0044] The pseudo-labels for the second training image are generated using any of the pseudo-label generation methods described above.

[0045] The second target detection model is trained using the labeled first training image and the second training image with pseudo-labels to obtain the final target detection model. The second target detection model is either a constructed target detection model or a model obtained by initially training the constructed target detection model using the labeled first training image.

[0046] A target detection method, comprising:

[0047] The target detection model, which has been pre-trained, is used to perform target detection on the image to be detected.

[0048] The target detection model is trained using the target detection model training method described above.

[0049] A pseudo-label generation device includes: a first training module, a known target feature acquisition module, and a pseudo-label generation module;

[0050] The first training module is used to train the constructed target detection model using the labeled first training image to obtain the first target detection model;

[0051] The known target feature acquisition module is used to acquire the target features corresponding to the labeled targets in the first training image using the first target detection model, and obtain the target features corresponding to several known targets respectively;

[0052] The pseudo-label generation module is used to generate pseudo-labels for the unlabeled second training image by using the first target detection model and supplementing it with the target features corresponding to the several known targets.

[0053] A target detection model training device includes: a training data acquisition module, the aforementioned pseudo-label generation device, and a second training module;

[0054] The training data acquisition module is used to acquire a labeled first training image and an unlabeled second training image;

[0055] The pseudo-label generation module is used to generate pseudo-labels for the second training image;

[0056] The second training module is used to train the second object detection model using the labeled first training image and the second training image with pseudo-labels to obtain the final object detection model. The second object detection model is either a constructed object detection model or a model obtained by initially training the constructed object detection model using the labeled first training image.

[0057] A target detection device includes: an image acquisition module and a target detection module;

[0058] The image acquisition module is used to acquire the image to be detected;

[0059] The target detection module is used to perform target detection on the image to be detected using a pre-trained target detection model;

[0060] The target detection model is obtained using the target detection model training device described above.

[0061] A processing device, comprising: a memory and a processor;

[0062] The memory is used to store programs;

[0063] The processor is configured to execute the program to implement each step of the pseudo-label generation method described above, or to implement each step of the target detection model training method described above, or to implement each step of the target detection method described above.

[0064] A computer-readable storage medium having a computer program stored thereon, wherein when executed by a processor, the computer program implements the steps of the pseudo-label generation method described above, or implements the steps of the target detection model training method described above, or implements the steps of the target detection method described above.

[0065] The pseudo-label generation method provided by this invention first trains a constructed target detection model using a labeled first training image to obtain a first target detection model. Then, based on the first target detection model, it obtains the target features corresponding to the labeled targets in the first training image, resulting in target features corresponding to several known targets. Next, based on the first target detection model and supplemented by the target features corresponding to the several known targets, pseudo-labels for unlabeled second training images are generated. On the one hand, the pseudo-label generation method provided by this invention can automatically generate pseudo-labels for unlabeled training images, greatly reducing labor costs compared to manual annotation. On the other hand, when generating pseudo-labels for unlabeled second training images, the pseudo-label generation method provided by this invention references the target features corresponding to the labeled targets in the first training image, that is, it references the target features corresponding to known targets. Referencing the target features corresponding to known targets can improve the recognition accuracy of the first target detection model for potential targets, thereby generating more accurate pseudo-labels.

[0066] This invention also provides a method for training an object detection model. This method first acquires a labeled first training image and an unlabeled second training image. Then, it uses the pseudo-label generation method provided by this invention to generate accurate pseudo-labels for the unlabeled second training image. Finally, it trains the final object detection model using the labeled first training image and the second training image with accurate pseudo-labels. A high-performance object detection model can be trained using the labeled first training image and the second training image with accurate pseudo-labels. This invention also provides an object detection method that uses the high-performance object detection model trained by the object detection model training method provided by this invention to perform object detection on the image to be detected. Using the high-performance object detection model to perform object detection on the image to be detected yields better detection results. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0068] Figure 1 This is a schematic diagram of the hardware architecture involved in the present invention;

[0069] Figure 2 This is a flowchart illustrating the pseudo-tag generation method provided in an embodiment of the present invention;

[0070] Figure 3 This is a schematic diagram of the pseudo-labels obtained by directly detecting objects on the second training image;

[0071] Figure 4 This is a schematic diagram of the pseudo-labels obtained by performing target detection on the second training image by referring to the target features corresponding to known targets;

[0072] Figure 5 This is a schematic diagram of the process of training a constructed target detection model using a labeled first training image, as provided in an embodiment of the present invention.

[0073] Figure 6 A schematic diagram of a structure of a target detection model provided in an embodiment of the present invention;

[0074] Figure 7 This is a schematic diagram of the decoder structure in the target detection model provided in an embodiment of the present invention;

[0075] Figure 8This is a flowchart illustrating how a first target detection model is used to obtain target features corresponding to labeled targets in a first training image, thereby obtaining target features corresponding to several known targets.

[0076] Figure 9 This is a schematic diagram illustrating the acquisition of candidate target detection results that match the annotation information of a first training image, as provided in an embodiment of the present invention.

[0077] Figure 10 This is a schematic diagram illustrating the acquisition of candidate target detection results that match the annotation information of another first training image, as provided in an embodiment of the present invention.

[0078] Figure 11 This is a flowchart illustrating how a first target detection model is used, supplemented by target features corresponding to several known targets, to generate pseudo-labels for an unlabeled second training image, according to an embodiment of the present invention.

[0079] Figure 12 This is a schematic diagram of the input to the decoder in the target detection model provided in an embodiment of the present invention;

[0080] Figure 13 This invention provides an example of using a first target detection model, supplemented by several target feature groups, to perform target detection on an unlabeled second training image;

[0081] Figure 14 This is a flowchart illustrating the target detection model training method provided in an embodiment of the present invention.

[0082] Figure 15 This is a schematic diagram of the structure of the pseudo-label generation device provided in an embodiment of the present invention;

[0083] Figure 16 This is a schematic diagram of the target detection model training device provided in an embodiment of the present invention;

[0084] Figure 17 This is a schematic diagram of the processing device provided in an embodiment of the present invention. Detailed Implementation

[0085] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] Most current model training methods are supervised training, which involves manually labeling training images to obtain labeled training images. These labeled training images are then used to train the constructed object detection model to obtain the final object detection model. Understandably, to obtain a high-performance object detection model, a large number of labeled training images are needed. However, in practical applications, the high cost of manual labor often leads to a lack of labeled training images, thus limiting model training.

[0087] To address the problem of limited model training due to a lack of labeled training images caused by high labor costs, the inventors of this case conducted research and conceived of a semi-supervised training method. This method involves first acquiring a small number of labeled first training images, then using these images to supervise the training of a constructed object detection model to obtain a first object detection model. This first model is then used to detect objects on unlabeled second training images to obtain pseudo-labels for those images. Finally, the labeled first training images and the pseudo-labeled second training images can be used to further train the first object detection model to obtain the final object detection model. Since this semi-supervised training method only requires labeling a small number of training images, it significantly reduces the cost of manual labeling.

[0088] The inventors of this case studied the above-mentioned semi-supervised training method and found that although the semi-supervised training method reduced the cost of manual annotation, it brought new problems: the quality of the pseudo-labels generated by the first object detection model on the unlabeled second training image was not high. The low quality of the pseudo-labels resulted in poor training effect when using the labeled first training image and the pseudo-labeled second training image to train the first object detection model, that is, the performance of the final object detection model was not good.

[0089] To address the aforementioned issues, the inventors continued their research and discovered that the core paradigm of the semi-supervised training method is to generate pseudo-labels for unlabeled second training images and then train the first object detection model accordingly. However, the generation process of pseudo-labels is completely independent of the labeled first training images, which is equivalent to blindly predicting potential targets. This results in poor quality of pseudo-labels generated for the second training images, which in turn leads to poor performance or even regression of the model after the second stage of training.

[0090] Based on the above findings, the inventors of this case devised a solution: when generating pseudo-labels for unlabeled second training images based on the first object detection model, the target features of known targets (i.e., labeled targets in the labeled first training images) can be referenced to improve the model's ability to identify potential targets, thereby improving the quality of the generated pseudo-labels and ultimately improving the performance of the finally trained object detection model.

[0091] Before introducing the solution provided by this invention, the hardware architecture involved in this invention will be described first.

[0092] In one possible implementation, such as Figure 1 As shown, the hardware architecture involved in this invention may include: electronic device 101 and server 102.

[0093] For example, electronic device 101 can be any electronic product that can interact with a user through one or more means such as a keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as PC, laptop, tablet, PDA, mobile phone, learning machine, smart TV, etc.

[0094] It should be noted that, Figure 1 This is just one example; there can be many types of electronic devices, not limited to... Figure 1 The laptop in the middle.

[0095] For example, server 102 can be a single server, a server cluster consisting of multiple servers, or a cloud computing server center. Server 102 may include processors, memory, and network interfaces, etc.

[0096] For example, electronic device 101 can establish a connection and communicate with server 102 through a wireless communication network; for example, electronic device 101 can establish a connection and communicate with server 102 through a wired communication network.

[0097] Electronic device 101 can acquire a labeled first training image and an unlabeled second training image, and send the labeled first training image and the unlabeled second training image to server 102. Server 102 generates pseudo-labels for the unlabeled second training image based on the labeled first training image.

[0098] Optionally, the server 102 can also train the final object detection model using the labeled first training image and the pseudo-labeled second training image.

[0099] Optionally, the electronic device 101 can also acquire the image to be detected and send the image to be detected to the server 102. After receiving the image to be detected, the server 102 can perform target detection on the image to be detected based on the trained target detection model.

[0100] In another possible implementation, the hardware architecture involved in this invention may include: an electronic device.

[0101] The electronic device is an electronic product with strong data processing capabilities. The electronic device can acquire a labeled first training image and an unlabeled second training image, and generate pseudo-labels for the unlabeled second training image based on the labeled first training image.

[0102] Optionally, the electronic device can also train the final object detection model using the labeled first training image and the pseudo-labeled second training image.

[0103] Optionally, the electronic device can also acquire the image to be detected and perform object detection on the image based on the trained object detection model.

[0104] Those skilled in the art should understand that the above-described electronic devices and servers are merely examples, and other existing or future electronic devices or servers that are applicable to this invention should also be included within the scope of protection of this invention, and are hereby incorporated by reference.

[0105] The pseudo-label generation method, target detection model training method, and target detection method provided by the present invention will be introduced through the following embodiments.

[0106] Please see Figure 2 The diagram illustrates a flowchart of a pseudo-tag generation method provided in an embodiment of the present invention. This pseudo-tag generation method may include:

[0107] Step S201: Use the labeled first training image to train the constructed object detection model to obtain the first object detection model.

[0108] The annotation information of the first training image includes the category and location of the target in the first training image.

[0109] Optionally, the model parameters of the constructed object detection model are randomly initialized. Of course, this embodiment is not limited to this; the model parameters of the constructed object detection model can also be parameters obtained through pre-training. Optionally, the object detection model can be a model based on the Transformer framework.

[0110] In this embodiment of the invention, a first training image labeled with the category and location of the target is used to train the constructed target detection model. During training, the training objective is to make the type and location of the target predicted by the target detection model for the first training image consistent with the type and location of the target labeled in the first training image.

[0111] The first object detection model, which is trained using the labeled first training image, has basic object detection capabilities.

[0112] Step S202: Using the first target detection model, obtain the target features corresponding to the labeled targets in the first training image, and obtain the target features corresponding to several known targets respectively.

[0113] Since the first training image is labeled with the category and location of the target, the target in the first training image is known. In order to generate accurate pseudo-labels in the future, after training the first target detection model, the target features corresponding to the labeled targets in the first training image are obtained based on the first target detection model, that is, the target features corresponding to the known targets.

[0114] Step S203: Using the first target detection model and supplemented by the target features corresponding to several known targets, generate pseudo-labels for the unlabeled second training images.

[0115] Specifically, the target features corresponding to the second training image and several known targets are input into the first target detection model. The first target detection model, supplemented by the target features corresponding to the several known targets, performs target detection on the second training image to obtain pseudo-labels for the second training image. It should be noted that the pseudo-labels for the second training image are the category and location of the target in the second training image.

[0116] In this embodiment of the invention, target features corresponding to known targets are used to assist a first target detection model in generating pseudo-labels for unlabeled second training images. Specifically, the first target detection model generates pseudo-labels for unlabeled second training images by referencing target features corresponding to known targets. Compared to blindly predicting potential targets, referencing target features corresponding to known targets improves the first target detection model's ability to identify potential targets, thereby improving the quality of the generated pseudo-labels.

[0117] Figure 3 This diagram illustrates the pseudo-labels obtained by directly detecting objects on the second training image. Figure 4 The diagram illustrates the pseudo-labels obtained by performing target detection on a second training image with reference to the target features corresponding to a known target. By comparison, it is easy to see that the generated pseudo-labels are incorrect when the target features corresponding to the known target are not referenced, while the correct pseudo-labels can be generated when the target features corresponding to the known target are referenced. It is evident that referencing the target features corresponding to the known target can improve the quality of the generated pseudo-labels.

[0118] The pseudo-label generation method provided in this invention first uses a labeled first training image to supervise the training of a constructed target detection model, obtaining a first target detection model. Then, based on the first target detection model, it obtains the target features corresponding to the labeled targets in the first training image, obtaining target features corresponding to several known targets. Next, based on the first target detection model and supplemented by the target features corresponding to the several known targets, it generates pseudo-labels for an unlabeled second training image. On the one hand, the pseudo-label generation method provided in this invention can automatically generate pseudo-labels for unlabeled training images, greatly reducing labor costs compared to manual annotation. On the other hand, when generating pseudo-labels for the unlabeled second training image, the pseudo-label generation method provided in this invention references the target features corresponding to the labeled targets in the first training image, that is, it references the target features corresponding to known targets. Referencing the target features corresponding to known targets can improve the recognition accuracy of the first target detection model for potential targets, thereby generating more accurate pseudo-labels. In summary, the pseudo-label generation method provided in this invention can automatically generate accurate pseudo-labels for unlabeled training images.

[0119] In another embodiment of the present invention, the step S201 of the above embodiment is described as follows: "Using the labeled first training image, the constructed target detection model is trained to obtain the first target detection model".

[0120] The process of training the constructed object detection model using the labeled first training image may include: performing object detection on the first training image based on the object detection model to obtain several candidate object detection results of the first training image; determining the model prediction loss based on the several candidate object detection results of the first training image and the annotation information of the first training image; and updating the parameters of the object detection model based on the model prediction loss.

[0121] The following is combined Figure 5 The following section describes in more detail the process of training the constructed object detection model using the labeled first training image, such as... Figure 5 As shown, the training process may include:

[0122] Step S501: Using the object detection model, extract features from the first training image to obtain the first feature of the first training image.

[0123] Figure 6 This embodiment illustrates one structure of an object detection model, supplemented by... Figure 6 The model structure shown illustrates the training process of the object detection model. It should be noted that... Figure 6 The model structure shown is merely an example, and this embodiment does not limit the structure of the object detection model to be specific. Figure 6 The structure shown.

[0124] like Figure 6 As shown, the object detection model may include a feature extractor, which can be used to extract features from the first training image. That is, the first training image is input into the feature extractor of the object detection model for feature extraction, and the feature extractor outputs the extracted features, which are the first features of the first training image.

[0125] Step S502: Using the object detection model, encode the first feature of the first training image to obtain the second feature of the first training image.

[0126] like Figure 6 As shown, the object detection model may include an encoder, which can be used to encode the first feature of the first training image. That is, the first feature of the first training image is input into the encoder for encoding, and the encoder outputs the encoded feature, which is the second feature of the first training image.

[0127] Step S503: Using the object detection model, decode the second feature of the first training image to obtain N candidate object features of the first training image.

[0128] In this context, each candidate target feature in the first training image is a target feature corresponding to a candidate target.

[0129] like Figure 6 As shown, the object detection model may include a decoder, which can be used to decode the second feature of the first training image. That is, the second feature of the first training image is input into the decoder of the object detection model for decoding, and the decoder outputs N candidate object features of the first training image, where N is an integer greater than 1. It should be noted that the specific value of N can be determined according to the specific application scenario.

[0130] Optionally, the decoder of the object detection model may include M cascaded decoding modules, each with the same structure. The input of each decoding module is the second feature of the first training image and the target query feature, and the output of each decoding module is N candidate target features. Here, M is an integer greater than 1.

[0131] In this module, the target query features of the first decoding module are N features created (e.g., N features all zeros, or N randomly initialized features), and the target query features of the other decoding modules are N candidate target features output by the previous decoding module. The decoding module uses the target query features to extract feature information related to the target detection task from the second feature of the first training image.

[0132] Optional, such as Figure 7As shown, each decoding module may include a self-attention module, a first feature processing module, a cross-attention module, a second feature processing module, a third feature processing module, and a fourth feature processing module. The self-attention module performs self-attention calculation on the input target query features. The input features and output features of the self-attention module are input to the first feature processing module. The first feature processing module sequentially performs fusion processing (e.g., addition) and normalization processing on the input features and output features of the self-attention module. The second features of the first training image and the features output by the first feature processing module are input to the cross-attention module. The cross-attention module processes the second features and first features of the first training image. The features output by the module are subjected to cross-attention calculation. The features output by the first feature processing module and the features output by the cross-attention module are input into the second feature processing module. The second feature processing module performs fusion processing and normalization processing on the features output by the first feature processing module and the features output by the cross-attention module in sequence. The features output by the second feature processing module are input into the third feature processing module. The third processing module performs linear transformation on the input features. The features output by the second feature processing module and the features output by the third feature processing module are input into the fourth feature processing module. The fourth feature processing module performs fusion processing (e.g., addition) and normalization processing on the features output by the second feature processing module and the features output by the third feature processing module in sequence.

[0133] The process by which the self-attention module performs self-attention calculation on the input target query features is as follows: for each feature F contained in the target query features... qi Calculate the feature F qi Each of the features (F) contained in the target query features q1 F q2 F qN The similarity between the features is used to calculate the weights, and then the weights α for each feature included in the target query feature are obtained. qi1 α qi2 ..., α qiN Using weight α qi1 α qi2 ..., α qiN Sum of the corresponding features after weighting (α) qi1* F q1 +α qi2* F q2 …+α qiN* F qN ), to obtain F qi The final output of the self-attention module is F. q1 ′、F q2 ′、…、F qN ′.

[0134] The process of cross-attention module performing cross-attention calculation on the second feature of the first training image and the feature output by the first feature processing module is as follows: for each feature F output by the first feature processing module... oi Calculate the feature F oi The similarity between each feature and the second feature of the first training image is calculated, and the resulting similarity is used as a weight. The weights corresponding to each feature in the second feature of the first training image are then summed using these weights to obtain F. oi The final output of the cross-attention module is F. o1 ′、F o2 ′、…、F oN ′.

[0135] The "N candidate target features of the first training image" mentioned above refer to the N features output by the Mth decoding module (i.e., the last decoding module) in a cascaded M decoding module. The N features output by the Mth decoding module can be represented as F 1,M F 2,M F N,M .

[0136] Step S504: Using the object detection model, convert each candidate object feature of the first training image into a candidate object detection result to obtain N candidate object detection results of the first training image.

[0137] Each candidate target detection result includes a category prediction result and a location prediction result for the candidate target.

[0138] like Figure 6 As shown, the object detection model may include a mapping module, which can be used to map each candidate object feature of the first training image to a candidate object detection result. That is, the mapping module of the object detection model inputs N candidate object features of the first training image into the mapping module of the object detection model, and the mapping module of the object detection model outputs N candidate object detection results of the first training image.

[0139] Step S505: Determine the model prediction loss based on the detection results of N candidate targets in the first training image and the annotation information of the first training image.

[0140] Specifically, the category prediction loss can be determined based on the category prediction results in the N candidate target detection results and the category of the target labeled in the first training image. The location prediction loss can be determined based on the location prediction results in the candidate target detection results where the category prediction result that matches the category of the target labeled in the first training image is located, and the location of the target labeled in the first training image. The category prediction loss and the location prediction loss are fused (e.g., summed or weighted summed), and the fused loss is used as the final model prediction loss.

[0141] Step S506: Update the parameters of the target detection model based on the model's predicted loss.

[0142] Using multiple different first training images, perform multiple iterative training steps S501 to S505 as described above until the training termination condition is met (e.g., reaching the set number of training iterations).

[0143] The object detection model constructed in the above manner is trained. After training, a first object detection model with basic object detection capabilities can be obtained.

[0144] In another embodiment of the present invention, the specific implementation process of "step S202: using the first target detection model to obtain the target features corresponding to the labeled targets in the first training image, and obtaining the target features corresponding to several known targets respectively" in the above embodiment will be described.

[0145] Please see Figure 8 This illustrates a flowchart of a process for obtaining target features corresponding to labeled targets in a first training image using a first target detection model, and obtaining target features corresponding to several known targets. The process may include:

[0146] Step S801: Use the first target detection model to obtain N candidate target features of the first training image.

[0147] The specific implementation process of this step can be found in steps S501 to S503 of the above embodiment, and will not be repeated here.

[0148] Step S802: Based on the first target detection model, convert each candidate target feature of the first training image into a candidate target detection result to obtain N candidate target detection results of the first training image.

[0149] The specific implementation process of this step can be found in step S504 of the above embodiment, and will not be repeated here.

[0150] Step S803: Based on the detection results of N candidate targets in the first training image, the annotation information of the first training image, and the N candidate target features of the first training image, determine the target features corresponding to the labeled targets in the first training image.

[0151] Specifically, the process of determining the target features corresponding to the labeled targets in the first training image based on the detection results of N candidate targets in the first training image, the annotation information of the first training image, and the N candidate target features of the first training image may include:

[0152] Step S8031: Select candidate target detection results that match the annotation information of the first training image from the N candidate target detection results of the first training image.

[0153] As mentioned above, each candidate target detection result includes a category prediction result and a location prediction result. In this embodiment, the candidate target detection result that matches the annotation information of the first training image is the candidate target detection result whose category prediction result is consistent with the category of the target annotated in the first training image and whose location prediction result is closest to the location of the target annotated in the first training image.

[0154] Specifically, the process of selecting candidate target detection results that match the annotation information of the first training image from the N candidate target detection results of the first training image may include: selecting candidate target detection results whose category prediction results are consistent with the category of the target annotated in the first training image from the N candidate target detection results of the first training image; and further selecting candidate target detection results whose position prediction results are closest to the position of the target annotated in the first training image from the selected candidate target detection results.

[0155] For example, a first training image is labeled with two targets, namely, a first target and a second target, where the category of the first target is "dog" and the category of the second target is "cat". First, from N candidate target detection results in the first training image, candidate target detection results with a predicted category of "dog" and candidate target detection results with a predicted category of "cat" are selected. Then, from the candidate target detection results with a predicted category of "dog", the candidate target detection result whose position prediction result is closest to the position (labeled position) of the first target is selected. Finally, from the candidate target detection results with a predicted category of "cat", the candidate target detection result whose position prediction result is closest to the position (labeled position) of the first target is selected. The candidate target detection results that are closest to the position prediction result of the second target (the labeled position) are selected from the results. For example, if the category prediction result is "dog", the candidate target detection results are (dog, pos1), (dog, pos2), and (dog, pos3), and the category prediction result is "cat", the candidate target detection results are (cat, pos4) and (cat, pos5). pos2 is closest to the position of the first target, and pos5 is closest to the position of the second target. Therefore, (dog, pos2) and (cat, pos5) are determined as candidate target detection results that match the labeled information of the first training image.

[0156] Figure 9 and Figure 10 This diagram illustrates the acquisition of candidate object detection results that match the annotation information of the first training image, as shown below. Figure 9 As shown, the candidate object detection results that match the annotation information of the first training image are the first candidate object detection result and the third candidate object detection result, as follows: Figure 10 As shown, the candidate target detection results that match the annotation information of the first training image are the second candidate target detection result and the third candidate target detection result.

[0157] Step S8032: Based on the selected candidate target detection results and the N candidate target features of the first training image, determine the target features corresponding to the labeled targets in the first training image.

[0158] Specifically, based on the selected candidate target detection results and the N candidate target features of the first training image, the target features corresponding to the labeled targets in the first training image are obtained from the features output by the decoder of the first target detection model.

[0159] For example, the N candidate target features of the first training image are F 1,M F 2,M F N-1,M F N,M The detection results of the N candidate objects in the first training image are R1, R2, ..., R. N-1 R N Where R1 is composed of candidate target features F 1,MR2 is derived from the candidate target features F 2,M It was converted from, ....R N-1 From the candidate target features F N-1,M Transformed from, R N From the candidate target features F N,M This is derived from the conversion, assuming R2 and R N-1 If the candidate target detection results are selected through step S5031, then F will be... 2,1 F 2,2 , … F 2,M The characteristic sequence F2 and F are composed of N-1,1 F N-1,2 , … F N-1,M The characteristic sequence F is composed of N-1 The target features identified are those corresponding to the labeled targets in the first training image. It should be noted that the number after the comma in the subscript of F indicates which decoding module outputs the feature.

[0160] By processing multiple labeled first training images according to the above process, the target features corresponding to several labeled targets can be obtained.

[0161] In another embodiment of the present invention, the specific implementation process of "step S203: using the first target detection model and supplemented by the target features corresponding to several known targets respectively to generate pseudo-labels for the unlabeled second training image" in the above embodiment will be described.

[0162] Please see Figure 11 The diagram illustrates a process for generating pseudo-labels for unlabeled second training images using a first object detection model, supplemented by target features corresponding to several known targets. This process may include:

[0163] Step S1101: Divide the target features corresponding to each labeled target of the same type into a group to obtain several target feature groups.

[0164] For example, group the target features corresponding to the category "dog" into one group, and group the target features corresponding to the category "cat" into another group.

[0165] Step S1101 can obtain several target feature groups corresponding to different categories. The category corresponding to each target feature group is the category of the target whose target features are contained in the target feature group. For example, if the target features corresponding to the target with the category "dog" are divided into a group, the target feature group corresponding to the category "dog" is obtained. If the target features corresponding to the target with the category "cat" are divided into a group, the target feature group corresponding to the category "cat" is obtained.

[0166] Step S1102: Using the first target detection model and supplemented by several target feature groups, target detection is performed on the second training image to obtain several candidate target detection results of the second training image.

[0167] Specifically, the process of using a first object detection model, supplemented by several object feature groups, to perform object detection on a second training image and obtain several candidate object detection results for the second training image may include:

[0168] Step S1102-1: Use the first target detection model to obtain the image features of the second training image.

[0169] For example, the structure of the first object detection model is as follows: Figure 6 The structure shown includes a feature extractor, an encoder, a decoder, and a mapper. The feature extractor of the first object detection model can be used to extract features from the second training image to obtain the first feature of the second training image. The encoder of the first object detection model can be used to encode the first feature of the second training image to obtain the second feature of the second training image. The aforementioned "image features of the second training image" are the second features of the second training image.

[0170] Step S1102-2: Using the first target detection model and supplemented by several target feature groups, decode the image features of the second training image to obtain several candidate target features of the second training image.

[0171] The decoder of the first object detection model, supplemented by several target feature groups, can be used to decode the second features of the second training image to obtain several candidate target features of the second training image.

[0172] As mentioned in the above embodiments, the decoder may include M cascaded decoding modules, each decoding module having the same structure. Each feature in each target feature group is a feature sequence, and the i-th (1<=i<=M) feature in each feature sequence is the feature output by the i-th decoding module. For example... Figure 12As shown, when using the decoder of the first object detection model, supplemented by several target feature groups, to decode the second features of the second training image, the input of each decoding module includes target query features, reference features, and the second features of the second training image. The output of each decoding module includes N candidate target features (which may also include reference features). Each decoding module uses the target query features and supplemented by reference features to extract features related to the object detection task from the second features of the second training image. The target query features input to the first decoding module are the N features created (e.g., N features all zeros, or N randomly initialized features). The reference features input to the first decoding module are the several features created. The target query features input to the j-th (2<=j<=M) decoding module are the N candidate target features output by the (j-1)-th decoding module. The reference features input to the j-th decoding module are the features output by the (j-1)-th decoding module in each feature sequence of the several target feature groups.

[0173] It should be noted that when inputting reference features into each decoding module, the self-attention module in each decoding module performs self-attention calculation as follows: for each feature F contained in the target query feature... qi Calculate the feature F qi Each of the features (F) contained in the target query features q1 F q2 F qN The similarity of ) is calculated, and the feature F is calculated. qi Each of the features contained in the reference feature (F) c1 F c2 The similarity of features (...) is used to calculate the weights, and the weights (α) corresponding to each feature included in the target query feature are obtained. qi1 α qi2 ..., α qiN ), and the weights (α) of each feature included in the reference feature. ci1 α ci2 Then, the calculated weights are used to weight the corresponding features and summed (α). qi1* F q1 +α qi2* F q2 …+α qiN* F qN +α ci1 *F c1 +α ci2 *F c2 +…), to get F qi The final output of the self-attention module is F. q1 ′、F q2 ′、…、F qNThrough attention calculation, feature information from the reference features is incorporated into the features output by the self-attention module.

[0174] Specifically, the process of using the decoder of the first object detection model, supplemented by several target feature groups, to decode the second features of the second training image and obtain several candidate target features of the second training image can include: traversing several target feature groups: based on the first object detection model and supplemented by the currently traversed target feature groups, decoding the second features of the second training image to obtain N candidate target features of the second training image. If there are P target feature groups, then after the traversal, P*N candidate target features will be obtained for the second training image.

[0175] Step S1102-3: Using the first target detection model, convert each candidate target feature of the second training image into a candidate target detection result to obtain several candidate target detection results of the second training image.

[0176] The mapper of the first object detection model can be used to map each candidate object feature of the second training image to a candidate object detection result, thereby obtaining several candidate object detection results of the second training image.

[0177] It should be noted that when using the first object detection model, supplemented by several object feature groups, to perform object detection on the second training image, if the object feature group corresponding to category x is also used, the object of category x can be accurately detected, such as... Figure 13 As shown, by supplementing the target feature group corresponding to the category "dog", targets of the category "dog" will be accurately detected; by supplementing the target feature group corresponding to the category "cat", targets of the category "cat" will be accurately detected.

[0178] Step S1103: Determine the pseudo-label of the second training image based on the detection results of several candidate targets in the second training image.

[0179] It should be noted that each candidate target detection result in the second training image corresponds to a confidence level. When determining the pseudo-label of the second training image based on several candidate target detection results, candidate target detection results with confidence levels lower than a preset confidence threshold can be filtered out first. Then, multiple candidate target detection results for the same target in the remaining candidate target detection results are deduplicated. Finally, only one candidate target detection result is retained for each target, and the retained candidate target detection result is used as the pseudo-label of the second training image. Optionally, the NMS (Non-Maximum Suppression) method can be used to deduplicate multiple candidate target detection results for the same target.

[0180] Through the above process, accurate pseudo-labels can be generated for the second training image.

[0181] Based on the pseudo-label generation method provided in the above embodiments, this invention also provides a target detection model training method. Please refer to [link to relevant documentation]. Figure 14 The diagram illustrates the training process of the object detection model, which may include:

[0182] Step S1401: Obtain the labeled first training image and the unlabeled second training image.

[0183] It should be noted that the "labeled first training image" in this step can be the same as or different from the "labeled first training image" in "step S201".

[0184] Step S1402: Generate pseudo-labels for the unlabeled second training image.

[0185] The pseudo-label generation method provided in the above embodiments can be used to generate pseudo-labels for the unlabeled second training image. For the unlabeled second training image, the pseudo-label generation method provided in the above embodiments can generate accurate pseudo-labels.

[0186] Step S1403: Use the labeled first training image and the second training image with pseudo-labels to train the second object detection model to obtain the final object detection model.

[0187] The second object detection model is either a constructed object detection model or a model obtained by initially training the constructed object detection model using the labeled first training image (for example, it can be the first object detection model in the above embodiment).

[0188] When training the second object detection model, the training objectives are to make the categories and locations of the objects predicted by the second object detection model for the first training image consistent with the categories and locations of the objects labeled in the first training image, and to make the categories and locations of the objects predicted by the first object detection model for the second training image consistent with the types and locations of the objects indicated by the pseudo-labels in the second training image.

[0189] The process of training the second target detection model using the labeled first training image and the second training image with pseudo-labels is similar to the process of "training the constructed target detection model using the labeled first training image" in the above embodiment, and will not be described in detail here.

[0190] It should be noted that if the second object detection model is a model initially trained using the labeled first training images, then the labeled first training images and the second training images with pseudo-labels can be used to train the second object detection model for one epoch (i.e., all training images are input into the second object detection model once) to obtain the third object detection model. After obtaining the third object detection model, it can be tested using a test set. If the performance of the third object detection model is better than that of the second object detection model, then the pseudo-labels of the second training images are regenerated based on the third object detection model. Then, the labeled first training images and the second training images with pseudo-labels (the latest pseudo-labels) are used to train the third object detection model for one epoch to obtain the fourth object detection model. The fourth object detection model is tested using a test set. If the performance of the fourth object detection model is better than that of the third object detection model, then the generation of pseudo-labels and model training continue. If the performance of the fourth object detection model is not better than that of the third object detection model, then the generation of pseudo-labels and model training end, and the third object detection model is used as the final object detection model.

[0191] The target detection model training method provided in this embodiment of the invention first obtains a labeled first training image and an unlabeled second training image. Then, it uses the pseudo-label generation method provided in the above embodiment to generate pseudo-labels for the unlabeled second training image. Finally, it uses the labeled first training image and the second training image with pseudo-labels to train the second target detection model, thus obtaining the final target detection model. On the one hand, the target detection model training method provided in this embodiment of the invention is a semi-supervised training method, that is, it does not require obtaining a large number of labeled (manually labeled) training images, so the cost of manual labeling is low. On the other hand, when generating pseudo-labels for unlabeled training images, the target detection model training method provided in this embodiment of the invention refers to the target features corresponding to labeled targets in the labeled training images, that is, the target features corresponding to known targets. Referring to the target features corresponding to known targets can improve the model's recognition of potential targets, thereby generating more accurate pseudo-labels. Therefore, by using the labeled first training image and the second training image with accurate pseudo-labels to train the second target detection model, a target detection model with better performance can be obtained.

[0192] Based on the target detection model training method provided in the above embodiments, this invention also provides a target detection method, which may include:

[0193] Step a1: Obtain the image to be detected.

[0194] Step a2: Use the pre-trained target detection model to perform target detection on the image to be detected.

[0195] The target detection model was trained using the target detection model training method provided in the above embodiments.

[0196] Since the target detection model training method provided in the above embodiments can train a target detection model with better performance, using the target detection model with better performance to perform target detection on the image to be detected can achieve better target detection results.

[0197] This invention also provides a pseudo-label generation device. The pseudo-label generation device provided in this invention will be described below. The pseudo-label generation device described below can be referred to in correspondence with the pseudo-label generation method described above.

[0198] Please see Figure 15 The diagram shows a schematic of the structure of a pseudo-label generation device 1500 provided in an embodiment of the present invention, which may include: a first training module 1501, a known target feature acquisition module 1502, and a pseudo-label generation module 1503.

[0199] The first training module 1501 is used to train the constructed object detection model using the labeled first training image to obtain the first object detection model.

[0200] The known target feature acquisition module 1502 is used to acquire the target features corresponding to the labeled targets in the first training image using the first target detection model, and obtain the target features corresponding to several known targets respectively.

[0201] The pseudo-label generation module 1503 is used to generate pseudo-labels for the unlabeled second training image by using the first object detection model and supplementing it with the target features corresponding to several known targets.

[0202] Optionally, the known target feature acquisition module 1502 includes: a first candidate target detection result acquisition submodule and a known target feature acquisition submodule.

[0203] The first candidate target detection result acquisition submodule is used to acquire N candidate target features of the first training image using the first target detection model, and to convert each candidate target feature of the first training image into a candidate target detection result using the first target detection model, thereby obtaining N candidate target detection results of the first training image, wherein each candidate target feature is a target feature corresponding to a candidate target, and N is an integer greater than 1.

[0204] The known target feature acquisition submodule is used to acquire the target features corresponding to the labeled targets in the first training image based on the detection results of N candidate targets in the first training image, the annotation information of the first training image, and the N candidate target features of the first training image.

[0205] Optionally, when the known target feature acquisition submodule acquires the target features corresponding to the labeled targets in the first training image based on the detection results of N candidate targets in the first training image, the annotation information of the first training image, and the N candidate target features of the first training image, it is specifically used for:

[0206] From the N candidate target detection results of the first training image, select the candidate target detection results that match the annotation information of the first training image; based on the selected candidate target detection results and the N candidate target features of the first training image, obtain the target features corresponding to the annotated targets in the first training image.

[0207] Optionally, the pseudo-label generation module 1503 includes: a feature grouping submodule, a second candidate target detection result acquisition submodule, and a pseudo-label determination submodule.

[0208] The feature grouping submodule is used to divide the target features corresponding to each labeled target of the same type into a group, resulting in several target feature groups.

[0209] The second candidate object detection result acquisition submodule is used to perform object detection on the second training image using the first object detection model and supplemented by several object feature groups, so as to obtain several candidate object detection results of the second training image.

[0210] The pseudo-label determination submodule is used to determine the pseudo-label of the second training image based on the detection results of several candidate objects in the second training image.

[0211] Optionally, when the second candidate object detection result acquisition submodule uses the first object detection model, supplemented by several object feature groups, to perform object detection on the second training image, it is specifically used for:

[0212] Using the first target detection model, image features of the second training image are obtained;

[0213] Using the first target detection model, supplemented by several target feature groups, the image features of the second training image are decoded to obtain several candidate target features of the second training image.

[0214] Using the first target detection model, each candidate target feature of the second training image is converted into a candidate target detection result, resulting in several candidate target detection results for the second training image.

[0215] Optionally, when the second candidate object detection result acquisition submodule uses the first object detection model, supplemented by several target feature groups, to decode the image features of the second training image, it is specifically used for:

[0216] Traverse several target feature groups: Based on the first target detection model and supplemented by the currently traversed target feature groups, decode the second feature of the second training image to obtain N candidate target features of the second training image.

[0217] Optionally, each candidate target detection result in the second training image corresponds to a confidence level;

[0218] The pseudo-label determination submodule, when determining pseudo-labels for the second training image based on the detection results of several candidate targets, is specifically used for:

[0219] Candidate target detection results with a confidence level lower than a preset confidence threshold are filtered out from several candidate target detection results of the second training image; multiple candidate target detection results for the same target in the remaining candidate target detection results are deduplicated, and the remaining candidate target detection results are used as pseudo-labels of the second training image.

[0220] Optionally, the first object detection model includes a feature extractor, an encoder, a decoder, and a mapper.

[0221] The second candidate object detection result acquisition submodule, when using the first object detection model and supplemented by several object feature groups to perform object detection on the second training image, is specifically used for:

[0222] The feature extractor of the first object detection model is used to extract features from the second training image to obtain the first feature of the second training image.

[0223] The encoder of the first object detection model is used to encode the first feature of the second training image to obtain the second feature of the second training image.

[0224] Using the decoder of the first object detection model, supplemented by several target feature groups, the second feature of the second training image is decoded to obtain several candidate target features of the second training image.

[0225] Using the mapper of the first object detection model, each candidate object feature of the second training image is mapped to a candidate object detection result, thus obtaining several candidate object detection results of the second training image.

[0226] Optionally, the decoder includes M cascaded decoding modules, each with the same structure, where M is an integer greater than 1; each target feature in each target feature group is a feature sequence, and the i-th feature in each feature sequence is the feature output by the i-th decoding module, where 1 <= i <= M.

[0227] When using the decoder of the first object detection model, supplemented by several target feature groups, to decode the second feature of the second training image, the input of each decoding module includes target query features, reference features and the second feature of the second training image, and the output of each decoding module includes N candidate target features. Each decoding module uses the target query features and supplemented by reference features to extract features related to the object detection task from the second feature of the second training image.

[0228] Wherein, the target query feature input to the first decoding module is the N features created, the reference feature input to the first decoding module is several features created, the target query feature input to the j-th decoding module is the N candidate target features output by the (j-1)-th decoding module, and the reference feature input to the j-th decoding module is the feature output by the (j-1)-th decoding module in each feature sequence of the several target feature groups, 2<=j<=M.

[0229] On the one hand, the pseudo-label generation device provided in this embodiment of the invention can automatically generate pseudo-labels for unlabeled training images, greatly reducing labor costs compared to manual labeling. On the other hand, when generating pseudo-labels for unlabeled second training images, the pseudo-label generation device provides reference to the target features corresponding to labeled targets in the first training image, that is, it references the target features corresponding to known targets. Referencing the target features corresponding to known targets can improve the recognition accuracy of the first target detection model for potential targets, thereby generating more accurate pseudo-labels. In summary, the pseudo-label generation device provided in this embodiment of the invention can automatically generate accurate pseudo-labels for unlabeled training images.

[0230] This invention also provides a target detection model training device. The target detection model training device provided in this invention will be described below. The target detection model training device described below can be referred to in correspondence with the target detection model training method described above.

[0231] Please see Figure 16 The diagram shows a schematic of the structure of the target detection model training device 1600 provided in the embodiment of the present invention, which may include: a training data acquisition module 1601, a pseudo-label generation device 1500 provided in the above embodiment, and a second training module 1602.

[0232] The training data acquisition module 1601 is used to acquire the labeled first training image and the unlabeled second training image;

[0233] The pseudo-label generation device 1500 is used to generate pseudo-labels for the second training image;

[0234] The second training module 1602 is used to train the second target detection model using the labeled first training image and the second training image with pseudo-labels to obtain the final target detection model.

[0235] The second target detection model is either a constructed target detection model or a model obtained by initially training the constructed target detection model using the labeled first training image.

[0236] The target detection model training device provided in this embodiment of the invention can train a target detection model with better performance.

[0237] This invention also provides a target detection device. The target detection device provided in this invention will be described below. The target detection device described below can be referred to in correspondence with the target detection method described above.

[0238] The target detection device provided in this embodiment of the invention may include: an image acquisition module and a target detection module.

[0239] The image acquisition module is used to acquire the image to be detected.

[0240] The object detection module is used to perform object detection on the image to be detected using a pre-trained object detection model.

[0241] The target detection model is obtained based on the target detection model training device provided in the above embodiments.

[0242] Since the target detection model training device provided in the above embodiments can obtain a target detection model with better performance, using this target detection model to perform target detection on the image to be detected can achieve better target detection results.

[0243] This invention also provides a processing device; please refer to [link / reference]. Figure 17 The diagram shows the structure of the processing device, which may include: at least one processor 1701, at least one communication interface 1702, at least one memory 1703 and at least one communication bus 1704.

[0244] In this embodiment of the invention, the number of processor 1701, communication interface 1702, memory 1703 and communication bus 1704 is at least one, and processor 1701, communication interface 1702 and memory 1703 communicate with each other through communication bus 1704.

[0245] Processor 1701 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0246] The memory 1703 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0247] The memory stores a program, which the processor can call. The program is used for:

[0248] The first target detection model is trained using the labeled first training image to obtain the first target detection model. The target features corresponding to the labeled targets in the first training image are obtained using the first target detection model to obtain the target features corresponding to several known targets. The pseudo-labels of the unlabeled second training image are generated using the first target detection model and the target features corresponding to several known targets.

[0249] Optionally, the refined and extended functions of the program can be found in the description above.

[0250] This invention also provides a computer-readable storage medium that stores a program suitable for execution by a processor, the program being used for:

[0251] The first target detection model is trained using the labeled first training image to obtain the first target detection model. The target features corresponding to the labeled targets in the first training image are obtained using the first target detection model to obtain the target features corresponding to several known targets. The pseudo-labels of the unlabeled second training image are generated using the first target detection model and the target features corresponding to several known targets.

[0252] Optionally, the refined and extended functions of the program can be found in the description above.

[0253] This invention also provides a processing device, the structure of which is as follows: Figure 17 As shown, the processing device has a memory that stores a program, and the processor can call the program stored in the memory. The program is used for:

[0254] Obtain a labeled first training image and an unlabeled second training image; generate pseudo-labels for the second training image using the pseudo-label generation method provided in the above embodiment; train the second target detection model using the labeled first training image and the second training image with pseudo-labels to obtain the final target detection model, wherein the second target detection model is the constructed target detection model, or the model obtained by initially training the constructed target detection model using the labeled first training image.

[0255] Optionally, the refined and extended functions of the program can be found in the description above.

[0256] This invention also provides a computer-readable storage medium that stores a program suitable for execution by a processor, the program being used for:

[0257] Obtain a labeled first training image and an unlabeled second training image; generate pseudo-labels for the second training image using the pseudo-label generation method provided in the above embodiment; train the second target detection model using the labeled first training image and the second training image with pseudo-labels to obtain the final target detection model, wherein the second target detection model is the constructed target detection model, or the model obtained by initially training the constructed target detection model using the labeled first training image.

[0258] Optionally, the refined and extended functions of the program can be found in the description above.

[0259] This invention also provides a processing device, the structure of which is as follows: Figure 17 As shown, the processing device has a memory that stores a program, and the processor can call the program stored in the memory. The program is used for:

[0260] Acquire the image to be detected; use the pre-trained target detection model to perform target detection on the image to be detected; wherein, the target detection model is trained using the target detection model training method provided in the above embodiments.

[0261] Optionally, the refined and extended functions of the program can be found in the description above.

[0262] This invention also provides a computer-readable storage medium that stores a program suitable for execution by a processor, the program being used for:

[0263] Acquire the image to be detected; use the pre-trained target detection model to perform target detection on the image to be detected; wherein, the target detection model is trained using the target detection model training method provided in the above embodiments.

[0264] Optionally, the refined and extended functions of the program can be found in the description above.

[0265] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0266] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0267] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating pseudo-tags, characterized in that, include: The first target detection model is trained using the labeled first training image to obtain the first target detection model; Using the first target detection model, target features corresponding to the labeled targets in the first training image are obtained, and target features corresponding to several known targets are obtained respectively; Using the first target detection model and supplemented by the target features corresponding to the known targets respectively, pseudo-labels for unlabeled second training images are generated. The second training images with pseudo-labels are used to train the constructed target detection model or the first target detection model. The step of using the first target detection model to obtain the target features corresponding to the labeled targets in the first training image includes: Based on the first target detection model, N candidate target features of the first training image are obtained, and each candidate target feature is converted into a candidate target detection result to obtain N candidate target detection results, where N is an integer greater than 1; Based on the N candidate target features and the candidate target detection results that match the annotation information of the first training image among the N candidate target detection results, the target features corresponding to the annotated targets in the first training image are determined; The step of generating pseudo-labels for unlabeled second training images using the first target detection model, supplemented by the target features corresponding to the known targets, includes: The target features corresponding to each labeled target of the same type are grouped together to obtain several target feature groups; Using the first target detection model and supplemented by the several target feature groups, target detection is performed on the second training image to obtain several candidate target detection results of the second training image; Based on the detection results of several candidate targets in the second training image, the pseudo-label of the second training image is determined.

2. The pseudo-tag generation method according to claim 1, characterized in that, The step of determining the target features corresponding to the labeled targets in the first training image based on the N candidate target features and the candidate target detection results that match the annotation information of the first training image from the N candidate target detection results includes: From the N candidate target detection results of the first training image, select the candidate target detection results that match the annotation information of the first training image; Based on the selected candidate target detection results and the N candidate target features of the first training image, the target features corresponding to the labeled targets in the first training image are obtained.

3. The pseudo-tag generation method according to claim 1, characterized in that, The step of using the first object detection model, supplemented by the plurality of object feature groups, to perform object detection on the second training image, and obtaining a plurality of candidate object detection results for the second training image, includes: Using the first target detection model, the image features of the second training image are obtained; Using the first target detection model and supplemented by the several target feature groups, the image features of the second training image are decoded to obtain several candidate target features of the second training image; Using the first object detection model, each candidate object feature of the second training image is converted into a candidate object detection result, resulting in several candidate object detection results for the second training image.

4. The pseudo-tag generation method according to claim 3, characterized in that, The first target detection model, supplemented by the plurality of target feature groups, is used to decode the image features of the second training image to obtain a plurality of candidate target features of the second training image, including: Traverse the aforementioned target feature groups: Based on the first target detection model and supplemented by the target feature group currently being traversed, the second feature of the second training image is decoded to obtain N candidate target features of the second training image.

5. The pseudo-tag generation method according to claim 1, characterized in that, Each candidate target detection result in the second training image corresponds to a confidence level; The step of determining the pseudo-label of the second training image based on the detection results of several candidate targets in the second training image includes: Filter out candidate target detection results in the second training image whose confidence level is less than a preset confidence threshold. The remaining candidate target detection results are deduplicated for multiple candidate target detection results for the same target, and the remaining candidate target detection results are used as pseudo-labels for the second training image.

6. The pseudo-tag generation method according to claim 1, characterized in that, The first target detection model includes: a feature extractor, an encoder, a decoder, and a mapper; The step of using the first object detection model, supplemented by the plurality of object feature groups, to perform object detection on the second training image, and obtaining a plurality of candidate object detection results for the second training image, includes: The feature extractor of the first target detection model is used to extract features from the second training image to obtain the first feature of the second training image; The encoder of the first target detection model is used to encode the first feature of the second training image to obtain the second feature of the second training image; Using the decoder of the first target detection model, supplemented by the several target feature groups, the second feature of the second training image is decoded to obtain several candidate target features of the second training image; Using the mapper of the first object detection model, each candidate object feature of the second training image is mapped to a candidate object detection result, thereby obtaining several candidate object detection results of the second training image.

7. The pseudo-tag generation method according to claim 6, characterized in that, The decoder comprises M cascaded decoding modules, each with the same structure, where M is an integer greater than 1; Each target feature in each target feature group is a feature sequence, and the i-th feature in each feature sequence is the feature output by the i-th decoding module, 1<=i<=M; When using the decoder of the first object detection model, supplemented by the several target feature groups, to decode the second feature of the second training image, the input of each decoding module includes target query features, reference features and the second feature of the second training image, and the output of each decoding module includes N candidate target features. Each decoding module uses the target query features and supplemented by the reference features to extract features related to the object detection task from the second feature of the second training image. Wherein, the target query feature input to the first decoding module is the N features created, the reference feature input to the first decoding module is several features created, the target query feature input to the j-th decoding module is the N candidate target features output by the (j-1)-th decoding module, and the reference feature input to the j-th decoding module is the feature output by the (j-1)-th decoding module in each feature sequence of the several target feature groups, 2<=j<=M.

8. A method for training an object detection model, characterized in that, include: Obtain the labeled first training image and the unlabeled second training image; The pseudo-labels for the second training image are generated using the pseudo-label generation method as described in any one of claims 1 to 7. The second target detection model is trained using the labeled first training image and the second training image with pseudo-labels to obtain the final target detection model. The second target detection model is either a constructed target detection model or a model obtained by initially training the constructed target detection model using the labeled first training image.

9. A target detection method, characterized in that, include: Acquire the image to be detected; The target detection model, which has been pre-trained, is used to perform target detection on the image to be detected. The target detection model is trained using the target detection model training method as described in claim 8.

10. A pseudo-label generation device, characterized in that, include: The first training module, the known target feature acquisition module, and the pseudo-label generation module; The first training module is used to train the constructed target detection model using the labeled first training image to obtain the first target detection model; The known target feature acquisition module is used to acquire the target features corresponding to the labeled targets in the first training image using the first target detection model, and obtain the target features corresponding to several known targets respectively; The pseudo-label generation module is used to generate pseudo-labels for unlabeled second training images by using the first target detection model and supplementing it with the target features corresponding to the several known targets. The second training images with pseudo-labels are used to train the constructed target detection model or the first target detection model. The known target feature acquisition module is specifically used to acquire N candidate target features of the first training image based on the first target detection model, and convert each candidate target feature into a candidate target detection result, where N is an integer greater than 1. Based on the N candidate target features and the candidate target detection results that match the annotation information of the first training image among the N converted candidate target detection results, the target features corresponding to the annotated targets in the first training image are determined. The pseudo-label generation module is specifically used to divide the target features corresponding to each labeled target of the same type into a group to obtain several target feature groups. Using the first target detection model and the several target feature groups, target detection is performed on the second training image to obtain several candidate target detection results of the second training image. Based on the several candidate target detection results of the second training image, the pseudo-label of the second training image is determined.

11. A target detection model training device, characterized in that, include: The training data acquisition module, the pseudo-label generation device as described in claim 10, and the second training module; The training data acquisition module is used to acquire the labeled first training image and the unlabeled second training image; The pseudo-label generation module is used to generate pseudo-labels for the second training image; The second training module is used to train the second object detection model using the labeled first training image and the second training image with pseudo-labels to obtain the final object detection model. The second object detection model is either a constructed object detection model or a model obtained by initially training the constructed object detection model using the labeled first training image.

12. A target detection device, characterized in that, include: Image acquisition module and target detection module; The image acquisition module is used to acquire the image to be detected; The target detection module is used to perform target detection on the image to be detected using a pre-trained target detection model; The target detection model is obtained using the target detection model training device as described in claim 11.

13. A processing apparatus, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the steps of the pseudo-label generation method as described in any one of claims 1 to 7, or to implement the steps of the target detection model training method as described in claim 8, or to implement the steps of the target detection method as described in claim 9.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the pseudo-label generation method as described in any one of claims 1 to 7, or implements the steps of the target detection model training method as described in claim 8, or implements the steps of the target detection method as described in claim 9.

Images