A method for training an object detection model, an object detection method and apparatus

Abstract

The embodiments of the present application provide a target detection model training method, a target detection method and a device, which relate to the field of deep learning technology. Based on a first detection model, each first sample image is detected to obtain a pseudo label of the object in each first sample image; first object image features of the object in each first sample image and second object image features of the object in each second sample image are obtained; based on each first object image feature and each second object image feature, a sample image to be calibrated is determined from each first sample image; based on the pseudo label of the object in the third sample image in each first sample image, the true label of the object in each second sample image, and the true label of the object in the sample image to be calibrated, a second detection model to be trained is trained to obtain a target detection model for the current detection scene, which can reduce detection costs and improve the generation efficiency of detection models.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a target detection model training method, a target detection method and a device. Background Art

[0002] With the rapid development of computer technology, image detection has been widely used in various fields. For example, in the field of video surveillance, target detection in surveillance images based on trained target detection models can determine the presence of predefined objects (such as animals or people) in the surveillance images, as well as the image area occupied by the predefined objects.

[0003] In related technologies, in order to improve the accuracy of target detection, for a certain detection scenario, it is necessary to obtain a large number of sample images corresponding to the detection scenario that are manually labeled with labels of target objects. Then, a convolutional neural network model with a preset structure can be trained based on these sample images to obtain a target detection model corresponding to the detection scenario.

[0004] However, manually labeling a large number of sample images will increase the complexity and cost of labeling, which will in turn increase the cost of detection and reduce the efficiency of generating detection models. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a target detection model training method, target detection method, and device to reduce the cost of detection and improve the efficiency of generating the detection model. The specific technical solution is as follows:

[0006] In a first aspect, in order to achieve the above-mentioned objectives, the present application discloses a method for training a target detection model, the method comprising:

[0007] Obtaining each first sample image of the current detection scene without a label;

[0008] Detecting each first sample image based on a first detection model to obtain a label of the object in each first sample image as a pseudo label; wherein the first detection model is trained based on each second sample image with a label of other detection scenes other than the current detection scene;

[0009] Acquire an image feature of an object in each first sample image as a first object image feature, and acquire an image feature of an object in each second sample image as a second object image feature;

[0010] Based on each first object image feature and each second object image feature, determining a sample image to be calibrated from each first sample image to obtain a true label of an object in the sample image to be calibrated by a user; wherein the sample image to be calibrated does not match the other detection scenes;

[0011] Based on the pseudo labels of objects in the third sample images other than the sample images to be calibrated in each first sample image, the true labels of objects in each second sample image, and the true labels of objects in the sample images to be calibrated, the second detection model to be trained is trained to obtain a target detection model for the current detection scene.

[0012] Optionally, determining the sample image to be calibrated from each first sample image based on each first object image feature and each second object image feature includes:

[0013] Clustering each second object image feature to obtain a plurality of object image feature clusters as first object image feature clusters;

[0014] determining a candidate object image feature from each first object image feature based on a distance from each first object image feature cluster; wherein the candidate object image feature does not match the second object image feature;

[0015] Based on the first sample images corresponding to the determined image features of each candidate object, a sample image to be calibrated is determined.

[0016] Optionally, each first sample image includes a plurality of sample image sets; each sample image set includes an unlabeled sample image to be enhanced of the current detection scene, and an enhanced sample image obtained by performing image enhancement processing on the sample image to be enhanced;

[0017] The determining of candidate object image features from each first object image feature based on the distance between the candidate object image features and each first object image feature cluster includes:

[0018] Clustering each first object image feature to obtain a plurality of object image feature clusters as second object image feature clusters;

[0019] determining a center of each second object image feature cluster as a central object image feature, and determining object image features other than the second object image feature cluster in each first object image feature as out-of-cluster object image features;

[0020] determining a first candidate object image feature from each central object image feature based on a distance from each first object image feature cluster; wherein the distance between the first candidate object image feature and each first object image feature cluster is greater than the distance between other object image features in each central object image feature and each first object image feature cluster;

[0021] Determining a second candidate object image feature from each out-of-cluster object image feature based on the distance from each first object image feature cluster and the label stability; wherein the label stability of an out-of-cluster object image feature represents: the degree of consistency of pseudo labels of each object corresponding to the out-of-cluster object image feature in each image included in the sample image set corresponding to the out-of-cluster object image feature;

[0022] The first candidate object image feature and the second candidate object image feature are determined as final candidate object image features.

[0023] Optionally, determining the first candidate object image feature from each central object image feature based on the distance from each first object image feature cluster includes:

[0024] For each central object image feature that is not currently selected, calculating the distance between the central object image feature and the current first set of image features to be compared as the feature distance corresponding to the central object image feature; wherein the current first set of image features to be compared includes the centers of each first object image feature cluster;

[0025] Select the central object image feature with the largest corresponding feature distance as the first candidate object image feature, add the central object image feature with the largest corresponding feature distance to the current first image feature set to be compared, and return to execute the step of calculating the distance between the central object image feature and the current first image feature set to be compared for each central object image feature that is not currently selected as the feature distance corresponding to the central object image feature, until a first preset number of central object image features are determined as the first candidate object image feature.

[0026] Optionally, before determining the second candidate object image features from each out-of-cluster object image feature based on the distance from each first object image feature cluster and the label stability, the method further includes:

[0027] For each out-of-cluster object image feature, calculating the entropy of the average value of the confidence of the pseudo-label of each object corresponding to the out-of-cluster object image feature in each image included in the sample image set corresponding to the out-of-cluster object image feature as a first value, and the average value of the entropy of the confidence of the pseudo-label of each object corresponding to the out-of-cluster object image feature as a second value;

[0028] The difference between the first value and the second value is calculated as the label stability of the image feature of the out-of-cluster object.

[0029] Optionally, determining the second candidate object image features from each out-of-cluster object image feature based on the distance from each first object image feature cluster and the label stability includes:

[0030] For each currently unselected out-of-cluster object image feature, calculating the distance between the out-of-cluster object image feature and the current second set of image features to be compared as the feature distance corresponding to the out-of-cluster object image feature; wherein the current second set of image features to be compared includes the centers of each first object image feature cluster;

[0031] Calculate the label stability of the image features of the out-of-cluster objects and the weighted sum of the corresponding feature distances to obtain the target value;

[0032] The out-of-cluster object image feature with the largest corresponding target value is selected as the second candidate object image feature, the out-of-cluster object image feature with the largest corresponding target value is added to the current second set of image features to be compared, and the step of calculating, for each out-of-cluster object image feature not currently selected, the distance between the out-of-cluster object image feature and the current second set of image features to be compared as the feature distance corresponding to the out-of-cluster object image feature is returned to execution, until a second preset number of out-of-cluster object image features are determined as the second candidate object image features.

[0033] Optionally, determining the sample image to be calibrated based on the first sample image corresponding to each determined image feature of the candidate object includes:

[0034] For each determined candidate object image feature, a sample image to be enhanced in the sample image set to which the first sample image corresponding to the candidate object image feature belongs is determined as a sample image to be calibrated.

[0035] Optionally, before performing model training on the second detection model to be trained based on the pseudo labels of objects in third sample images other than the sample images to be calibrated in each of the first sample images, the true labels of objects in each of the second sample images, and the true labels of objects in the sample images to be calibrated to obtain the object detection model for the current detection scene, the method further includes:

[0036] The pseudo label of the object corresponding to the object image feature of the object in each sample image to be calibrated that belongs to the same second object image feature cluster is replaced by the real label marked by the user for the object in the sample image to be calibrated.

[0037] Optionally, acquiring the image features of the objects in each first sample image as the first object image features, and acquiring the image features of the objects in each second sample image as the second object image features, includes:

[0038] For each first sample image, input the first sample image into the first detection model and the universal detection model, respectively, to obtain first object image features of the object in the first sample image output by the feature extraction layers of the first detection model and the universal detection model; wherein the universal detection model is trained based on sample images of multiple different detection scenarios;

[0039] For each second sample image, the second sample image is input into the first detection model and the universal detection model respectively to obtain the second object image features of the object in the second sample image output by the feature extraction layers of the first detection model and the universal detection model.

[0040] In a second aspect, in order to achieve the above-mentioned objectives, the present application discloses a target detection method, which includes:

[0041] Obtain the image to be detected;

[0042] The image to be detected is input into the target detection model of the current detection scene to obtain the detection result of the image to be detected; wherein, the target detection model is obtained by using the target detection model training method described in any one of the first aspects above.

[0043] Optionally, the detection result includes the position of the object in the image to be detected, and / or the probability that the image to be detected contains a preset object.

[0044] In a third aspect, in order to achieve the above-mentioned objectives, the present application discloses an object detection model training device, the device comprising:

[0045] A first sample image acquisition module is used to acquire first sample images without labels of the current detection scene;

[0046] a pseudo-label acquisition module, configured to detect each first sample image based on a first detection model and obtain a label of the object in each first sample image as a pseudo-label; wherein the first detection model is obtained by training each second sample image with a label of a detection scene other than the current detection scene;

[0047] an object image feature acquisition module, configured to acquire image features of objects in each first sample image as first object image features, and to acquire image features of objects in each second sample image as second object image features;

[0048] a module for determining sample images to be calibrated, configured to determine a sample image to be calibrated from each first sample image based on each first object image feature and each second object image feature, so as to obtain a true label of an object in the sample image to be calibrated, the sample image to be calibrated not matching the other detection scenes;

[0049] The model training module is used to perform model training on the second detection model to be trained based on the pseudo labels of objects in the third sample images other than the sample images to be calibrated in each first sample image, the true labels of objects in each second sample image, and the true labels of objects in the sample images to be calibrated, so as to obtain the target detection model of the current detection scene.

[0050] Optionally, the module for determining the sample image to be calibrated includes:

[0051] a first object image feature cluster determination submodule, configured to cluster the second object image features to obtain a plurality of object image feature clusters as first object image feature clusters;

[0052] a candidate object image feature determination submodule, configured to determine a candidate object image feature from each first object image feature based on a distance from each first object image feature cluster; wherein the candidate object image feature does not match the second object image feature;

[0053] The submodule for determining the sample image to be calibrated is used to determine the sample image to be calibrated based on the first sample image corresponding to the determined image features of each candidate object.

[0054] Optionally, each first sample image includes a plurality of sample image sets; each sample image set includes an unlabeled sample image to be enhanced of the current detection scene, and an enhanced sample image obtained by performing image enhancement processing on the sample image to be enhanced;

[0055] The candidate object image feature determination submodule includes:

[0056] a second object image feature cluster determining unit, configured to cluster the first object image features to obtain a plurality of object image feature clusters as second object image feature clusters;

[0057] a first processing unit configured to determine a center of each second object image feature cluster as a central object image feature, and to determine object image features other than the second object image feature cluster in each first object image feature as out-of-cluster object image features;

[0058] a first candidate object image feature determining unit configured to determine a first candidate object image feature from each central object image feature based on a distance from each first object image feature cluster; wherein the distance between the first candidate object image feature and each first object image feature cluster is greater than the distance between other object image features in each central object image feature and each first object image feature cluster;

[0059] a second candidate object image feature determining unit configured to determine a second candidate object image feature from each out-of-cluster object image feature based on a distance from each first object image feature cluster and a label stability; wherein the label stability of an out-of-cluster object image feature indicates a degree of consistency between pseudo labels of each object corresponding to the out-of-cluster object image feature in each image included in the sample image set corresponding to the out-of-cluster object image feature;

[0060] The final candidate object image feature determining unit is configured to determine the first candidate object image feature and the second candidate object image feature as the final candidate object image feature.

[0061] Optionally, the first candidate object image feature determination unit is specifically configured to calculate, for each central object image feature that is not currently selected, a distance between the central object image feature and a current first set of image features to be compared, as a feature distance corresponding to the central object image feature; wherein the current first set of image features to be compared includes the centers of each first object image feature cluster;

[0062] Select the central object image feature with the largest corresponding feature distance as the first candidate object image feature, add the central object image feature with the largest corresponding feature distance to the current first image feature set to be compared, and return to execute the step of calculating the distance between the central object image feature and the current first image feature set to be compared for each central object image feature that is not currently selected as the feature distance corresponding to the central object image feature, until a first preset number of central object image features are determined as the first candidate object image feature.

[0063] Optionally, the device further includes:

[0064] a label stability determination module configured to calculate, for each out-of-cluster object image feature, an entropy of an average value of confidences of pseudo labels of each object corresponding to the out-of-cluster object image feature in each image included in the sample image set corresponding to the out-of-cluster object image feature, as a first value, and an average value of entropies of confidences of pseudo labels of each object corresponding to the out-of-cluster object image feature, as a second value, before determining the second candidate object image feature from each out-of-cluster object image feature based on the distance to each first object image feature cluster and the label stability;

[0065] The difference between the first value and the second value is calculated as the label stability of the image feature of the out-of-cluster object.

[0066] Optionally, the second candidate object image feature determination unit is specifically configured to calculate, for each out-of-cluster object image feature that is not currently selected, a distance between the out-of-cluster object image feature and a current second set of image features to be compared, as the feature distance corresponding to the out-of-cluster object image feature; wherein the current second set of image features to be compared includes the centers of each first object image feature cluster;

[0067] Calculate the label stability of the image features of the out-of-cluster objects and the weighted sum of the corresponding feature distances to obtain the target value;

[0068] The out-of-cluster object image feature with the largest corresponding target value is selected as the second candidate object image feature, the out-of-cluster object image feature with the largest corresponding target value is added to the current second set of image features to be compared, and the step of calculating, for each out-of-cluster object image feature not currently selected, the distance between the out-of-cluster object image feature and the current second set of image features to be compared as the feature distance corresponding to the out-of-cluster object image feature is returned to execution, until a second preset number of out-of-cluster object image features are determined as the second candidate object image features.

[0069] Optionally, the submodule for determining the sample image to be calibrated is specifically configured to determine, for each determined candidate object image feature, a sample image to be enhanced in the sample image set to which the first sample image corresponding to the candidate object image feature belongs, as the sample image to be calibrated.

[0070] Optionally, the device further includes:

[0071] A replacement module is used to replace the pseudo labels of objects corresponding to object image features belonging to the same second object image feature cluster as the object image features of the objects in each sample image to be calibrated with the real labels marked by the user for the objects in the sample image to be calibrated before training the second detection model to be trained based on the pseudo labels of objects in the third sample image other than the sample image to be calibrated in each first sample image, the real labels of objects in each second sample image, and the real labels of objects in the sample image to be calibrated to obtain the target detection model of the current detection scene.

[0072] Optionally, the object image feature acquisition module is specifically configured to input each first sample image into the first detection model and the universal detection model, respectively, to obtain first object image features of the object in the first sample image output by the feature extraction layers of the first detection model and the universal detection model; wherein the universal detection model is trained based on sample images of multiple different detection scenarios;

[0073] For each second sample image, the second sample image is input into the first detection model and the universal detection model respectively to obtain the second object image features of the object in the second sample image output by the feature extraction layers of the first detection model and the universal detection model.

[0074] In a fourth aspect, in order to achieve the above-mentioned objectives, an embodiment of the present application discloses a target detection device, the device comprising:

[0075] An image acquisition module to be detected, used to acquire the image to be detected;

[0076] A detection module is used to input the image to be detected into the target detection model of the current detection scene to obtain the detection result of the image to be detected; wherein, the target detection model is obtained using the target detection model training method described in any one of the first aspects above.

[0077] Optionally, the detection result includes the position of the object in the image to be detected, and / or the probability that the image to be detected contains a preset object.

[0078] In another aspect of the present application, in order to achieve the above-mentioned purpose, an embodiment of the present application further discloses an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0079] The memory is used to store computer programs;

[0080] The processor is used to implement the target detection model training method described in the first aspect or the target detection method described in the second aspect when executing the program stored in the memory.

[0081] In another aspect of the implementation of the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the target detection model training method described in the first aspect or the target detection method described in the second aspect is implemented.

[0082] In another aspect of the implementation of the present application, an embodiment of the present application also provides a computer program product comprising instructions, which, when run on a computer, enables the computer to execute the target detection model training method described in the first aspect above, or the target detection method described in the second aspect.

[0083] An embodiment of the present application provides a target detection model training method, which can obtain each first sample image without a label of the current detection scene; detect each first sample image based on the first detection model to obtain the label of the object in each first sample image as a pseudo label; wherein the first detection model is: obtained by training each second sample image with a label of other detection scenes other than the current detection scene; obtain the image features of the object in each first sample image as the first object image features, and obtain the image features of the object in each second sample image as the second object image features; based on each first object image feature and each second object image feature, determine the sample image to be calibrated from each first sample image to obtain the true label marked by the user for the object in the sample image to be calibrated; wherein the sample image to be calibrated does not match the other detection scenes; based on the pseudo labels of the objects in the third sample image other than the sample image to be calibrated in each first sample image, the true labels of the objects in each second sample image, and the true labels of the objects in the sample image to be calibrated, perform model training on the second detection model to be trained to obtain the target detection model of the current detection scene.

[0084] Based on the target detection model training method provided in the embodiment of the present application, it is possible to determine the sample image to be calibrated in the first sample image, and the sample image to be calibrated does not match other detection scenes. In other words, the first detection model has not learned the image features of the object in the sample image to be calibrated, and the accuracy of the pseudo-label of the object in the sample image to be calibrated is low. Therefore, such sample images can be pushed to the user for labeling to obtain their real labels for modeling the second detection model, so that the trained target detection model can learn the image features of the object in the sample image to be calibrated. Similarly, the accuracy of the pseudo-label of the object in the third sample image is high, so its pseudo-label can be directly used to model the second detection model. In this way, it is only necessary to manually label the objects in a part of the first sample image (i.e., the sample image to be calibrated) to obtain the target detection model of the current detection scene, thereby reducing the cost of detection and improving the generation efficiency of the detection model. In addition, using the trained first detection model to guide the training of the second detection model can reduce the false alarm rate of the target detection model obtained and improve the detection accuracy.

[0085] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0087] Figure 1 A flowchart of a target detection model training method provided in an embodiment of the present application;

[0088] Figure 2 A flowchart of another target detection model training method provided in an embodiment of the present application;

[0089] Figure 3 A flowchart of another target detection model training method provided in an embodiment of the present application;

[0090] Figure 4 A flowchart of another target detection model training method provided in an embodiment of the present application;

[0091] Figure 5 A schematic diagram illustrating the principles of a target detection model training method provided in an embodiment of the present application;

[0092] Figure 6 A schematic diagram of the structure of an active push module provided in an embodiment of the present application;

[0093] Figure 7 A structural diagram of a target detection model training device provided in an embodiment of the present application;

[0094] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0095] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0096] In related technologies, for a specific detection scenario, a large number of labeled sample images corresponding to the detection scenario need to be obtained. Then, based on these labeled sample images, a training model can be developed to obtain the target detection model corresponding to the detection scenario. However, manually labeling a large number of sample images increases the complexity and cost of labeling, which in turn increases the cost of detection and reduces the efficiency of generating the detection model.

[0097] To solve the above problems, an embodiment of the present application provides a method for training a target detection model, which can be applied to an electronic device. For example, the electronic device can be a server equipped with a GPU (Graphics Processing Unit) or a file storage server.

[0098] See also Figure 1 , Figure 1 A flowchart of a target detection model training method provided in an embodiment of the present application, the method may include the following steps:

[0099] S101: Acquire first sample images without labels of a current detection scene.

[0100] S102: Detect each first sample image based on the first detection model to obtain a label of the object in each first sample image as a pseudo label.

[0101] The first detection model is obtained by training based on second sample images with labeled labels of other detection scenes other than the current detection scene.

[0102] S103: Acquire image features of objects in each first sample image as first object image features, and acquire image features of objects in each second sample image as second object image features.

[0103] S104: Based on the first object image features and the second object image features, determine a sample image to be calibrated from the first sample images to obtain a true label of the object in the sample image to be calibrated by the user.

[0104] Among them, the sample image to be calibrated does not match other detection scenes.

[0105] S105: Based on the pseudo labels of objects in the third sample images other than the sample images to be calibrated in each first sample image, the true labels of objects in each second sample image, and the true labels of objects in the sample images to be calibrated, the second detection model to be trained is trained to obtain a target detection model for the current detection scene.

[0106] The target detection model training method provided in the embodiment of the present application can determine the sample image to be calibrated in the first sample image, and the sample image to be calibrated does not match other detection scenes. In other words, the first detection model has not learned the image features of the object in the sample image to be calibrated, and the accuracy of the pseudo-label of the object in the sample image to be calibrated is low. Therefore, such sample images can be pushed to the user for labeling to obtain their real labels for modeling the second detection model, so that the trained target detection model can learn the image features of the object in the sample image to be calibrated. Similarly, the accuracy of the pseudo-label of the object in the third sample image is high, so its pseudo-label can be directly used to model the second detection model. In this way, it is only necessary to manually label the objects in a part of the first sample image (i.e., the sample image to be calibrated) to obtain the target detection model of the current detection scene, thereby reducing the cost of detection and improving the generation efficiency of the detection model. In addition, using the trained first detection model to guide the training of the second detection model can reduce the false alarm rate of the target detection model obtained and improve the detection accuracy.

[0107] In practical applications, there are usually many different detection scenarios, such as detection scenarios for surveillance video images at park entrances, detection scenarios for surveillance video images at shopping mall entrances, and detection scenarios for surveillance video images at street intersections.

[0108] For each detection scene, the surveillance video image of the detection scene can be detected based on the corresponding object detection model to determine the label of the object in the surveillance video image. The object in the surveillance video image can be an animal, a person, or a vehicle.

[0109] Since there are differences between surveillance video images of different detection scenarios, for each detection scenario, model training can be performed based on the sample images of the detection scenario to obtain a target detection model suitable for the detection scenario (i.e., the target detection model corresponding to the detection scenario).

[0110] Regarding step S101, in one embodiment, to enrich the first sample images and improve the accuracy of the object detection model trained based on the first sample images, each first sample image includes multiple sample image sets. Each sample image set includes an unlabeled sample image to be enhanced of the current detection scene, and an enhanced sample image obtained by performing image enhancement processing on the sample image to be enhanced. The label of the sample image may include the location and category of the object in the sample image.

[0111] That is, a plurality of unlabeled sample images to be enhanced of the current detection scene can be obtained, and then image enhancement processing is performed on each sample image to obtain a corresponding enhanced sample image. Each sample image to be enhanced and the corresponding enhanced sample image are the first sample images.

[0112] The image enhancement process may include at least one of the following: adjusting the resolution of the sample image to be enhanced (eg, increasing the resolution, decreasing the resolution), and adjusting the brightness of the sample image to be enhanced (eg, increasing the brightness, decreasing the brightness).

[0113] With respect to step S102 , when it is necessary to obtain a target detection model applicable to the current detection scene, a first detection model may be obtained, that is, a detection model obtained by training based on second sample images of detection scenes other than the current detection scene.

[0114] In one embodiment, the first detection model can be a fast region convolutional neural network (Faster-RCNN, Faster-Region Convolutional Neural Networks) model, or, it can also be a YOLO model, or, it can also be a masked region convolutional neural network (Mask-RCNN, Mask-Regi on Convolutional Neural Networks) model.

[0115] It is understandable that because the first detection model is trained based on sample images of other detection scenarios, it may not be well suited for the current detection scenario. In other words, the accuracy of the pseudo-labels of objects in the first sample images determined based on the first detection model may be high or low.

[0116] Regarding step S103, in one embodiment, the following steps may be included:

[0117] Step 1: For each first sample image, input the first sample image into the first detection model and the universal detection model respectively to obtain the first object image features of the object in the first sample image output by the feature extraction layers of the first detection model and the universal detection model.

[0118] Among them, the general detection model is obtained by training based on sample images of multiple different detection scenarios.

[0119] Step 2: For each second sample image, input the second sample image into the first detection model and the universal detection model respectively to obtain the second object image features of the object in the second sample image output by the feature extraction layers of the first detection model and the universal detection model.

[0120] In one embodiment of the present application, the first detection model may include a ROIAlign (Region of Interest Align) layer. For the sample images (i.e., the first sample image and the second sample image), image features output by the ROIAlign layer may be obtained and processed by Global Average Pooling to obtain image features of objects in the sample images.

[0121] In one implementation, the universal detection model is obtained by training a convolutional neural network model based on sample images of multiple different detection scenarios. That is, the sample images used to train the universal detection model may include the second sample image, and may also include sample images of other scenarios with labels that are different from the second sample image and the first sample image.

[0122] That is, in the embodiment of the present application, for each first sample image, object image features obtained based on the first detection model (which may be referred to as domain image features) can be obtained, and object image features obtained based on the universal detection model (which may be referred to as universal image features) can also be obtained. Similarly, for each second sample image, object image features obtained based on the first detection model, i.e., domain image features, can also be obtained, and object image features obtained based on the universal detection model, i.e., universal image features, can also be obtained.

[0123] Based on the above processing, combining two different detection models (i.e., the first detection model and the general detection model), the first object image features and the second object image features can be enriched. Furthermore, based on step S104, the reliability of the determined sample image to be calibrated can be improved to further improve the accuracy of the target detection model.

[0124] Regarding step S103 , in another embodiment, the first object image features of the objects in the first sample image and the second object image features of the objects in the second sample image may be extracted based only on the first detection model.

[0125] In another embodiment, the first object image feature of the object in the first sample image and the second object image feature of the object in the second sample image may be extracted based only on the general detection model.

[0126] With respect to step S104 , the sample image to be calibrated is the sample image that needs to be pushed to the user. Accordingly, the user can mark the objects in the sample image to be calibrated, and thus obtain the true labels of the objects in the sample image to be calibrated.

[0127] In the embodiment of the present application, since the sample image to be calibrated does not match the other detection scenes (i.e., the detection scene of the second sample image), the accuracy of the pseudo-labels of the objects in the sample image to be calibrated obtained based on the first detection model is relatively low, and the user needs to mark such sample images. Conversely, the accuracy of the pseudo-labels of the objects in sample images other than the sample image to be calibrated (i.e., the third sample image in the embodiment of the present application) is relatively high, and does not require user marking and can be used directly to train the second detection model.

[0128] In one embodiment, the object image features can be represented in the form of feature vectors. Accordingly, for each first object image feature, the sum of the distances between the first object image feature and each second object image feature can be calculated, and the first sample image corresponding to the first object image feature with the larger calculated sum is used as the sample image to be calibrated. The first sample image corresponding to the first object image feature is, that is, the first sample image to which the object corresponding to the first object image feature belongs.

[0129] In another embodiment, see Figure 2 ,exist Figure 1 On the basis of, the above step S104 may include the following steps:

[0130] S1041: Clustering the second object image features to obtain a plurality of object image feature clusters as first object image feature clusters.

[0131] S1042: Determine candidate object image features from each first object image feature based on the distance between each first object image feature cluster.

[0132] The candidate object image feature does not match the second object image feature.

[0133] S1043: Determine a sample image to be calibrated based on the first sample image corresponding to each determined image feature of the candidate object.

[0134] In the embodiment of the present application, during clustering, similar second object image features can be grouped into a first object image feature cluster. Each first object image feature cluster has a center (which can be referred to as a first center object image feature). By calculating the distance between the first object image feature and the first object image feature cluster to determine the sample image to be calibrated, rather than calculating the distance between the first object image feature and each second object image feature, the amount of computation can be reduced, thereby improving the efficiency of determining the sample image to be calibrated.

[0135] The clustering may be performed using a hierarchical clustering algorithm, or may be performed using, for example, a K-means (K-Means Clustering Algorithm).

[0136] In one implementation, the distance between the first object image feature and the first central object image feature may be calculated as the distance between the first object image feature and the first object image feature cluster.

[0137] Alternatively, the sum of the distances between the first object image feature and a specified number of second object image features in each first object image feature cluster may be calculated as the distance between the first object image feature and the first object image feature cluster.

[0138] Regarding step S1042, in one embodiment, for each first object image feature, the sum of the distances between the first object image feature and each first central object image feature can be calculated, and the first sample image corresponding to the first object image feature with the larger calculated sum value is used as the sample image to be calibrated.

[0139] In another embodiment, see Figure 3 ,exist Figure 2 On the basis of, the above step S1042 may include the following steps:

[0140] S10421: Clustering each first object image feature to obtain a plurality of object image feature clusters as second object image feature clusters.

[0141] S10422: Determine the center of each second object image feature cluster as the central object image feature, and determine the object image features other than the second object image feature cluster in each first object image feature as the out-of-cluster object image features.

[0142] S10423: Determine a first candidate object image feature from each central object image feature based on the distance from each first object image feature cluster.

[0143] The distance between the first candidate object image feature and each first object image feature cluster is greater than the distance between other object image features in each central object image feature and each first object image feature cluster.

[0144] S10424: Determine second candidate object image features from each out-of-cluster object image feature based on the distance from each first object image feature cluster and the label stability.

[0145] The label stability of an out-of-cluster object image feature indicates the degree of consistency of pseudo labels of objects corresponding to the out-of-cluster object image feature in each image contained in the sample image set corresponding to the out-of-cluster object image feature.

[0146] S10425: Determine the first candidate object image feature and the second candidate object image feature as the final candidate object image feature.

[0147] In the embodiment of the present application, in order to further reduce the amount of computation and improve the efficiency of determining the sample image to be calibrated, each first object image feature can be clustered to obtain multiple second object image feature clusters. Each second object image feature cluster has a center (which can be referred to as a second center object image feature). In addition, first object image features that do not belong to the second object image feature cluster (i.e., out-of-cluster object image features in the embodiment of the present application) can also be determined.

[0148] Accordingly, the second central object image features and the out-of-cluster object image features may be processed separately to determine candidate object image features therefrom.

[0149] That is, from each second central object image feature, an object image feature having a larger distance from each first object image feature cluster is determined as the first candidate object image feature.

[0150] From the out-of-cluster object image features, object image features having a larger distance from each first object image feature cluster and a lower label stability are determined as second candidate object image features.

[0151] The label stability is low, meaning that across all images in the sample image set corresponding to the out-of-cluster object image feature, the pseudo-labels for each object corresponding to the out-of-cluster object image feature are not consistent. This means that the first detection model cannot effectively identify the first sample image containing this type of out-of-cluster object image feature. In other words, the pseudo-labels for each object corresponding to this type of out-of-cluster object image feature are not accurate. Therefore, these labels can be pushed to the user to obtain the true labels.

[0152] In one embodiment, see Figure 4 ,exist Figure 3 On the basis of, the above step S10423 may include the following steps:

[0153] S104231: For each central object image feature that is not currently selected, calculate the distance between the central object image feature and the current first set of image features to be compared as the feature distance corresponding to the central object image feature.

[0154] The current first to-be-compared image feature set includes the centers of each first object image feature cluster.

[0155] S104232: Select the central object image feature with the largest corresponding feature distance as the first candidate object image feature, add the central object image feature with the largest corresponding feature distance to the current first image feature set to be compared, and return to execute step S104231 until a first preset number of central object image features are determined as the first candidate object image feature.

[0156] For example, the second central object image feature includes: central object image feature 1, central object image feature 2, central object image feature 3 and central object image feature 4; the first central object image feature includes: central object image feature 5, central object image feature 6, central object image feature 7 and central object image feature 8.

[0157] That is, the initial first set of image features to be compared includes: central object image feature 5 , central object image feature 6 , central object image feature 7 , and central object image feature 8 .

[0158] At this time, the currently unselected second central object image features include: central object image feature 1, central object image feature 2, central object image feature 3, and central object image feature 4. The sum of the distances between each currently unselected second central object image feature and each object image feature in the current first image feature set to be compared (i.e., feature distance) can be calculated respectively, and the second central object image feature with the largest corresponding feature distance (e.g., central object image feature 4) can be selected as the first candidate object image feature.

[0159] Then, the central object image feature 4 is added to the current first image feature set to be compared, that is, the current first image feature set to be compared includes: central object image feature 4, central object image feature 5, central object image feature 6, central object image feature 7 and central object image feature 8.

[0160] Accordingly, the currently unselected second central object image features include: central object image feature 1, central object image feature 2, and central object image feature 3. The sum of the distances between each currently unselected second central object image feature and each object image feature in the current first image feature set to be compared (i.e., feature distance) can be calculated, and the second central object image feature with the largest corresponding feature distance (e.g., central object image feature 2) can be selected as the first candidate object image feature.

[0161] Then, the central object image feature 2 is added to the current first image feature set to be compared, that is, the current first image feature set to be compared includes: central object image feature 2, central object image feature 4, central object image feature 5, central object image feature 6, central object image feature 7 and central object image feature 8.

[0162] The same process is repeated until a first preset number of second central object image features are determined, that is, a first preset number of first candidate object image features are determined.

[0163] In one embodiment, see Figure 4 ,exist Figure 3 On the basis of, the above step S10424 may include the following steps:

[0164] S104241: For each out-of-cluster object image feature that is not currently selected, calculate the distance between the out-of-cluster object image feature and the current second set of image features to be compared as the feature distance corresponding to the out-of-cluster object image feature.

[0165] The current second image feature set to be compared includes the centers of each first object image feature cluster.

[0166] S104242: Calculate the label stability of the image feature of the out-of-cluster object and the weighted sum of the corresponding feature distance to obtain the target value.

[0167] S104243: Select the out-of-cluster object image feature with the largest corresponding target value as the second candidate object image feature, add the out-of-cluster object image feature with the largest corresponding target value to the current second set of image features to be compared, and return to step S104241 until a second preset number of out-of-cluster object image features are determined as the second candidate object image features.

[0168] For example, each out-of-cluster object image feature includes: out-of-cluster object image feature 1, out-of-cluster object image feature 2, out-of-cluster object image feature 3 and out-of-cluster object image feature 4; the first central object image feature includes: central object image feature 5, central object image feature 6, central object image feature 7 and central object image feature 8.

[0169] That is, the initial second set of image features to be compared includes: central object image feature 5 , central object image feature 6 , central object image feature 7 , and central object image feature 8 .

[0170] At this point, the currently unselected out-of-cluster object image features include: out-of-cluster object image feature 1, out-of-cluster object image feature 2, out-of-cluster object image feature 3, and out-of-cluster object image feature 4. The sum of the distances between each currently unselected out-of-cluster object image feature and each object image feature in the current second image feature set to be compared (i.e., feature distance) can be calculated, and a weighted sum of the feature distances and the label stability of the out-of-cluster object image feature can be calculated to obtain a target value. The out-of-cluster object image feature with the largest corresponding target value (e.g., out-of-cluster object image feature 4) is selected as the second candidate object image feature.

[0171] Then, the out-of-cluster object image feature 4 is added to the current second image feature set to be compared, that is, the current second image feature set to be compared includes: out-of-cluster object image feature 4, central object image feature 5, central object image feature 6, central object image feature 7 and central object image feature 8.

[0172] Accordingly, the currently unselected out-of-cluster object image features include: out-of-cluster object image feature 1, out-of-cluster object image feature 2, and out-of-cluster object image feature 3. The sum of the distances between each currently unselected out-of-cluster object image feature and each object image feature in the current second image feature set to be compared (i.e., feature distance) can be calculated, and a weighted sum of the feature distance and the label stability of the out-of-cluster object image feature can be calculated to obtain a target value. The out-of-cluster object image feature with the largest corresponding target value (e.g., out-of-cluster object image feature 2) is selected as the second candidate object image feature.

[0173] Then, the out-of-cluster object image feature 2 is added to the current second image feature set to be compared, that is, the current second image feature set to be compared includes: out-of-cluster object image feature 2, out-of-cluster object image feature 4, central object image feature 5, central object image feature 6, central object image feature 7 and central object image feature 8.

[0174] The same process is repeated until a second preset number of out-of-cluster object image features are determined, that is, a second preset number of second candidate object image features are determined.

[0175] In one embodiment, before the above step S10424, the method may further include the following steps:

[0176] Step 1: For each out-of-cluster object image feature, calculate the entropy of the average value of the confidence of the pseudo-label of each object corresponding to the out-of-cluster object image feature in each image contained in the sample image set corresponding to the out-of-cluster object image feature as the first value, and the average value of the entropy of the confidence of the pseudo-label of each object corresponding to the out-of-cluster object image feature as the second value.

[0177] Step 2: Calculate the difference between the first value and the second value as the label stability of the image feature of the out-of-cluster object.

[0178] In the embodiment of the present application, for each object in the first sample image, the confidence level of the pseudo label of the object can be determined based on the first detection model. For example, the confidence level can be the probability that the object is a preset object.

[0179] Accordingly, a sample image set (referred to as a target sample image set) corresponding to each out-of-cluster object image feature can be determined. That is, the sample image set to which the sample image corresponding to each out-of-cluster object image feature belongs belongs. It is understood that the sample image to which the object corresponding to an out-of-cluster object image feature belongs can be either a sample image to be enhanced or an enhanced sample image.

[0180] A target sample image set contains multiple images with identical content. Each of these images has a corresponding pseudo-label for the same object. Therefore, the average confidence level of each pseudo-label for the same object in each image can be calculated, and the entropy of this average value can be calculated as a first value. Furthermore, the entropy of each pseudo-label confidence level for the same object in each image can be calculated, and the average of these entropies can be calculated as a second value. The difference between the first and second values represents the label stability corresponding to the target sample image set, i.e., the label stability of each object image feature for each image in the target sample image set.

[0181] That is to say, each image in a target sample image set shares the same label stability for each object image feature of the same object.

[0182] In one embodiment, the above step S1043 may include the following steps:

[0183] For each determined candidate object image feature, a sample image to be enhanced in the sample image set to which the first sample image corresponding to the candidate object image feature belongs is determined as a sample image to be calibrated.

[0184] In the embodiment of the present application, the determined candidate object image features may be image features of an object in a sample image to be enhanced. Accordingly, the sample image to be enhanced may be determined as a sample image to be calibrated.

[0185] The determined candidate object image features may also be image features of the object in the enhanced sample image. Accordingly, the to-be-enhanced sample image corresponding to the enhanced sample image may be determined as the to-be-calibrated sample image.

[0186] In one embodiment, each first sample image may contain multiple objects. That is, a first sample image may correspond to one candidate object image feature or multiple candidate object image features. Accordingly, if there are a large number of first sample images (referred to as candidate sample images) corresponding to each candidate object image feature, a third preset number of sample images may be selected from each candidate sample image based on the number of corresponding candidate object image features as the sample images to be calibrated.

[0187] For example, a sample image having a larger number of corresponding candidate object image features may be selected from the sample images to be selected as the sample image to be calibrated.

[0188] In one embodiment, before the above step S105, the method may further include the following steps:

[0189] The pseudo label of the object corresponding to the object image feature of the object in each sample image to be calibrated that belongs to the same second object image feature cluster is replaced by the real label marked by the user for the object in the sample image to be calibrated.

[0190] In an embodiment of the present application, after obtaining the true label of the object in the sample image to be calibrated by the user, it is also possible to determine the object image features (which can be referred to as the object image features to be processed) that belong to the same cluster (i.e., the second object image feature cluster) as the object image features of the object in the sample image to be calibrated.

[0191] Because the image features of objects belonging to the same cluster are similar, the pseudo-labels of the objects corresponding to the image features of the object to be processed can be replaced with the true labels of the objects in the sample image to be calibrated, making the replaced pseudo-labels closer to their true labels. Accordingly, based on step S105, the second detection model can be trained according to the replaced pseudo-labels, thereby improving the accuracy of the trained object detection model.

[0192] Based on the above processing, the user only needs to label a portion of the first sample image. Accordingly, the real labels marked by the user can be diffused, thereby reducing the user's labeling cost and improving the accuracy of the trained target detection model.

[0193] Regarding step S105, in one implementation, the second detection model may be a model that is not trained based on sample images. Accordingly, the target detection model obtained by training the second detection model can effectively detect images of the current detection scene.

[0194] In another implementation, the second detection model can also be a model trained based on the second sample image, and the model structure of the first detection model is more complex than that of the second detection model. That is, the second detection model has a different model structure from the first detection model, and for the detection scene corresponding to the second sample image, the detection performance of the first detection model is better than that of the second detection model. Accordingly, the target detection model obtained by training the second detection model can be applied to both the current detection scene and the detection scene corresponding to the second sample image.

[0195] In one embodiment, see Figure 5 , Figure 5 A schematic diagram of the principles of a target detection model training method provided in an embodiment of the present application.

[0196] The input data includes: labeled data (i.e., the second sample images with labeled labels of other detection scenes in the embodiments of the present application), unlabeled data (i.e., the sample images to be enhanced without labeled labels of the current detection scene in the present application) and the basic model (i.e., the second detection model in the present application).

[0197] System processing includes:

[0198] The high-precision model training module 101 uses the labeled data to train a high-precision detection model (ie, the first detection model in the embodiment of the present application) to obtain pseudo labels of objects in each first sample image.

[0199] Active push module 102 uses a high-precision detection model to actively push unlabeled data to obtain data to be calibrated (i.e., sample images to be calibrated in this application) to facilitate manual labeling of the sample images to be calibrated. Specifically, first object image features of objects in each first sample image and second object image features of objects in each second sample image are obtained. Based on the first and second object image features, a sample image to be calibrated is determined from each first sample image.

[0200] Then, the semi-supervised model training module 103 performs semi-supervised training to obtain a semi-supervised model. That is, based on the pseudo labels of the objects in the third sample image, the true labels of the objects in each second sample image, and the true labels of the objects in the sample images to be calibrated, the second detection model to be trained is trained to obtain an object detection model for the current detection scene.

[0201] See also Figure 6 , Figure 6 A schematic diagram of the structure of an active push module provided in an embodiment of the present application.

[0202] The label candidate pool generation unit 201 performs image enhancement processing on the unlabeled data. Specifically, it generates a first sample image from the sample image to be enhanced. It then performs forward prediction based on the high-precision detection model to generate a label candidate pool. Specifically, it detects each first sample image based on the first detection model to obtain pseudo labels for the objects in each first sample image.

[0203] The feature extraction unit 202 takes the first sample image and the second sample image as input, and outputs the image features output by the RoIAlign layer of the high-precision detection model, which are subjected to global average pooling to obtain domain image features, and uses a pre-trained convolutional neural network general model to extract general image features.

[0204] The hierarchical clustering unit 203 is configured to perform hierarchical clustering on the domain image features and the universal image features of the first sample image output by the feature extraction unit 202 to obtain a second object image feature cluster. Furthermore, the hierarchical clustering unit 203 is configured to perform hierarchical clustering on the domain image features and the universal image features of the second sample image to obtain a first object image feature cluster.

[0205] The pushing unit 204 obtains a push list (ie, sample images to be calibrated) based on the first object image feature cluster and the second object image feature cluster.

[0206] In one embodiment, a target detection method is also provided, which can obtain an image to be detected, input the image to be detected into a target detection model of the current detection scene obtained by using the target detection model training method in the above embodiment, and obtain a detection result of the image to be detected.

[0207] The target detection method provided by the embodiment of the present application can determine the sample image to be calibrated in the first sample image, and the sample image to be calibrated does not match other detection scenes. In other words, the first detection model has not learned the image features of the object in the sample image to be calibrated, and the accuracy of the pseudo-label of the object in the sample image to be calibrated is low. Therefore, such sample images can be pushed to the user for labeling to obtain their real labels, which are used to model the second detection model, so that the trained target detection model can learn the image features of the object in the sample image to be calibrated. Similarly, the accuracy of the pseudo-label of the object in the third sample image is high, so its pseudo-label can be directly used to model the second detection model. In this way, it is only necessary to manually label the objects in a part of the first sample image (i.e., the sample image to be calibrated) to obtain the target detection model of the current detection scene, thereby reducing the cost of detection, improving the generation efficiency of the detection model, and accordingly, improving the efficiency of detection. In addition, using the trained first detection model to guide the training of the second detection model can reduce the false alarm rate of the target detection model obtained and improve the detection accuracy.

[0208] In one embodiment, the detection result may include the position of the object in the image to be detected, and / or the probability that the image to be detected contains a preset object.

[0209] In one embodiment, the target detection model is used to detect a preset object. Specifically, by setting different output parameters of the target detection model, the target detection model can output the location of the object in the image to be detected and the probability that the object is the preset object.

[0210] Furthermore, if there are multiple preset objects, a probability that the object contained in the image to be detected is the preset object can be obtained for each preset object. In other words, a single object in the image to be detected can correspond to multiple probabilities. If one of the probabilities is greater than a preset probability threshold, it indicates that the object is the preset object corresponding to that probability, meaning that the image to be detected contains the preset object.

[0211] In addition, the position information of the object in the image to be detected can also be determined. For example, the coordinates of the vertices of the minimum circumscribed rectangular frame of the object in the image to be detected can be determined.

[0212] It is understood that the detection results output by the object detection model are not limited to the data shown in the above embodiment. By setting the output parameters of the object detection model, other different types of detection results can also be obtained. For example, the number of objects contained in the image to be detected, the image frame of the objects contained in the image to be detected, etc. can also be obtained.

[0213] Based on the same inventive concept, the present application also provides a target detection model training device, see Figure 7 , Figure 7 This is a structural diagram of a target detection model training device provided in an embodiment of the present application, which may include:

[0214] A first sample image acquisition module 701 is used to acquire first sample images without labels of the current detection scene;

[0215] The pseudo-label acquisition module 702 is configured to detect each first sample image based on a first detection model and obtain a label of the object in each first sample image as a pseudo-label; wherein the first detection model is trained based on each second sample image with a label of a detection scene other than the current detection scene;

[0216] The object image feature acquisition module 703 is configured to acquire the image features of the object in each first sample image as the first object image feature, and acquire the image features of the object in each second sample image as the second object image feature;

[0217] a sample image to be calibrated determining module 704, configured to determine a sample image to be calibrated from each first sample image based on each first object image feature and each second object image feature, so as to obtain a true label of the object in the sample image to be calibrated, which is marked by the user; wherein the sample image to be calibrated does not match the other detection scenes;

[0218] The model training module 705 is used to perform model training on the second detection model to be trained based on the pseudo labels of objects in the third sample images other than the sample images to be calibrated in each first sample image, the true labels of objects in each second sample image, and the true labels of objects in the sample images to be calibrated, so as to obtain the target detection model of the current detection scene.

[0219] Optionally, the sample image to be calibrated determining module 704 includes:

[0220] a first object image feature cluster determination submodule, configured to cluster each second object image feature to obtain a plurality of object image feature clusters as first object image feature clusters;

[0221] a candidate object image feature determination submodule, configured to determine a candidate object image feature from each first object image feature based on a distance from each first object image feature cluster; wherein the candidate object image feature does not match the second object image feature;

[0222] The submodule for determining the sample image to be calibrated is used to determine the sample image to be calibrated based on the first sample image corresponding to the determined image features of each candidate object.

[0223] Optionally, each first sample image includes a plurality of sample image sets; each sample image set includes an unlabeled sample image to be enhanced of the current detection scene, and an enhanced sample image obtained by performing image enhancement processing on the sample image to be enhanced;

[0224] The candidate object image feature determination submodule includes:

[0225] a second object image feature cluster determining unit, configured to cluster the first object image features to obtain a plurality of object image feature clusters as second object image feature clusters;

[0226] a first processing unit configured to determine a center of each second object image feature cluster as a central object image feature, and to determine object image features other than the second object image feature cluster in each first object image feature as out-of-cluster object image features;

[0227] a first candidate object image feature determining unit configured to determine a first candidate object image feature from each central object image feature based on a distance from each first object image feature cluster; wherein the distance between the first candidate object image feature and each first object image feature cluster is greater than the distance between other object image features in each central object image feature and each first object image feature cluster;

[0228] a second candidate object image feature determining unit configured to determine a second candidate object image feature from each out-of-cluster object image feature based on a distance from each first object image feature cluster and a label stability; wherein the label stability of an out-of-cluster object image feature indicates a degree of consistency between pseudo labels of each object corresponding to the out-of-cluster object image feature in each image included in the sample image set corresponding to the out-of-cluster object image feature;

[0229] The final candidate object image feature determining unit is configured to determine the first candidate object image feature and the second candidate object image feature as the final candidate object image feature.

[0230] Optionally, the first candidate object image feature determination unit is specifically configured to calculate, for each central object image feature that is not currently selected, a distance between the central object image feature and a current first set of image features to be compared, as a feature distance corresponding to the central object image feature; wherein the current first set of image features to be compared includes the centers of each first object image feature cluster;

[0231] Select the central object image feature with the largest corresponding feature distance as the first candidate object image feature, add the central object image feature with the largest corresponding feature distance to the current first image feature set to be compared, and return to execute the step of calculating the distance between the central object image feature and the current first image feature set to be compared for each central object image feature that is not currently selected as the feature distance corresponding to the central object image feature, until a first preset number of central object image features are determined as the first candidate object image feature.

[0232] Optionally, the device further includes:

[0233] a label stability determination module configured to calculate, for each out-of-cluster object image feature, an entropy of an average value of confidences of pseudo labels of each object corresponding to the out-of-cluster object image feature in each image included in the sample image set corresponding to the out-of-cluster object image feature, as a first value, and an average value of entropies of confidences of pseudo labels of each object corresponding to the out-of-cluster object image feature, as a second value, before determining the second candidate object image feature from each out-of-cluster object image feature based on the distance to each first object image feature cluster and the label stability;

[0234] The difference between the first value and the second value is calculated as the label stability of the image feature of the out-of-cluster object.

[0235] Optionally, the second candidate object image feature determination unit is specifically configured to calculate, for each out-of-cluster object image feature that is not currently selected, a distance between the out-of-cluster object image feature and a current second set of image features to be compared, as the feature distance corresponding to the out-of-cluster object image feature; wherein the current second set of image features to be compared includes the centers of each first object image feature cluster;

[0236] Calculate the label stability of the image features of the out-of-cluster objects and the weighted sum of the corresponding feature distances to obtain the target value;

[0237] The out-of-cluster object image feature with the largest corresponding target value is selected as the second candidate object image feature, the out-of-cluster object image feature with the largest corresponding target value is added to the current second set of image features to be compared, and the step of calculating, for each out-of-cluster object image feature not currently selected, the distance between the out-of-cluster object image feature and the current second set of image features to be compared as the feature distance corresponding to the out-of-cluster object image feature is returned to execution, until a second preset number of out-of-cluster object image features are determined as the second candidate object image features.

[0238] Optionally, the submodule for determining the sample image to be calibrated is specifically configured to determine, for each determined candidate object image feature, a sample image to be enhanced in the sample image set to which the first sample image corresponding to the candidate object image feature belongs, as the sample image to be calibrated.

[0239] Optionally, the device further includes:

[0240] A replacement module is used to replace the pseudo labels of objects corresponding to object image features belonging to the same second object image feature cluster as the object image features of the objects in each sample image to be calibrated with the real labels marked by the user for the objects in the sample image to be calibrated before training the second detection model to be trained based on the pseudo labels of objects in the third sample image other than the sample image to be calibrated in each first sample image, the real labels of objects in each second sample image, and the real labels of objects in the sample image to be calibrated to obtain the target detection model of the current detection scene.

[0241] Optionally, the object image feature acquisition module 703 is specifically configured to input each first sample image into the first detection model and the universal detection model, respectively, to obtain first object image features of the object in the first sample image output by the feature extraction layers of the first detection model and the universal detection model; wherein the universal detection model is trained based on sample images of multiple different detection scenarios;

[0242] For each second sample image, the second sample image is input into the first detection model and the universal detection model respectively to obtain the second object image features of the object in the second sample image output by the feature extraction layers of the first detection model and the universal detection model.

[0243] Based on the same inventive concept, an embodiment of the present application further provides a target detection device, which may include:

[0244] An image acquisition module to be detected, used to acquire the image to be detected;

[0245] A detection module is used to input the image to be detected into the target detection model of the current detection scene to obtain the detection result of the image to be detected; wherein, the target detection model is obtained using the target detection model training method described in any one of the first aspects above.

[0246] Optionally, the detection result includes the position of the object in the image to be detected, and / or the probability that the image to be detected contains a preset object.

[0247] The present application also provides an electronic device, such as Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0248] Memory 803, used for storing computer programs;

[0249] The processor 801 is configured to implement the target detection model training method or the target detection method in the above embodiment when executing the program stored in the memory 803 .

[0250] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0251] The communication interface is used for communication between the above electronic device and other devices.

[0252] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0253] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0254] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the target detection model training method or target detection method provided in the embodiment of the present application.

[0255] An embodiment of the present application also provides another computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the target detection model training method or target detection method provided in an embodiment of the present application.

[0256] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0257] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0258] Each embodiment in this specification is described in a related manner. Similar portions between embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, since the apparatus, electronic device, computer-readable storage medium, and computer program product embodiments are generally similar to the method embodiments, their descriptions are relatively simplified. For related portions, reference can be made to the descriptions of the method embodiments.

[0259] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A target detection model training method, characterized in that: The method comprises: Obtaining each first sample image of the current detection scene without a label; Detecting each first sample image based on a first detection model to obtain a label of the object in each first sample image as a pseudo label; wherein the first detection model is trained based on each second sample image with a label of other detection scenes other than the current detection scene; Acquire an image feature of an object in each first sample image as a first object image feature, and acquire an image feature of an object in each second sample image as a second object image feature; Based on each first object image feature and each second object image feature, determining a sample image to be calibrated from each first sample image to obtain a true label of an object in the sample image to be calibrated by a user; wherein the sample image to be calibrated does not match the other detection scenes; Based on the pseudo labels of objects in the third sample image other than the sample image to be calibrated in each first sample image, the true labels of objects in each second sample image, and the true labels of objects in the sample image to be calibrated, the second detection model to be trained is trained to obtain a target detection model of the current detection scene; wherein, the target detection model of the current detection scene is used to detect the image of the current detection scene.

2. The method according to claim 1, characterized in that The step of determining the sample images to be calibrated from the first sample images based on the first object image features and the second object image features includes: Clustering each second object image feature to obtain a plurality of object image feature clusters as first object image feature clusters; determining a candidate object image feature from each first object image feature based on a distance from each first object image feature cluster; wherein the candidate object image feature does not match the second object image feature; Based on the first sample images corresponding to the determined image features of each candidate object, a sample image to be calibrated is determined.

3. The method according to claim 2, characterized in that Each first sample image includes a plurality of sample image sets; each sample image set includes an unlabeled sample image to be enhanced of the current detection scene, and an enhanced sample image obtained by performing image enhancement processing on the sample image to be enhanced; The determining of candidate object image features from each first object image feature based on the distance between the candidate object image features and each first object image feature cluster includes: Clustering each first object image feature to obtain a plurality of object image feature clusters as second object image feature clusters; determining a center of each second object image feature cluster as a central object image feature, and determining object image features other than the second object image feature cluster in each first object image feature as out-of-cluster object image features; determining a first candidate object image feature from each central object image feature based on a distance from each first object image feature cluster; wherein the distance between the first candidate object image feature and each first object image feature cluster is greater than the distance between other object image features in each central object image feature and each first object image feature cluster; Determining a second candidate object image feature from each out-of-cluster object image feature based on the distance from each first object image feature cluster and the label stability; wherein the label stability of an out-of-cluster object image feature represents: the degree of consistency of pseudo labels of each object corresponding to the out-of-cluster object image feature in each image included in the sample image set corresponding to the out-of-cluster object image feature; The first candidate object image feature and the second candidate object image feature are determined as final candidate object image features.

4. The method according to claim 3, characterized in that The determining of the first candidate object image feature from each central object image feature based on the distance from each first object image feature cluster includes: For each central object image feature that is not currently selected, calculating the distance between the central object image feature and the current first set of image features to be compared as the feature distance corresponding to the central object image feature; wherein the current first set of image features to be compared includes the centers of each first object image feature cluster; Select the central object image feature with the largest corresponding feature distance as the first candidate object image feature, add the central object image feature with the largest corresponding feature distance to the current first image feature set to be compared, and return to execute the step of calculating the distance between the central object image feature and the current first image feature set to be compared for each central object image feature that is not currently selected as the feature distance corresponding to the central object image feature, until a first preset number of central object image features are determined as the first candidate object image feature.

5. The method according to claim 3, characterized in that Before determining the second candidate object image features from each out-of-cluster object image feature based on the distance from each first object image feature cluster and the label stability, the method further includes: For each out-of-cluster object image feature, calculating the entropy of the average value of the confidence of the pseudo-label of each object corresponding to the out-of-cluster object image feature in each image included in the sample image set corresponding to the out-of-cluster object image feature as a first value, and the average value of the entropy of the confidence of the pseudo-label of each object corresponding to the out-of-cluster object image feature as a second value; The difference between the first value and the second value is calculated as the label stability of the image feature of the out-of-cluster object.

6. The method according to claim 3, characterized in that The determining of the second candidate object image features from each out-of-cluster object image feature based on the distance from each first object image feature cluster and the label stability includes: For each currently unselected out-of-cluster object image feature, calculating the distance between the out-of-cluster object image feature and the current second set of image features to be compared as the feature distance corresponding to the out-of-cluster object image feature; wherein the current second set of image features to be compared includes the centers of each first object image feature cluster; Calculate the label stability of the image features of the out-of-cluster objects and the weighted sum of the corresponding feature distances to obtain the target value; The out-of-cluster object image feature with the largest corresponding target value is selected as the second candidate object image feature, the out-of-cluster object image feature with the largest corresponding target value is added to the current second set of image features to be compared, and the step of calculating, for each out-of-cluster object image feature not currently selected, the distance between the out-of-cluster object image feature and the current second set of image features to be compared as the feature distance corresponding to the out-of-cluster object image feature is returned to execution, until a second preset number of out-of-cluster object image features are determined as the second candidate object image features.

7. The method according to claim 3, characterized in that The step of determining the sample image to be calibrated based on the first sample image corresponding to each candidate object image feature determined includes: For each determined candidate object image feature, a sample image to be enhanced in the sample image set to which the first sample image corresponding to the candidate object image feature belongs is determined as a sample image to be calibrated.

8. The method according to claim 3, characterized in that Before performing model training on the second detection model to be trained based on the pseudo labels of objects in third sample images other than the sample images to be calibrated in each of the first sample images, the true labels of objects in each of the second sample images, and the true labels of objects in the sample images to be calibrated to obtain the object detection model of the current detection scene, the method further includes: The pseudo label of the object corresponding to the object image feature of the object in each sample image to be calibrated that belongs to the same second object image feature cluster is replaced by the real label marked by the user for the object in the sample image to be calibrated.

9. The method according to claim 1, characterized in that The acquiring of the image features of the objects in each first sample image as the first object image features, and the acquiring of the image features of the objects in each second sample image as the second object image features, include: For each first sample image, input the first sample image into the first detection model and the universal detection model, respectively, to obtain first object image features of the object in the first sample image output by the feature extraction layers of the first detection model and the universal detection model; wherein the universal detection model is trained based on sample images of multiple different detection scenarios; For each second sample image, the second sample image is input into the first detection model and the universal detection model respectively to obtain the second object image features of the object in the second sample image output by the feature extraction layers of the first detection model and the universal detection model.

10. A target detection method, characterized in that: The method comprises: Obtain the image to be detected; The image to be detected is input into the target detection model of the current detection scene to obtain the detection result of the image to be detected; wherein, the target detection model is obtained using the target detection model training method described in any one of claims 1 to 9.

11. The method according to claim 10, characterized in that The detection result includes the position of the object in the image to be detected and / or the probability that the image to be detected contains a preset object.

12. A target detection model training device, characterized in that: The device comprises: A first sample image acquisition module is used to acquire first sample images without labels of the current detection scene; a pseudo-label acquisition module, configured to detect each first sample image based on a first detection model and obtain a label of the object in each first sample image as a pseudo-label; wherein the first detection model is obtained by training each second sample image with a label of a detection scene other than the current detection scene; an object image feature acquisition module, configured to acquire image features of objects in each first sample image as first object image features, and to acquire image features of objects in each second sample image as second object image features; a module for determining sample images to be calibrated, configured to determine a sample image to be calibrated from each first sample image based on each first object image feature and each second object image feature, so as to obtain a true label of an object in the sample image to be calibrated, the sample image to be calibrated not matching the other detection scenes; A model training module is used to perform model training on the second detection model to be trained based on the pseudo labels of objects in the third sample images other than the sample images to be calibrated in each first sample image, the true labels of objects in each second sample image, and the true labels of objects in the sample images to be calibrated, so as to obtain a target detection model for the current detection scene; wherein the target detection model for the current detection scene is used to detect images of the current detection scene.

13. A target detection device, characterized in that: The device comprises: An image acquisition module to be detected, used to acquire the image to be detected; A detection module is used to input the image to be detected into a target detection model of the current detection scene to obtain a detection result of the image to be detected; wherein, the target detection model is obtained using the target detection model training method described in any one of claims 1 to 9.

14. The device according to claim 13, characterized in that The detection result includes the position of the object in the image to be detected and / or the probability that the image to be detected contains a preset object.

15. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; The processor is configured to implement the method steps described in any one of claims 1-9, or 10-11 when executing a program stored in a memory.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1-9, or 10-11 are implemented.

Images