Target detection method, device, electronic device and computer-readable storage medium
By splitting the data set and using unlabeled categories and backgrounds as combinations for loss calculations and adjusting the object detection model parameters, the problems of high labeling cost and low efficiency in the existing technology are solved, and fast training and efficient object detection are achieved.
Patent Information
- Application Number
- CN202210675998.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-15
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-06-15
AI Technical Summary
In the prior art, the training of the object detection model requires manual supplementary labeling of different data sets, resulting in high labeling cost and low efficiency, and it cannot be quickly applied to the object detection scenario.
By splitting the categories of the annotated target into a single-category dataset, unlabeled categories and backgrounds are combined for each sample data, the predictive loss is calculated and the target detection model parameters are adjusted to avoid supplementary annotation of different datasets.
Save human resources, reduce labeling costs, quickly train to obtain target detection models, and improve target detection efficiency.
Smart Images

Figure CN115049960B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer application technology, and in particular to a target detection method, device, electronic equipment and computer-readable storage medium. Background Art
[0002] With the rapid development of computer technology, target detection technology has gradually developed and has been applied in many fields, such as video production and vehicle recognition.
[0003] Take the conversion of massive horizontal videos into vertical videos in the field of video production as an example. If the horizontal videos are edited manually to be converted into vertical videos, the efficiency is low and the effect is uncontrollable. Therefore, the horizontal-to-vertical screen conversion technology of video automation is often used for secondary video production. In the process of converting the horizontal screen to the vertical screen of the video, the main thing is to determine the target subject and find the area of interest to the human eye. In its algorithm flow, conventional target detection, face detection, and saliency detection algorithms are used. That is, the image is sequentially passed through three independent models: the conventional target detection model, the face detection model, and the saliency detection model. Among them, the conventional target detection model and the face detection model are trained using different data sets, and the conventional targets and faces are actually targets. Therefore, the conventional target detection model and the face detection model are both used for target detection. That is, here it is equivalent to deploying two target detection models separately, which requires two computing resources. In order to save resources, it is possible to consider fusing the data set and training a target detection model, which can realize the functions of both the conventional target detection model and the face detection model.
[0004] At present, in order to fuse two datasets and train a target detection model, it is necessary to fully annotate all data for each dataset, that is, to supplement the dataset used by the conventional target detection model with face targets, and to supplement the dataset used by the face detection model with conventional targets.
[0005] This method requires manual supplementary annotation of the data set, which consumes a lot of human resources and has high annotation costs. It is impossible to quickly train the target detection model, and it is also impossible to quickly apply the target detection model to the target detection scenario, resulting in low target detection efficiency. Summary of the invention
[0006] The purpose of the embodiments of the present invention is to provide a target detection method, device, electronic device and computer-readable storage medium to save labeling costs, quickly train to obtain a target detection model, and improve target detection efficiency. The specific technical solution is as follows:
[0007] In a first aspect, an embodiment of the present invention provides a target detection method, including:
[0008] Obtain a first data set and a second data set, wherein the first data set has X categories of labeled objects, and the second data set has Y categories of labeled objects, where X and Y are positive integers;
[0009] The first data set and the second data set are split according to the categories of the labeled targets to obtain N single-category data sets, where each sample data in each single-category data set is labeled with only one category of targets, N≤X+Y;
[0010] Based on the sample data in N single-category data sets, a target detection model is trained to perform target detection using the trained target detection model; wherein, during the training process, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data are taken as a combination, the prediction loss is determined, and the parameters of the target detection model are adjusted.
[0011] In a specific implementation of the present invention, the first data set and the second data set are split according to the category of the labeled object to obtain N single-category data sets, including:
[0012] For each category of the labeled object, traverse the first data set and the second data set to determine sample data containing the labeled object of the current category;
[0013] The sample data containing the labeled targets of the current category are synthesized into a single-category dataset.
[0014] In a specific implementation of the present invention, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data are taken as a combination, the prediction loss is determined, and the parameters of the target detection model are adjusted, including:
[0015] For each sample data input to the target detection model, obtain classification scores and coordinate frame regression values of multiple categories predicted by the target detection model based on the input current sample data;
[0016] Taking the target and background of the unlabeled category of the current sample data as a combination, and calculating the cross entropy loss and the coordinate frame regression loss based on the classification score and the coordinate frame regression value;
[0017] The parameters of the object detection model are adjusted according to the cross entropy loss and the coordinate box regression loss.
[0018] In a specific implementation of the present invention, the calculating the cross entropy loss and the coordinate frame regression loss based on the classification score and the coordinate frame regression value includes:
[0019] Calculating a cross entropy loss based on the classification scores using a group normalized exponential function;
[0020] The coordinate box loss is calculated based on the coordinate box regression value using the generalized intersection-over-union ratio.
[0021] In a specific embodiment of the present invention, after training the target detection model based on the sample data in the N single-category data sets, the method further includes:
[0022] Obtaining a newly added category data set, wherein the category of the labeled target included in the sample data in the newly added category data set is the newly added category;
[0023] The target detection model is trained based on the sample data in the N single-category data sets and the newly added category data set, so as to detect the targets of the newly added category using the trained target detection model.
[0024] In a specific embodiment of the present invention, it also includes:
[0025] Updating sample data in a first single-category data set, where the first single-category data set is any one of the N single-category data sets;
[0026] The object detection model is trained based on the sample data in the updated N single-category data sets.
[0027] In a specific embodiment of the present invention, the output of the target detection model includes a classification branch and a coordinate regression branch.
[0028] In a specific embodiment of the present invention, the target detection using the trained target detection model includes:
[0029] Inputting the data to be tested into the trained target detection model;
[0030] Obtain classification scores of multiple categories through the classification branches;
[0031] Based on the classification score, determining the category of the target contained in the test data;
[0032] The coordinate position of the target contained in the data to be measured is obtained through the coordinate regression branch.
[0033] In a second aspect, an embodiment of the present invention provides a target detection device, including:
[0034] A first acquisition module is used to obtain a first data set and a second data set, wherein the first data set has X categories of labeled objects, and the second data set has Y categories of labeled objects, where X and Y are positive integers;
[0035] A second acquisition module is used to split the first data set and the second data set according to the category of the labeled target to obtain N single-category data sets, where each sample data in each single-category data set is labeled with only one category of targets, N≤X+Y;
[0036] A detection module is used to train a target detection model based on sample data in N single-category data sets, so as to use the trained target detection model to perform target detection; wherein, during the training process, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data are taken as a combination, the prediction loss is determined, and the parameters of the target detection model are adjusted.
[0037] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0038] Memory, used to store computer programs;
[0039] The processor is used to implement the steps of the target detection method described in the first aspect when executing the program stored in the memory.
[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the target detection method as described in the first aspect.
[0041] In a fifth aspect, an embodiment of the present invention provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium and are suitable for being read and executed by a processor, so that an electronic device having the processor performs the target detection method as described in the first aspect.
[0042] By applying the technical solution provided in the embodiment of the present invention, different data sets are first split according to the categories of the labeled targets to obtain multiple single-category data sets, and each sample data in each single-category data set is labeled with only one category of targets. Then, based on the sample data in the multiple single-category data sets, the target detection model is trained. During the training process, for each sample data input into the target detection model, the targets and background of the categories that are not labeled in the current sample data are taken as a combination to determine the prediction loss, and the parameters of the target detection model are adjusted. There is no need to perform additional labeling on different data sets, which can save a lot of human resources and labeling costs. The target detection model can be quickly trained so that the trained target detection model can be quickly applied to the target detection scenario, thereby improving the target detection efficiency.
[0043] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.
[0045] Figure 1 is a flowchart of an implementation method of a target detection method in an embodiment of the present invention;
[0046] Figure 2 Schematic diagram of a target detection model in an embodiment of the present invention;
[0047] Figure 3 is a schematic structural diagram of a target detection device in an embodiment of the present invention;
[0048] Figure 4 The figure is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION
[0049] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0050] The core of the present invention is to provide a target detection method, which can be applied to various scenarios such as video production, vehicle identification, pedestrian tracking, etc. The first data set and the second data set can be split according to the category of the labeled target to obtain multiple single-category data sets, each sample data in each single-category data set only labels a target of one category, and then based on the sample data in the multiple single-category data sets, the target detection model is trained. During the training process, for each sample data input to the target detection model, the target and background of the category not labeled in the current sample data are taken as a combination to determine the prediction loss, and the parameters of the target detection model are adjusted. There is no need to supplement the annotation of different data sets, which can save more human resources and annotation costs, and can quickly train to obtain the target detection model, so that the trained target detection model can be quickly applied to the target detection scene, thereby improving the target detection efficiency.
[0051] See also Figure 1 FIG. 1 is a flowchart of an implementation of a target detection method provided by an embodiment of the present invention. The method may include the following steps:
[0052] S110: Obtain a first data set and a second data set.
[0053] There are X categories of labeled targets in the first data set, and there are Y categories of labeled targets in the second data set, where X and Y are positive integers.
[0054] In an embodiment of the present invention, the first data set and the second data set may be existing data sets with object annotation, and the categories of the annotated objects in the first data set may be X, and the categories of the annotated objects in the second data set may be Y. For example, the first data set may be the MSCOCO (MicroSoft Common Objects in COntext) data set provided by the Microsoft team, which contains annotated objects such as human bodies, cats, cups, and bicycles, and has 80 categories of annotated objects. The second data set may be the WIDER FACE data set provided by the Chinese University of Hong Kong team, which contains annotated objects of faces, and has 1 category of annotated objects.
[0055] S120: Split the first data set and the second data set according to the categories of the labeled objects to obtain N single-category data sets.
[0056] Each sample data in each single-category dataset is labeled with only one category of targets, N≤X+Y.
[0057] There are X categories of labeled targets in the obtained first data set, and there are Y categories of labeled targets in the obtained second data set. The first data set and the second data set can be split according to the categories of labeled targets, and sample data containing labeled targets of each category in the first data set and the second data set are determined to obtain N single-category data sets. Each sample data in each single-category data set is labeled with only one category of targets, that is, for each single-category data set, the sample data in the single-category data set is labeled with only the targets of that category.
[0058] For example, the MSCOCO dataset can be split into 80 single-category datasets, which can be human datasets, cat datasets, cup datasets, bicycle datasets, etc. Each sample data in the cat dataset is only labeled with targets of the cat category, and each sample data in the bicycle dataset is only labeled with targets of the bicycle category. Because the WIDER FACE dataset only contains one category of labeled targets, namely, the face, there is no need to split the WIDERFACE dataset, or the face dataset obtained after splitting the dataset is the same as the dataset.
[0059] Because the categories of the labeled objects in the first data set may be the same as or different from the categories of the labeled objects in the second data set, the number of categories of the labeled objects in the first data set and the second data set as a whole is less than or equal to the sum of the number of categories of the labeled objects in the first data set and the second data set, that is, N≤X+Y.
[0060] For each of the N single-category datasets, each sample data in the single-category dataset contains the same category of the labeled target. For example, each sample data in the human dataset contains the labeled target of human, and each sample data in the cat dataset contains the labeled target of cat. A category identifier can be set for each single-category dataset, and each sample data in each single-category dataset can be associated with a corresponding category identifier.
[0061] It should be noted that, for any sample data in the first data set and the second data set, if the labeled target contained in the sample data has multiple categories, the first data set and the second data set are split according to the categories of the labeled targets, and the sample data will belong to multiple single-category data sets. For example, if a sample data contains labeled targets of two categories, human and cat, the sample data is included in both the human data set and the cat data set in the multiple single-category data sets obtained, but in the human data set, the sample data is only labeled with human targets, and in the cat data set, the sample data is only labeled with cat targets, that is, the labeled targets of the same sample data in different single-category data sets are different.
[0062] S130: Based on the sample data in N single-category data sets, the target detection model is trained to perform target detection using the trained target detection model. During the training process, for each sample data input into the target detection model, the target and background of the category that is not labeled in the current sample data are taken as a combination to determine the prediction loss and adjust the parameters of the target detection model.
[0063] In an embodiment of the present invention, after the first data set and the second data set are split according to the categories of the labeled targets to obtain N single-category data sets, the target detection model can be trained based on the sample data in the N single-category data sets.
[0064] Specifically, a target detection model can be pre-built, the parameters of each layer thereof can be initialized, and then each sample data in the N single-category data sets can be input into the target detection model, and each sample data can be used to train the target detection model. It is also possible to select part of the sample data in each single-category data set to train the target detection model. During the training process, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data can be taken as a combination to determine the prediction loss, and the parameters of the target detection model can be adjusted based on the prediction loss. The current sample data is the sample data targeted by the current operation.
[0065] The number of iterations of the target detection model can be set in advance. The prediction loss is obtained in each iteration. By minimizing the loss target, the model parameters are adjusted by reverse gradient update. During the continuous iteration process, the loss function value will reach a convergence state. When the number of iterations is completed, the training of the target detection model is completed. The trained target detection model can be used for target detection. For example, the image to be detected is input into the trained target detection model, and the target contained in the image to be detected is determined according to the prediction result of the target detection model.
[0066] By applying the method provided in the embodiment of the present invention, different data sets are first split according to the categories of the labeled targets to obtain multiple single-category data sets, and each sample data in each single-category data set is labeled with only one category of targets. Then, based on the sample data in the multiple single-category data sets, the target detection model is trained. During the training process, for each sample data input into the target detection model, the targets and background of the categories that are not labeled in the current sample data are taken as a combination to determine the prediction loss, and the parameters of the target detection model are adjusted. There is no need to perform additional labeling on different data sets, which can save a lot of human resources and labeling costs. The target detection model can be quickly trained so that the trained target detection model can be quickly applied to the target detection scenario, thereby improving the target detection efficiency.
[0067] In one embodiment of the present invention, splitting the first data set and the second data set according to the category of the labeled object to obtain N single-category data sets may include the following steps:
[0068] Step 1: for each category of the labeled target, traverse the first data set and the second data set to determine sample data containing the labeled target of the current category;
[0069] Step 2: Synthesize the sample data containing the labeled targets of the current category into a single-category dataset.
[0070] For the convenience of description, the above two steps are combined for explanation.
[0071] After obtaining the first data set and the second data set, it is possible to know which categories of labeled targets are in the first data set and which categories of labeled targets are in the second data set. For each category of the labeled targets, the first data set and the second data set can be traversed to determine the sample data containing the labeled targets of the current category. The current category is the category targeted by the current operation.
[0072] In a specific embodiment of the present invention, for each category of the labeled target in the first data set, the first data set can be traversed to determine the sample data of the labeled target of the current category in the first data set. At the same time, for each category of the labeled target in the second data set, the second data set can be traversed to determine the sample data of the labeled target of the current category in the second data set. In this way, sample data corresponding to each category of the labeled target in the first data set and each category of the labeled target in the second data set can be obtained. Considering that the category of the labeled target in the first data set may be partially the same as the category of the labeled target in the second data set, the sample data corresponding to the same category can be grouped together to obtain sample data of each category of the labeled target in the first data set and the second data set.
[0073] In another specific embodiment of the present invention, for each category of the labeled objects in the first data set and the second data set, the first data set and the second data set can be traversed simultaneously to determine the sample data containing the labeled objects of the current category. In this way, the sample data corresponding to each category of the labeled objects in the first data set and the second data set can also be obtained.
[0074] For each category of the labeled target, after determining the sample data containing the labeled target of the current category, the sample data containing the labeled target of the current category can be further synthesized into a single-category data set. Specifically, a category identifier can be set for each single-category data set, and each sample data in each single-category data set can be associated with a corresponding category identifier.
[0075] For each category of the labeled objects in the first data set and the second data set, a single-category data set corresponding to the category can be obtained. Based on the sample data in the obtained single-category data set, the target detection model can be trained, so that the trained target detection model can be used for target detection. There is no need to supplement the labeling of the sample data, which can save the labeling cost.
[0076] In one embodiment of the present invention, for each sample data input into the target detection model, the unlabeled target and background of the current sample data are taken as a combination, the prediction loss is determined, and the parameters of the target detection model are adjusted, which may include the following steps:
[0077] The first step: for each sample data input to the target detection model, obtain the classification scores and coordinate frame regression values of multiple categories predicted by the target detection model based on the input current sample data;
[0078] The second step: treat the target and background of the unlabeled category of the current sample data as a combination, and calculate the cross entropy loss and coordinate frame regression loss based on the classification score and coordinate frame regression value;
[0079] The third step: adjust the parameters of the object detection model according to the cross entropy loss and coordinate box regression loss.
[0080] For the convenience of description, the above three steps are combined for explanation.
[0081] The first data set and the second data set are split according to the categories of the targets annotated in the first data set and the second data set. After obtaining multiple single-category data sets, each sample data in each single-category data set can be input into the target detection model in turn to train the target detection model.
[0082] The target detection model can predict the classification scores of the input sample data, including the target belonging to each category. In the process of training the target detection model, for each sample data input to the target detection model, the classification scores and coordinate frame regression values of multiple categories predicted by the target detection model based on the current sample data input can be obtained. For example, based on the sample data in 81 single-category data sets, the target detection model is trained. Based on each input sample data, the target detection model can predict the classification scores and coordinate frame regression values of the corresponding sample data including the target belonging to 81 categories. The current sample data is the sample data targeted by the current operation. The coordinate frame regression value refers to the target coordinates obtained using the regression method.
[0083] The classification score can be a normalized score, and the sum of the classification scores of all categories is 1. For example, the classification scores of multiple categories predicted by the object detection model based on the current sample data are: the classification score of the human category is 0.11, the classification score of the cat category is 0.62, the classification score of the cup category is 0.02, the classification score of the bicycle category is 0.25, and the classification scores of other categories are all 0. For any category, the higher the classification score of the category, the greater the possibility that the sample data belongs to the category.
[0084] The targets and background of the unlabeled categories of the current sample data can be taken as a combination. Based on the classification score and the coordinate frame regression value, the cross entropy loss and the coordinate frame regression loss can be calculated. The prediction loss can be the sum of the cross entropy loss and the coordinate frame regression loss. Specifically, the group normalization exponential function can be used to calculate the cross entropy loss based on the classification score, and the generalized intersection over union (GIU) can be used to calculate the coordinate frame loss based on the coordinate frame regression value. The group normalization exponential function is the GroupSoftmax loss function. The target detection model outputs the classification scores of multiple categories. It only calculates the loss of the category corresponding to the current sample data and the loss of the target and background of other unlabeled categories that form a group.
[0085] In one embodiment of the present invention, after training the target detection model based on the sample data in the N single-category data sets, the method may further include the following steps:
[0086] Step 1: Obtain a new category data set, where the category of the labeled target contained in the sample data in the new category data set is the new category;
[0087] Step 2: Based on the sample data in the N single-category data sets and the newly added category data sets, the target detection model is retrained to detect the targets of the newly added category using the retrained target detection model.
[0088] For the convenience of description, the above two steps are combined for explanation.
[0089] In an embodiment of the present invention, the first data set and the second data set are split according to the categories of the labeled targets to obtain N single-category data sets. After the target detection model is trained based on the sample data in the N single-category data sets, if there is a need to detect targets of a new category, a new category data set can be obtained, and the category of the labeled targets contained in the sample data in the new category data set is a new category, that is, a new category relative to the category corresponding to the N single-category data sets.
[0090] Based on the sample data in N single-category data sets and newly added category data sets, the target detection model can be trained, which can be retraining or incremental training. During the training process, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data can be taken as a combination to determine the prediction loss and adjust the parameters of the target detection model. The target detection model trained in this way can increase the detection of targets of the newly added category, and the trained target detection model can be used to detect targets of N categories and newly added categories. The applicability of the target detection model is enhanced.
[0091] In one embodiment of the present invention, the method may further include the following steps:
[0092] The first step: update the sample data in the first single-category data set, where the first single-category data set is any one of the N single-category data sets;
[0093] The second step is to train the target detection model based on the sample data in the updated N single-category datasets.
[0094] For the convenience of description, the above two steps are combined for explanation.
[0095] In an embodiment of the present invention, the first data set and the second data set are split according to the categories of the labeled targets in the first data set and the second data set to obtain N single-category data sets. After the target detection model is trained based on the sample data in the N single-category data sets, the sample data in any one of the single-category data sets can be added according to the actual situation to increase the number of samples and improve the accuracy of the target detection model.
[0096] The sample data in the first single-category dataset can be updated, and the first single-category dataset is any one of the N single-category datasets. Based on the sample data in the updated N single-category datasets, the target detection model can be trained. It can be retraining or incremental training. During the training process, for each sample data input to the target detection model, the target and background of the category not labeled in the current sample data can be taken as a combination to determine the prediction loss and adjust the parameters of the target detection model. In this way, the trained target detection model can be more accurate.
[0097] In one embodiment of the present invention, the output of the target detection model may include a classification branch and a coordinate regression branch.
[0098] In the embodiment of the present invention, Figure 2 As shown in the figure, the target detection model can be created based on RetinaNet. The target detection model can use ResNet50 as the backbone network for feature extraction to obtain convolution features C3, C4, and C5, and then use the Feature Pyramid Network (FPN) structure to obtain pyramid features P3, P4, and P5. This hierarchical feature facilitates the detection of targets of different sizes. The output of the target detection model can include a classification branch and a coordinate regression branch, which predict the classification scores and coordinate positions of multiple categories respectively.
[0099] Taking the MSCOCO dataset as the first data set and the WIDER FACE dataset as the second data set as an example, the MSCOCO dataset contains 118,287 images, and the WIDER FACE dataset contains 32,203 images, including 158,989 face annotation targets. After splitting the first data set and the second data set according to the category of the annotation target and obtaining N single-category data sets, the target detection model can be trained based on the sample data in the N single-category data sets. The training framework can use Pytorch1.6, and 4 NVIDIA Tesla V100 GPUs can be used for training. The batch size can be set to 32, 120 rounds of training, the initial learning rate is 0.1, and the cosine descent method is used to adjust. The SGD (stochastic gradient descent) optimizer is used, and the sample data input size is 320x320. After training, the target detection model has the ability to detect 81 categories of targets. When testing images containing human bodies, both face frames and human frames can be detected.
[0100] When using the trained target detection model for target detection, the data to be tested can be input into the trained target detection model, and classification scores of multiple categories can be obtained through the classification branch. Based on the classification scores, the category of the target contained in the data to be tested can be determined. At the same time, the coordinate position of the target contained in the data to be tested can be obtained through the coordinate regression branch.
[0101] The embodiment of the present invention splits multiple different data sets previously used by the target detection model without increasing the labeling cost to obtain multiple single-category data sets. If a new category is needed after the splitting, the new category data set can be directly added. If a certain category of target is to be optimized, it is only necessary to specifically label the sample data of the category, and it is not necessary to label all categories. If there are overlapping categories, it is also convenient to merge them. It is realized that a target detection model that can detect N types of targets is trained using N single-category data sets, and the target detection model is used for target detection, saving computing resources.
[0102] Corresponding to the above method embodiment, an embodiment of the present invention further provides a target detection device. The target detection device described below and the target detection method described above can be referred to each other.
[0103] See also Figure 3 As shown, the target detection device 300 may include the following modules:
[0104] A first obtaining module 310 is used to obtain a first data set and a second data set, wherein the first data set has X categories of labeled objects, and the second data set has Y categories of labeled objects, where X and Y are positive integers;
[0105] The second acquisition module 320 is used to split the first data set and the second data set according to the categories of the labeled targets to obtain N single-category data sets, where each sample data in each single-category data set is labeled with only one category of targets, N≤X+Y;
[0106] The detection module 330 is used to train the target detection model based on the sample data in N single-category data sets, so as to use the trained target detection model to perform target detection; wherein, during the training process, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data are taken as a combination, the prediction loss is determined, and the parameters of the target detection model are adjusted.
[0107] By using the device provided by the embodiment of the present invention, different data sets are first split according to the categories of the labeled targets to obtain multiple single-category data sets, in which each sample data in each single-category data set is labeled with only one category of targets, and then the target detection model is trained based on the sample data in the multiple single-category data sets. During the training process, for each sample data input into the target detection model, the targets and background of the categories that are not labeled in the current sample data are taken as a combination to determine the prediction loss, and the parameters of the target detection model are adjusted. There is no need to perform additional labeling on different data sets, which can save a lot of human resources and labeling costs, and the target detection model can be quickly trained so that the trained target detection model can be quickly applied to the target detection scenario, thereby improving the target detection efficiency.
[0108] In a specific implementation of the present invention, the second obtaining module 320 is used to:
[0109] For each category of the labeled target, traverse the first data set and the second data set to determine sample data containing the labeled target of the current category;
[0110] The sample data containing the labeled targets of the current category are synthesized into a single-category dataset.
[0111] In a specific embodiment of the present invention, the detection module 330 is used to:
[0112] For each sample data input into the target detection model, obtain the classification scores and coordinate frame regression values of multiple categories predicted by the target detection model based on the input current sample data;
[0113] The target and background of the unlabeled category of the current sample data are taken as a combination, and the cross entropy loss and coordinate frame regression loss are calculated based on the classification score and coordinate frame regression value;
[0114] The parameters of the object detection model are adjusted based on the cross entropy loss and coordinate box regression loss.
[0115] In a specific embodiment of the present invention, the detection module 330 is used to:
[0116] The cross entropy loss is calculated based on the classification scores using the group normalized exponential function;
[0117] The coordinate box loss is calculated based on the coordinate box regression value using the generalized intersection-over-union ratio.
[0118] In a specific implementation of the present invention, the target detection device 300 further includes a first processing module, which is used to:
[0119] After training the target detection model based on the sample data in the N single-category data sets, a new category data set is obtained, and the category of the labeled target included in the sample data in the new category data set is the new category;
[0120] Based on the sample data in the N single-category data sets and the newly added category data sets, the target detection model is trained to detect the newly added category targets using the trained target detection model.
[0121] In a specific implementation of the present invention, the target detection device 300 further includes a second processing module, which is used to:
[0122] Update sample data in a first single-category data set, where the first single-category data set is any one of the N single-category data sets;
[0123] The target detection model is trained based on the sample data in the updated N single-category datasets.
[0124] In a specific embodiment of the present invention, the output of the target detection model includes a classification branch and a coordinate regression branch.
[0125] In a specific embodiment of the present invention, the detection module 330 is used to:
[0126] Input the data to be tested into the trained object detection model;
[0127] Obtain classification scores for multiple categories through classification branches;
[0128] Based on the classification score, determine the category of the target contained in the test data;
[0129] The coordinate position of the target contained in the data to be measured is obtained through the coordinate regression branch.
[0130] Corresponding to the above method embodiment, the embodiment of the present invention further provides an electronic device, such as Figure 4As shown, it includes a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.
[0131] Memory 403, used for storing computer programs;
[0132] The processor 401 is used to execute the program stored in the memory 403 to implement the following steps:
[0133] Obtain a first data set and a second data set, wherein the number of categories of the labeled targets in the first data set is X, and the number of categories of the labeled targets in the second data set is Y, where X and Y are positive integers;
[0134] The first data set and the second data set are split according to the categories of the labeled targets to obtain N single-category data sets, where each sample data in each single-category data set is labeled with only one category of targets, N≤X+Y;
[0135] Based on the sample data in N single-category data sets, the target detection model is trained to perform target detection using the trained target detection model; wherein, during the training process, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data are taken as a combination, the prediction loss is determined, and the parameters of the target detection model are adjusted.
[0136] The communication bus 404 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 404 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0137] The communication interface 402 is used for communication between the electronic device and other devices.
[0138] The memory 403 may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory 403 may also be at least one storage device located away from the aforementioned processor.
[0139] The above-mentioned processor 401 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0140] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the target detection method in the above embodiment is implemented.
[0141] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute the target detection method in the above embodiment.
[0142] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0143] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0144] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0145] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A target detection method, characterized in that: include: Obtain a first image data set and a second image data set, wherein the first image data set has X categories of labeled objects, and the second image data set has Y categories of labeled objects, where X and Y are positive integers; The first image dataset and the second image dataset are split according to the categories of the labeled targets to obtain N single-category datasets, where each sample data in each single-category dataset is labeled with only one category of targets, N≤X+Y; Based on the sample data in the N single-category data sets, the target detection model is trained to perform target detection using the trained target detection model; wherein, during the training process, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data are taken as a combination, the prediction loss is determined, and the parameters of the target detection model are adjusted; The method of determining the prediction loss for each sample data input into the target detection model, taking the target and background of the category not labeled in the current sample data as a combination, and adjusting the parameters of the target detection model includes: For each sample data input to the target detection model, obtain classification scores and coordinate frame regression values of multiple categories predicted by the target detection model based on the input current sample data; Taking the target and background of the unlabeled category of the current sample data as a combination, and calculating the cross entropy loss and the coordinate frame regression loss based on the classification score and the coordinate frame regression value; The parameters of the object detection model are adjusted according to the cross entropy loss and the coordinate box regression loss.
2. The target detection method according to claim 1, characterized in that: The step of splitting the first image dataset and the second image dataset according to the categories of the labeled objects to obtain N single-category datasets includes: For each category of the labeled object, traverse the first image data set and the second image data set to determine sample data containing the labeled object of the current category; The sample data containing the labeled targets of the current category are synthesized into a single-category dataset.
3. The target detection method according to claim 1, characterized in that: The calculating the cross entropy loss and the coordinate frame regression loss based on the classification score and the coordinate frame regression value includes: Calculating a cross entropy loss based on the classification scores using a group normalized exponential function; The coordinate box loss is calculated based on the coordinate box regression value using the generalized intersection-over-union ratio.
4. The target detection method according to claim 1, characterized in that: After the target detection model is trained based on the sample data in the N single-category data sets, the method further includes: Obtaining a newly added category data set, wherein the category of the labeled target included in the sample data in the newly added category data set is the newly added category; The target detection model is trained based on the sample data in the N single-category data sets and the newly added category data set, so as to detect the targets of the newly added category using the trained target detection model.
5. The target detection method according to claim 1, characterized in that: Also includes: Updating sample data in a first single-category data set, where the first single-category data set is any one of the N single-category data sets; The object detection model is trained based on the sample data in the updated N single-category data sets.
6. The target detection method according to any one of claims 1 to 5, characterized in that: The output of the target detection model includes a classification branch and a coordinate regression branch.
7. The target detection method according to claim 6, characterized in that: The method of performing target detection using the trained target detection model includes: Inputting the data to be tested into the trained target detection model; Obtain classification scores of multiple categories through the classification branches; Based on the classification score, determining the category of the target contained in the test data; The coordinate position of the target contained in the data to be measured is obtained through the coordinate regression branch.
8. A target detection device, characterized in that: include: A first acquisition module is used to obtain a first image data set and a second image data set, wherein the first image data set has X categories of labeled objects, and the second image data set has Y categories of labeled objects, where X and Y are positive integers; A second acquisition module is used to split the first image data set and the second image data set according to the categories of the labeled targets to obtain N single-category data sets, where each sample data in each single-category data set is labeled with only one category of targets, N≤X+Y; A detection module, used to train a target detection model based on sample data in N single-category data sets, so as to perform target detection using the trained target detection model; wherein, during the training process, for each sample data input into the target detection model, the target and background of the category not labeled in the current sample data are taken as a combination, the prediction loss is determined, and the parameters of the target detection model are adjusted; The method of determining the prediction loss for each sample data input into the target detection model, taking the target and background of the category not labeled in the current sample data as a combination, and adjusting the parameters of the target detection model includes: For each sample data input to the target detection model, obtain classification scores and coordinate frame regression values of multiple categories predicted by the target detection model based on the input current sample data; Taking the target and background of the unlabeled category of the current sample data as a combination, and calculating the cross entropy loss and the coordinate frame regression loss based on the classification score and the coordinate frame regression value; The parameters of the object detection model are adjusted according to the cross entropy loss and the coordinate box regression loss.
9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the steps of the target detection method according to any one of claims 1 to 7 when executing a program stored in a memory.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the target detection method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Data processing method and device and electronic equipment
CN111724371A
Target detection model training method and device based on deep learning and storage medium
CN111783844A
Model training method, device and equipment, and storage medium
CN112712094A
Network training method, image detection method and medium
CN113379687A
End-to-end semi-supervised target detection method based on improved yolov5
CN114399683A
