Label assignment method and apparatus based on target detection

Abstract

The present disclosure provides a label assignment method and device based on target detection. The label assignment method based on target detection comprises: obtaining training samples of a target detection model; determining positive samples and negative samples in the training samples according to the joint loss value between each training sample and the ground truth box corresponding to each training sample; determining the predicted box category corresponding to the negative sample according to the intersection over union between the predicted box corresponding to the negative sample and the ground truth box, wherein the predicted box category corresponding to the negative sample comprises a first type of false positive predicted box, a second type of false positive predicted box and a third type of false positive predicted box; and determining the class label of the positive sample and the regression loss weight of the positive sample according to the target of suppressing the predicted box category corresponding to the negative sample. Through the present disclosure, the generation of different types of false positive predicted boxes is effectively suppressed, the target detection model is optimized, the accuracy of target detection is improved, and target detection of objects with complex textures can also be realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a label assignment method and apparatus based on target detection. Background Technology

[0002] Object detection is a crucial task in computer vision. Understanding image content is a prerequisite for intelligent surveillance and autonomous driving technologies, and accurate object detection is the foundation of image content understanding. Label assignment is a critical step in the training phase of object detection models, significantly impacting their accuracy. The object detection task involves finding objects of a specific category in a given image, determining bounding boxes around the objects, and predicting the probability that the objects within those boxes belong to a particular category.

[0003] Existing object detection algorithms generate a large number of densely distributed prediction boxes, requiring label assignment for each prediction box. This includes setting the target category and regression target for each prediction box. Under special conditions, it is also necessary to set regression loss weights for each positive sample.

[0004] Label assignment methods include static and dynamic methods. In static methods, the label of each training sample remains unchanged throughout the training process. Positive and negative samples are distinguished based on the overlap or relative position between the anchor box or anchor point and the ground truth box. The class label of the training sample is usually a hard label with a value of 0 or 1. In dynamic methods, the label of each training sample changes continuously during the training process. Positive and negative samples are distinguished based on the loss function value of the training sample. The class label of the training sample can be either a soft label or a hard label. Compared with static methods, dynamic methods are more robust and have significant advantages when applied to objects with complex textures or concentrated object distribution. However, in scenes with dense object distribution, dynamic methods can still produce detection errors. Suppressing detection errors and improving the performance of object detection algorithms remains an important research direction in the field of computer vision. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the purpose of this disclosure is to provide a label allocation method and apparatus based on target detection.

[0006] To achieve the above objectives, according to one aspect of this disclosure, a label assignment method based on object detection is provided, comprising:

[0007] Obtain training samples for the object detection model;

[0008] Based on the joint loss value between each training sample and the ground truth box corresponding to each training sample, the positive and negative samples in the training samples are determined.

[0009] Based on the intersection-union ratio between the predicted bounding box and the ground truth bounding box corresponding to the negative sample, the type of predicted bounding box corresponding to the negative sample is determined. The types of predicted bounding boxes corresponding to the negative sample include the first type of false positive predicted bounding box, the second type of false positive predicted bounding box, and the third type of false positive predicted bounding box.

[0010] Based on the objective of suppressing the predicted box types corresponding to the negative samples, the class labels of the positive samples and the regression loss weights of the positive samples are determined.

[0011] Optionally, determining the class label of the positive sample and the regression loss weight of the positive sample based on the objective of suppressing the predicted box types corresponding to the negative samples includes:

[0012] If the goal is to suppress the first type of false positive prediction boxes, the first category label of the positive sample is determined based on the intersection-union ratio between the prediction box and the ground truth box corresponding to the positive sample.

[0013] If the goal is to suppress the second type of false positive prediction boxes, the second category label of the positive sample and the regression loss weight of the positive sample are determined based on the true positive prediction box probability of the positive sample.

[0014] Optionally, if the goal is to suppress the first type of false positive prediction boxes, determining the first category label of the positive sample based on the intersection-union ratio (IUU) between the prediction box and the ground truth box corresponding to the positive sample includes:

[0015]

[0016]

[0017] in, This refers to the positive sample. The first category label of the positive sample is represented by sigmoid, which represents the sigmoid function. This represents the prediction box corresponding to the positive sample. This represents the truth box corresponding to the positive sample. This represents the intersection-union ratio between the predicted bounding box and the ground truth bounding box corresponding to the positive sample.

[0018] Optionally, determining the class label of the positive sample and the regression loss weight of the positive sample based on the objective of suppressing the predicted box type corresponding to the negative sample further includes:

[0019] If the goal is to suppress the second type of false positive prediction boxes, the confidence of the positive samples after the model training of the target detection model is completed is modeled using a Gaussian distribution to determine the probability of the true positive prediction boxes of the positive samples.

[0020] Optionally, if the goal is to suppress the second type of false positive prediction boxes, the confidence scores of the positive samples after model training of the target detection model are modeled using a Gaussian distribution to determine the true positive prediction box probability of the positive samples, including:

[0021]

[0022]

[0023]

[0024] in, This represents the confidence level of the positive samples after the target detection model has been trained, and N represents the identifier of the Gaussian distribution. This represents the mean of the Gaussian distribution. This represents the standard deviation of the Gaussian distribution. denoted by , c represents the confidence level of the positive sample, T represents the total number of training iterations of the object detection model, and t represents the current number of training iterations of the object detection model.

[0025]

[0026] in, F represents the probability of a true positive prediction box for the positive sample. X Let F represent the probability density function of the random variable X. X Let u represent the cumulative distribution function of the random variable X. k This indicates the confidence level of the positive sample.

[0027] Optionally, if the goal is to suppress the second type of false positive prediction boxes, determining the second category label of the positive sample and the regression loss weight of the positive sample based on the true positive prediction box probability of the positive sample includes:

[0028]

[0029]

[0030] in, The second category label represents the positive sample. This represents the crossover ratio (CROR) between positive sample k and the true positive prediction boxes. This represents the intersection-union ratio (CUC) between positive sample i and the true positive prediction box. This represents the crossover-union ratio between the positive sample i and the positive sample m.

[0031]

[0032]

[0033] f gate =sigmoid(10×(IOU′-IOU″))

[0034]

[0035]

[0036]

[0037] in, This represents the regression loss weight of the positive samples. This indicates the regression loss weights used to suppress the second type of false positive prediction boxes. The regression loss weight, f, is used to suppress the first type of false positive prediction boxes. gate A soft gating function indicates that the intersection-union ratio (CUI) between the positive sample and the true positive prediction box is greater than the CUI between the positive sample and the true value box. The value represents the confidence level of the positive sample, IOU′ represents the expected value of the intersection-union ratio between the true positive prediction box and the ground truth box, and IOU″ represents the intersection-union ratio between the positive sample and the true positive prediction box. This represents the intersection-union ratio (IUR) between the positive sample m and the truth box. This represents the probability that the positive sample m is a true positive sample. This represents the cross-union ratio between the positive sample k and the true positive prediction box.

[0038] Optionally, the method further includes:

[0039] Determine the classification loss value and regression loss value between each training sample and the ground truth box corresponding to each training sample;

[0040] The classification loss value and the regression loss value are weighted to determine the joint loss value between each training sample and the ground truth box corresponding to each training sample.

[0041] Optionally, determining the positive and negative samples in the training samples based on the joint loss value between each training sample and the ground truth box corresponding to each training sample includes:

[0042] The training samples that have a minimum joint loss value with each ground truth box are determined as the positive samples;

[0043] Training samples that are not the positive samples are identified as negative samples.

[0044] Optionally, determining the type of the predicted bounding box corresponding to the negative sample based on the intersection-union ratio (IU) between the predicted bounding box and the ground truth bounding box corresponding to the negative sample includes:

[0045] If the intersection-union ratio of the predicted bounding box and the ground truth bounding box corresponding to the negative sample is not less than the first preset threshold and less than the second preset threshold, the predicted bounding box of the negative sample is determined to be a first type of false positive predicted bounding box.

[0046] If the intersection-union ratio of the predicted bounding box and the ground truth bounding box corresponding to the negative sample is not less than the second preset threshold, the predicted bounding box of the negative sample is determined to be a second type of false positive predicted bounding box.

[0047] If the intersection-union ratio (IUU) of the predicted bounding box and the ground truth bounding box corresponding to the negative sample is less than the first preset threshold, the predicted bounding box of the negative sample is determined to be a third type of false positive predicted bounding box.

[0048] According to a second aspect of this disclosure, a tag allocation apparatus based on target detection is provided, comprising:

[0049] The acquisition module is used to acquire training samples for the object detection model;

[0050] The first determining module is used to determine the positive and negative samples in the training samples based on the joint loss value between each training sample and the ground truth box corresponding to each training sample.

[0051] The second determining module is used to determine the type of prediction box corresponding to the negative sample based on the intersection-union ratio between the prediction box and the ground truth box corresponding to the negative sample. The type of prediction box corresponding to the negative sample includes a first type of false positive prediction box, a second type of false positive prediction box, and a third type of false positive prediction box.

[0052] The third determining module is used to determine the category label of the positive sample and the regression loss weight of the positive sample based on the objective of suppressing the predicted box type corresponding to the negative sample.

[0053] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0054] The above technical solution divides positive and negative samples in the training samples based on the joint loss value between each training sample and its corresponding ground truth box. By adopting a dynamic label allocation method, it is possible to detect objects with complex textures. Based on the intersection-union ratio (IUU) of the predicted box and the ground truth box corresponding to the negative sample, the predicted box type of the negative sample is determined, which classifies the false detections in the target detection results. For the predicted box type of the negative sample, the corresponding positive sample category label and regression loss are designed to specifically suppress the occurrence of false target detections and improve the accuracy of target detection. Furthermore, the determined positive sample category label and regression loss can also be used to train the target detection model, dynamically improving the detection performance of the target detection model. Attached Figure Description

[0055] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0056] Figure 1 This is a flowchart illustrating a label assignment method based on object detection according to an exemplary embodiment.

[0057] Figure 2 This is a flowchart illustrating another object detection-based label assignment method according to an exemplary embodiment.

[0058] Figure 3 This is a block diagram illustrating a tag allocation device based on target detection according to an exemplary embodiment.

[0059] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0060] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0061] Figure 1 This is a flowchart illustrating a label assignment method based on object detection, according to an exemplary embodiment. Figure 1 As shown, a label assignment method based on object detection includes steps S11 to S14.

[0062] S11, Obtain training samples for the object detection model.

[0063] Obtain training samples for model training using the object detection algorithm. The set of training samples is denoted as S = {s1, s2, ...}, where S represents the set of training samples and s1, s2, ... represent training samples.

[0064] In one possible embodiment, the set of truth boxes for the training samples is denoted as... Among them B GT This represents the set of truth boxes corresponding to the training sample set. This represents the truth boxes corresponding to the training samples.

[0065] In this context, training samples can be assigned to multiple ground truth boxes simultaneously, meaning that each ground truth box can correspond to multiple training samples, and each training sample can also correspond to multiple ground truth boxes.

[0066] S12, determine the positive and negative samples in the training samples based on the joint loss value between each training sample and the ground truth box corresponding to each training sample.

[0067] When training the target detection model using training samples, each training sample s is determined. i The ground truth box corresponding to each training sample The joint loss value between them is denoted as

[0068] In one possible embodiment, a preset number of training samples that have a minimum joint loss value with each truth box are identified as positive samples; training samples that are not said positive samples are identified as negative samples.

[0069] As an example, for each truth box The k training samples with the smallest joint loss values ​​with each ground truth box are assigned to that ground truth box as its positive samples.

[0070] If a training sample corresponds to multiple ground truth boxes, then the training sample is considered as the positive sample of the ground truth box with which it has the smallest joint loss value.

[0071] S13. Determine the type of prediction box corresponding to the negative sample based on the intersection-union ratio between the predicted box and the ground truth box corresponding to the negative sample.

[0072] Among them, the prediction box types corresponding to negative samples include Type I false positive prediction boxes, Type II false positive prediction boxes, and Type III false positive prediction boxes.

[0073] In one possible embodiment, the predicted bounding boxes of negative samples can be classified based on the intersection-over-union (IoU) ratio between the predicted bounding boxes and the ground truth bounding boxes. The IoU ratio represents the regression accuracy of the predicted bounding boxes; a higher IoU ratio indicates higher regression accuracy.

[0074] As an example, if the intersection-union ratio of the predicted bounding box and the ground truth bounding box corresponding to the negative sample is not less than the first preset threshold and less than the second preset threshold, the predicted bounding box of the negative sample is determined to be a first type of false positive predicted bounding box.

[0075] The first preset threshold can be set to 0.1, and the second preset threshold can be set to 0.5. Therefore, the intersection-union ratio (IUU) of the first type of false positive prediction boxes is 0.1 ≤ A < 0.5.

[0076] As another example, if the intersection-union ratio of the predicted bounding box and the ground truth bounding box corresponding to the negative sample is not less than the second preset threshold, the predicted bounding box of the negative sample is determined to be a second type of false positive predicted bounding box.

[0077] In this disclosure, the crossover ratio of the second type of false positive prediction boxes can be B≥0.5.

[0078] As another example, if the intersection-union ratio of the predicted bounding box and the ground truth bounding box corresponding to the negative sample is less than the first preset threshold, the predicted bounding box of the negative sample is determined to be a third type of false positive predicted bounding box.

[0079] In this disclosure, the crossover ratio of the third type of false positive prediction boxes can be C < 0.1.

[0080] In this way, it is not necessary to assign negative samples to any truth boxes.

[0081] S14. Based on the objective of suppressing the predicted box types corresponding to negative samples, determine the class labels and regression loss weights of positive samples.

[0082] In this disclosure, the predicted bounding boxes corresponding to negative samples are classified as first-type false positive predicted boxes and second-type false positive predicted boxes as erroneous target detection results. To suppress the occurrence of erroneous target detection results, the category labels of positive samples are assigned. When a second-type false positive predicted box appears, not only the category label of the positive sample is determined, but also the regression loss weight of the positive sample is determined. The target detection model is retrained based on the positive samples with category labels and the regression loss weight of the positive samples to suppress the occurrence of erroneous target detection results.

[0083] As an example, if the goal is to suppress false positive prediction boxes of the first type, the first category label of the positive sample is determined based on the cross-union ratio between the prediction box and the ground truth box corresponding to the positive sample.

[0084] As another example, if the goal is to suppress false positive prediction boxes of type II, the second category label of positive samples and the regression loss weight of positive samples are determined based on the probability of true positive prediction boxes of positive samples.

[0085] The above technical solution divides positive and negative samples in the training samples based on the joint loss value between each training sample and its corresponding ground truth box. By adopting a dynamic label allocation method, it is possible to detect objects with complex textures. Based on the intersection-union ratio (IUU) of the predicted box and the ground truth box corresponding to the negative sample, the predicted box type of the negative sample is determined, which classifies the false detections in the target detection results. For the predicted box type of the negative sample, the corresponding positive sample category label and regression loss are designed to specifically suppress the occurrence of false target detections and improve the accuracy of target detection. Furthermore, the determined positive sample category label and regression loss can also be used for model training of the target detection model, dynamically improving the detection performance of the target detection model.

[0086] Figure 2 This is a flowchart illustrating another object detection-based label assignment method according to an exemplary embodiment.

[0087] like Figure 2 As shown, in some possible embodiments, a label assignment method based on object detection may further include S15 to S16 before S12, determining the joint loss value between each training sample and the ground truth box corresponding to each training sample.

[0088] S15, determine the classification loss value and regression loss value between each training sample and the ground truth box corresponding to each training sample.

[0089] Since negative samples are not assigned to any truth boxes, there is no need to calculate classification loss and regression loss values ​​for them.

[0090] In this disclosure, the classification loss value between each training sample and its corresponding ground truth box is calculated using the following formula:

[0091]

[0092] Among them, s i Indicates a positive sample. Represents a truth box. conf(s) represents the classification loss value between a positive sample and its corresponding ground truth box. i ) represents positive sample s i The confidence level.

[0093] In this disclosure, the regression loss value between each training sample and the ground truth box corresponding to each training sample is calculated using the following formula:

[0094]

[0095] Among them, L r Let b(s) represent the regression loss value. i () represents the prediction box corresponding to a positive sample. This represents the truth box corresponding to a positive sample. This represents the cross-union ratio between the predicted bounding box and the ground truth bounding box corresponding to the positive sample.

[0096] In the process of target detection, it is necessary to train the target detection model. Based on the error between the predicted box and the ground truth box corresponding to each training sample, the regression loss value is determined. By reducing the regression loss value, the target detection error is reduced, thereby improving the detection accuracy and performance of the target detection model.

[0097] S16, weight the classification loss value and the regression loss value to determine the joint loss value between each training sample and the ground truth box corresponding to each training sample.

[0098] The weighted processing includes a weighted summation process. The classification loss value and its weight, and the regression loss value and its weight are weighted and summed, and the result is used as the joint loss value between each training sample and its corresponding ground truth box.

[0099] Predicted boxes with higher confidence have a higher output probability. To ensure the regression accuracy of predicted boxes with higher confidence, an appropriate regression loss weight can be assigned to each training sample. The regression loss weight corresponding to the training sample with higher confidence is also higher. The regression loss value of each training sample and its corresponding regression loss weight are multiplied and added to the joint loss consideration to optimize the object detection model and reduce the object detection error.

[0100] The above technical solution determines the joint loss value between each training sample and the ground truth box corresponding to each training sample. The joint loss value can be used to classify positive and negative samples in the training samples, and can also be used as an evaluation index to optimize the performance of the object detection model.

[0101] In this disclosure, the truth box will be assigned The set of positive samples is denoted as Positive samples The predicted bounding box for positive samples is

[0102] In one possible embodiment, if the goal is to suppress false positive prediction boxes of the first type, determining the first category label of a positive sample based on the intersection-union ratio (IUU) between the predicted box and the ground truth box corresponding to the positive sample includes:

[0103]

[0104]

[0105] in, Indicates a positive sample. The first class label represents the positive sample, and sigmoid represents the sigmoid function. This represents the prediction box corresponding to a positive sample. This represents the truth box corresponding to a positive sample. This represents the cross-union ratio between the predicted bounding box and the ground truth bounding box corresponding to the positive sample.

[0106] The sigmoid function is used to map real numbers to the range 0 and 1.

[0107] In this disclosure, the first category label of the positive sample is set as a soft label, and the value of the first category label is proportional to the regression accuracy, that is, the value of the first category label is proportional to the intersection-union ratio between the predicted box and the ground truth box corresponding to the positive sample.

[0108] In another possible embodiment, if the goal is to suppress the second type of false positive prediction boxes, the confidence of positive samples after the target detection model has been trained is modeled using a Gaussian distribution to determine the probability of true positive prediction boxes for positive samples.

[0109] Among them, after training the target detection model using training samples, the confidence scores of the positive samples follow a Gaussian distribution:

[0110]

[0111]

[0112]

[0113] in, This represents the confidence level of positive samples after the object detection model has been trained, where N represents the identifier of the Gaussian distribution. This represents the mean of a Gaussian distribution. This represents the standard deviation of a Gaussian distribution. represents the confidence level of positive samples, c represents the hyperparameter, T represents the total number of training iterations of the object detection model, and t represents the current number of training iterations of the object detection model.

[0114] The probability of a positive sample being classified as the highest among all positive samples in the same true bounding box at the end of the model training of the object detection model is taken as the true positive prediction box probability of each positive sample.

[0115] Probability of true positive prediction boxes for positive samples:

[0116]

[0117] in, f represents the probability of a true positive prediction box for a positive sample. X Let denot X be the probability density function of random variable X, and let FX be the cumulative distribution function of random variable X. k This indicates the confidence level of positive samples.

[0118] In another possible embodiment, if the goal is to suppress second-type false positive prediction boxes, the determination of the second-class label of positive samples and the regression loss weights of positive samples based on the true positive prediction box probabilities of positive samples includes:

[0119] Determine the second category label for positive samples:

[0120]

[0121]

[0122] in, The second category label represents the positive sample. This represents the crossover ratio (CROR) between positive sample k and the true positive prediction boxes. This represents the crossover ratio (CROR) between positive sample i and the true positive prediction box. This represents the crossover ratio (CROR) between positive sample i and positive sample m.

[0123] In this disclosure, the second category label of the positive sample is set as a soft label, and the value of the second category label is inversely proportional to the probability of the true positive prediction box output by the positive sample, and the regression loss weight of the positive sample is determined.

[0124] Determining the regression loss weights for positive samples includes:

[0125]

[0126]

[0127] f gate =sigmoid(10×(IOU′-IOU”))

[0128]

[0129]

[0130]

[0131] in, This represents the regression loss weights for positive samples. This indicates the regression loss weights used to suppress type II false positive prediction boxes. The regression loss weights, f, are used to suppress false positive prediction boxes of type I. gate This indicates a soft gating function where the cross-union ratio (CUI) between positive samples and true positive predicted boxes is greater than the CUI between positive samples and true positive predicted boxes. represents the confidence level of the positive sample, IOU′ represents the expected value of the intersection-union ratio between the true positive prediction box and the ground truth box, and I0U″ represents the intersection-union ratio between the positive sample and the true positive prediction box. This represents the intersection-union ratio (IUR) between the positive sample m and the truth box. This represents the probability that a positive sample m is a true positive sample. This represents the cross-union ratio between positive sample k and the true positive prediction box.

[0132] In some possible implementations, the object detection model is retrained based on the class labels of positive samples and the regression loss weights to optimize the parameters and performance of the object detection model.

[0133] The above technical solution enables the design of targeted positive sample category labels based on different types of false positive prediction boxes, and the design of regression loss weights for the second type of false positive prediction boxes. This effectively suppresses different types of false positive prediction boxes, reduces the probability of erroneous target detection results, and improves the detection performance of the target detection model in scenarios with densely distributed target objects.

[0134] Based on the same concept, this disclosure also provides a tag allocation device based on target detection. Figure 3 This is a block diagram illustrating a tag allocation device based on target detection according to an exemplary embodiment. (Refer to...) Figure 3 The target detection-based label allocation device 100 includes: an acquisition module 110, a first determination module 120, a second determination module 130, and a third determination module 140.

[0135] The acquisition module 110 is used to acquire training samples for the object detection model;

[0136] The first determining module 120 is used to determine the positive and negative samples in the training samples based on the joint loss value between each training sample and the ground truth box corresponding to each training sample.

[0137] The second determining module 130 is used to determine the type of prediction box corresponding to the negative sample based on the intersection-union ratio between the prediction box and the ground truth box corresponding to the negative sample. The type of prediction box corresponding to the negative sample includes a first type of false positive prediction box, a second type of false positive prediction box, and a third type of false positive prediction box.

[0138] The third determining module 140 is used to determine the category label of the positive sample and the regression loss weight of the positive sample based on the objective of suppressing the predicted box type corresponding to the negative sample.

[0139] The above technical solution divides positive and negative samples in the training samples based on the joint loss value between each training sample and its corresponding ground truth box. By adopting a dynamic label allocation method, it is possible to detect objects with complex textures. Based on the intersection-union ratio (IUU) of the predicted box and the ground truth box corresponding to the negative sample, the predicted box type of the negative sample is determined, which classifies the false detections in the target detection results. For the predicted box type of the negative sample, the corresponding positive sample category label and regression loss are designed to specifically suppress the occurrence of false target detections and improve the accuracy of target detection. Furthermore, the determined positive sample category label and regression loss can also be used to train the target detection model, dynamically improving the detection performance of the target detection model.

[0140] Optionally, the third determining module 140 includes:

[0141] The first determining submodule is used to determine the first category label of the positive sample based on the intersection-union ratio between the predicted box and the ground truth box corresponding to the positive sample, with the goal of suppressing the first type of false positive prediction box;

[0142] The second determining submodule is used to determine the second category label of the positive sample and the regression loss weight of the positive sample based on the true positive prediction box probability of the positive sample, if the goal is to suppress the second type of false positive prediction box.

[0143] Optionally, the first determining submodule includes:

[0144]

[0145]

[0146] in, This refers to the positive sample. The first category label of the positive sample is represented by sigmoid, which represents the sigmoid function. This represents the prediction box corresponding to the positive sample. This represents the truth box corresponding to the positive sample. This represents the intersection-union ratio between the predicted bounding box and the ground truth bounding box corresponding to the positive sample.

[0147] Optionally, the third determining module 140 further includes:

[0148] The third determining submodule is used to determine the probability of true positive prediction boxes for positive samples by modeling the confidence of the positive samples after model training of the target detection model using a Gaussian distribution if the goal is to suppress the second type of false positive prediction boxes.

[0149] Optionally, the third determining submodule includes:

[0150]

[0151]

[0152]

[0153] in, This represents the confidence level of the positive samples after the target detection model has been trained, and N represents the identifier of the Gaussian distribution. This represents the mean of the Gaussian distribution. This represents the standard deviation of the Gaussian distribution. denoted by , c represents the confidence level of the positive sample, T represents the total number of training iterations of the object detection model, and t represents the current number of training iterations of the object detection model.

[0154]

[0155] in, F represents the probability of a true positive prediction box for the positive sample. X Let F represent the probability density function of the random variable X. X Let u represent the cumulative distribution function of the random variable X. k This indicates the confidence level of the positive sample.

[0156] Optionally, the second determining submodule includes:

[0157]

[0158]

[0159] in, The second category label represents the positive sample. This represents the crossover ratio (CROR) between positive sample k and the true positive prediction boxes. This represents the intersection-union ratio (CUC) between positive sample i and the true positive prediction box. This represents the crossover-union ratio between the positive sample i and the positive sample m.

[0160]

[0161]

[0162] f gate =sigmoid(10×(IOU′-IOU″))

[0163]

[0164]

[0165]

[0166] in, This represents the regression loss weight of the positive samples. This indicates the regression loss weights used to suppress the second type of false positive prediction boxes. The regression loss weight, f, is used to suppress the first type of false positive prediction boxes. gate A soft gating function indicates that the intersection-union ratio (CUI) between the positive sample and the true positive prediction box is greater than the CUI between the positive sample and the true value box. The value represents the confidence level of the positive sample, IOU′ represents the expected value of the intersection-union ratio between the true positive prediction box and the ground truth box, and IOU″ represents the intersection-union ratio between the positive sample and the true positive prediction box. This represents the intersection-union ratio (IUR) between the positive sample m and the truth box. This represents the probability that the positive sample m is a true positive sample. This represents the cross-union ratio between the positive sample k and the true positive prediction box.

[0167] Optionally, the device 100 further includes:

[0168] The fourth determination submodule is used to determine the classification loss value and regression loss value between each training sample and the ground truth box corresponding to each training sample;

[0169] The fifth determination submodule is used to perform weighted processing on the classification loss value and the regression loss value to determine the joint loss value between each training sample and the ground truth box corresponding to each training sample.

[0170] Optionally, the first determining module 120 includes:

[0171] The sixth determination submodule is used to determine the training samples that have a minimum joint loss value with each truth box as the positive samples;

[0172] The seventh determination submodule is used to determine training samples that are not the positive samples as negative samples.

[0173] Optionally, the second determining module 130 includes:

[0174] The first classification submodule is used to determine the type of the predicted box of the negative sample as a first type of false positive predicted box if the intersection-union ratio of the predicted box and the ground truth box corresponding to the negative sample is not less than a first preset threshold and less than a second preset threshold.

[0175] The second classification submodule is used to determine the type of the predicted box of the negative sample as a second type of false positive predicted box if the intersection-union ratio of the predicted box and the ground truth box corresponding to the negative sample is not less than the second preset threshold.

[0176] The third classification submodule is used to determine that the predicted box of the negative sample is a third type of false positive predicted box if the intersection-union ratio of the predicted box and the ground truth box corresponding to the negative sample is less than the first preset threshold.

[0177] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0178] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 4 As shown, in some possible embodiments, this disclosure may also provide an electronic device 400, which may include: a processor 401, a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input / output interface 404, and a communication component 405.

[0179] The processor 401 controls the overall operation of the electronic device 400 to complete all or part of the steps in the target detection-based tag allocation method of the first aspect described above. The memory 402 stores various types of data to support the operation of the electronic device 400. This data may include, for example, instructions for any application or method operating on the electronic device 400, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 403 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 402 or transmitted via communication component 405. The audio component also includes at least one speaker for outputting audio signals. Input / output interface 404 provides an interface between processor 401 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 305 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0180] In another exemplary embodiment, a non-transitory computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the target detection-based tag allocation method of the first aspect described above. For example, the computer-readable storage medium may be the memory including the program instructions described above, which may be executed by a processor of an electronic device to complete the target detection-based tag allocation method described above.

[0181] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described target detection-based tag allocation method when executed by the programmable device.

[0182] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. A target detection-based label assignment method, characterized by, The method comprises: obtaining training samples of a target detection model; determining positive samples and negative samples in the training samples according to a joint loss value between each training sample and a ground truth box corresponding to each training sample; determining a predicted box category corresponding to the negative sample according to an intersection over union between a predicted box corresponding to the negative sample and the ground truth box, the predicted box category corresponding to the negative sample comprising a first type of false positive predicted box, a second type of false positive predicted box, and a third type of false positive predicted box; determining a class label of the positive sample and a regression loss weight of the positive sample according to a target of suppressing the predicted box category corresponding to the negative sample, the predicted box category corresponding to the negative sample being the first type of false positive predicted box, the second type of false positive predicted box being an incorrect target detection result, assigning the class label of the positive sample for suppressing the occurrence of the incorrect target detection result, and determining the class label of the positive sample and the regression loss weight of the positive sample when the second type of false positive predicted box occurs, retraining the target detection model according to the positive sample with the class label and the regression loss weight of the positive sample to suppress the occurrence of the incorrect target detection result; wherein the determining the class label of the positive sample and the regression loss weight of the positive sample according to the target of suppressing the predicted box category corresponding to the negative sample comprises: if the target is to suppress the first type of false positive predicted box, determining a first class label of the positive sample according to the intersection over union between the predicted box corresponding to the positive sample and the ground truth box; if the target is to suppress the second type of false positive predicted box, determining a second class label of the positive sample and the regression loss weight of the positive sample according to a true positive predicted box probability of the positive sample.

2. The method of claim 1, wherein, The determining the first class label of the positive sample according to the target of suppressing the first type of false positive predicted box comprises: wherein, represents the positive sample, represents a first class label of the positive sample, sigmoid represents a sigmoid function, represents a predicted bounding box corresponding to the positive sample, represents a ground truth bounding box corresponding to the positive sample, represents an intersection over union between the predicted bounding box and the ground truth bounding box corresponding to the positive sample.

3. The method of claim 1, wherein, The determining the class label of the positive sample and the regression loss weight of the positive sample according to the target of suppressing the predicted box category corresponding to the negative sample further comprises: if the target is to suppress the second type of false positive predicted box, modeling a confidence of the positive sample after the model training of the target detection model is completed by using a Gaussian distribution to determine a true positive predicted box probability of the positive sample.

4. The method of claim 3, wherein, The determining the second class label of the positive sample and the regression loss weight of the positive sample according to the target of suppressing the second type of false positive predicted box comprises: wherein, denotes the confidence of the positive sample after the model training of the target detection model is completed, N denotes an identifier of the Gaussian distribution, denotes a mean of the Gaussian distribution, denotes a standard deviation of the Gaussian distribution, denotes the confidence of the positive sample, c denotes a hyperparameter, T denotes a total number of model training of the target detection model, and t denotes a current model training number of the target detection model. wherein, denotes the true positive prediction box probability of the positive sample, f X denotes the probability density function of the random variable X, F X denotes the cumulative distribution function of the random variable X, u k denotes the confidence of the positive sample.

5. The method of claim 4, wherein, The method further comprises: determining a classification loss value and a regression loss value between each training sample and a ground truth box corresponding to each training sample; wherein, represents a second class label of the positive sample, represents an intersection over union between the positive sample k and the true positive prediction box, represents an intersection over union between the positive sample i and the true positive prediction box, represents an intersection over union between the positive sample i and the positive sample m; wherein, represents a regression loss weight of the positive sample, represents a regression loss weight for suppressing the second type of false positive predicted frame, represents a regression loss weight for suppressing the first type of false positive predicted frame, gate represents a soft gating function that the intersection over union between the positive sample and the true positive predicted frame is greater than the intersection over union between the positive sample and the true value frame, represents the confidence of the positive sample, IOU' represents the expected value of the intersection over union between the true positive predicted frame and the true value frame, and IOU" represents the intersection over union between the positive sample and the true positive predicted frame, represents the intersection over union between the positive sample m and the true value frame, represents the probability that the positive sample m is the true positive positive sample, represents the intersection over union between the positive sample k and the true positive predicted frame.

6. The method of claim 1, wherein, ​ ​ The classification loss value and the regression loss value are weighted to determine a joint loss value between each training sample and a ground truth box corresponding to each training sample.

7. The method of claim 6, wherein, The joint loss value between each training sample and a ground truth box corresponding to each training sample is used to determine positive samples and negative samples in the training samples. A preset number of training samples with the smallest joint loss value between each ground truth box are determined as the positive samples. The training samples other than the positive samples are determined as negative samples.

8. The method of claim 1, wherein, The IOU between the prediction box corresponding to the negative sample and the ground truth box is used to determine the prediction box category of the prediction box corresponding to the negative sample. If the IOU between the prediction box corresponding to the negative sample and the ground truth box is not less than a first preset threshold and less than a second preset threshold, the prediction box category of the negative sample is determined as a first type of false positive prediction box. If the IOU between the prediction box corresponding to the negative sample and the ground truth box is not less than the second preset threshold, the prediction box category of the negative sample is determined as a second type of false positive prediction box. If the IOU between the prediction box corresponding to the negative sample and the ground truth box is less than the first preset threshold, the prediction box category of the negative sample is determined as a third type of false positive prediction box.

9. A target detection-based label assignment apparatus characterized by comprising: The method comprises the following steps: An acquisition module is configured to acquire training samples of a target detection model. A first determination module is configured to determine positive samples and negative samples in the training samples according to a joint loss value between each training sample and a ground truth box corresponding to each training sample. A second determination module is configured to determine a prediction box category of a prediction box corresponding to the negative sample according to an IOU between the prediction box corresponding to the negative sample and the ground truth box, wherein the prediction box category of the prediction box corresponding to the negative sample comprises a first type of false positive prediction box, a second type of false positive prediction box, and a third type of false positive prediction box. A third determination module is configured to determine a class label of the positive sample and a regression loss weight of the positive sample according to a target for suppressing the prediction box category of the negative sample, wherein the prediction box category of the negative sample is the first type of false positive prediction box or the second type of false positive prediction box is an incorrect target detection result, the class label of the positive sample is assigned for suppressing the occurrence of the incorrect target detection result, the class label of the positive sample and the regression loss weight of the positive sample are determined when the second type of false positive prediction box occurs, the target detection model is retrained according to the positive sample with the class label and the regression loss weight of the positive sample to suppress the occurrence of the incorrect target detection result. The third determination module comprises a first determination submodule configured to determine a first class label of the positive sample according to an IOU between a prediction box corresponding to the positive sample and a ground truth box if the first type of false positive prediction box is targeted for suppression, and a second determination submodule configured to determine a second class label of the positive sample and a regression loss weight of the positive sample according to a true positive prediction box probability of the positive sample if the second type of false positive prediction box is targeted for suppression.

Images