An optimization method for target detection models

Abstract

The present invention relates to a method for optimizing an object detection model, comprising: 1) finding a grid for predicting a true box during model training; 2) calculating a regression loss (mSIoU) for the predicted box; 3) allocating positive and negative samples according to an IoU (Intersection over Union) threshold to establish a positive sample candidate set; calculating different positive and negative sample weights for predicted boxes inside and outside the positive sample candidate set, and finally obtaining a classification loss; 4) adding the regression loss and the classification loss to obtain a final loss, and optimizing the model through backpropagation using an optimizer. Positive and negative samples are allocated according to an IoU (Intersection over Union) threshold to establish a positive sample candidate set; calculating different positive and negative sample weights for predicted boxes inside and outside the candidate set, and finally obtaining a classification loss; adding the regression loss and the classification loss to obtain a final loss, and optimizing the model through backpropagation using an optimizer.

Description

Technical Field

[0001] The present invention relates to a target detection model optimization technology, belonging to the field of deep learning target detection technology. Background Art

[0002] With the development of deep learning, object detection technology in the field of computer vision has become increasingly mature. By reading an image or capturing a video using a camera, a detection model can identify all objects of interest in the image and determine their locations. The detection model outputs a large number of prediction boxes, which contain three types of information: location, confidence, and classification probability. During training, the detection model is optimized by distinguishing positive and negative samples, constructing a loss function, and performing backpropagation optimization.

[0003] SIoU (SCYLLA-IoU) is a loss function with stronger learning capabilities for bounding box regression. Traditional GIOU and CIOU: Traditional regression box losses primarily consider three aspects: the overlap area between the two boxes, the distance between the two boxes, and the aspect ratio of the two boxes. Most algorithms express IOU (Intersection over Union), which is calculated by calculating the ratio of the intersection and union (sum) between the ground truth box and the predicted box.

[0004] The loss function is primarily divided into three parts: position, confidence, and classification probability. The position loss primarily uses the Intersection over Union (IoU) loss, while the confidence and classification probability loss primarily use the cross-entropy loss. Many current object detection model optimizations combine confidence and classification probability. IoU suffers from the drawbacks of not accurately reflecting the overlap between the predicted and ground-truth boxes, and also fails to reflect the distance between them when there is no overlap. Generalized Intersection over Union (GIoU) ​​incorporates the smallest box enclosing the predicted and ground-truth boxes, Distance Intersection over Union (DIoU) incorporates distance information, and Complete Intersection over Union (CIoU) incorporates aspect ratio, or shape information. The newly proposed SIoU (SCYLLA-IoU) incorporates angular information to accelerate model convergence. SIoU does not significantly improve model performance, and there is room for further improvement in accelerating model convergence.

[0005] Distinguishing between positive and negative samples, also known as label assignment, is divided into hard and soft assignment methods. Hard assignment assumes that each predicted box can only be positive or negative, with weights of 0 and 1, respectively. This method treats all samples equally, ignoring the fact that samples are not equally important. Soft assignment addresses this inequality by treating samples differently. Existing methods primarily focus on designing weights for positive samples, while negative sample weights are derived solely from them and are highly correlated with them. These weights provide little new supervision and may limit the learning ability of the detection model. "A Dual Weighting Label Assignment Scheme for Object Detection," published at CVPR 2022 (IEEE Conference on Computer Vision and Pattern Recognition 2022), proposes a dual-weighted label assignment method that assigns weights to positive and negative samples from different perspectives, effectively distinguishing the importance of different predicted boxes to an object. Dual-weighted label assignment constructs a set of candidate positive samples for each ground-truth box. Predicted boxes outside this set are considered negative samples and are not included in the weighting function design process. When the IoU is large, the weight of the negative sample in the candidate set is small, or even not optimized. When the positive sample score is small, the weight of the positive sample in the candidate set is small. This will slow down the convergence of the model and is unreasonable. Summary of the Invention

[0006] The technical problems to be solved and the goals to be achieved by the present invention are as follows: SIoU in the target detection model optimization method adds angle information to accelerate model convergence, but SIoU has almost no improvement on model performance; in order to improve the consistency of prediction box regression and classification, the dual-weighted label assignment method specifies the weights of positive and negative samples from different angles. However, when the IoU is large, the weight of the negative sample is small or even not optimized. When the positive sample score is small, the weight of the positive sample is small, which will slow down the model convergence speed and is unreasonable.

[0007] The present invention is used to solve the problem that SIoU has almost no improvement on the performance of the target detection model, and further accelerates the model convergence based on SIoU. At the same time, it solves the problem that the double-weighted label assignment method does not optimize the negative sample component of the candidate set when the IoU is large, and optimizes the positive sample component of the candidate set slowly when the positive sample score is small.

[0008] The technical solution of the present invention is: a target detection model optimization method, comprising the following steps:

[0009] 1) When training the object detection model, find the grid of the predicted ground truth box, 2) calculate the predicted box regression loss mSIoU (Modified-SCYLLA-IoU), 3) assign positive and negative samples based on the IoU (Intersection over Union) threshold to establish a candidate set of positive samples;

[0010] For the prediction boxes inside and outside the positive sample candidate set, different positive and negative sample weights are calculated, and finally the classification loss is obtained;

[0011] 4) Add the regression loss and classification loss to get the final loss, and use the optimizer to backpropagate and optimize the model.

[0012] 1) Find the grid of the predicted true box: Based on the large amount of predicted box information output by the detection model, calculate which grid the center of the true box falls within, then find the other three grids closest to the center point, and consider these four grids that form a square as the predicted true box;

[0013] 2) Calculate the prediction box regression loss function mSIoU: mSIoU consists of four cost functions, which optimize angle, distance, shape, and intersection over union (IoU) respectively;

[0014] The optimization angle: optimize the predicted box B and the real box B gt The smaller angle between the center point line and the x-axis or y-axis is min{α,β} to 0; optimize the area around B and B gt The angle between the diagonal of the minimum rectangle γ and B gt The corresponding diagonal angles θ are equal;

[0015] The angle cost is:

[0016] Optimize distance: optimize the predicted box B and the real box B gt The distance c from the center point in the x direction w and wrap B, B gt The minimum rectangular width d w The ratio is 0;

[0017] Optimize the predicted box B and the real box B gt The distance c from the center point in the y direction h and wrap B, B gt The minimum rectangular height d h The ratio is 0;

[0018] The distance cost is:

[0019] Optimize shape: Optimize the width w of the predicted box B and the real box B gt Width w gtThe difference between the two is greater than max{w, w gt} is 0; optimize the height h of the predicted box B and the real box B gt High h gt The difference between the two is greater than max{h, h gt} is 0; optimize the predicted box B and the real box B gt The aspect ratio is equal;

[0020] Shape cost is:

[0021] mSIoU consists of four cost functions: mSIoU = IoU-(Δ+Ω) / 2, and the prediction box regression loss, i.e., IoU-Loss, is L box =1-IoU+(Δ+Ω) / 2.

[0022] Among them: 3) Allocate positive and negative samples and establish a positive sample candidate set:

[0023] For all predicted boxes and real boxes in the grid, mSIoU is calculated again, and those greater than a certain threshold are added to the positive sample candidate set;

[0024] If both are less than the threshold, select the prediction box with the largest IoU to ensure that each true box is assigned at least one prediction box;

[0025] All samples outside the candidate set are considered negative samples;

[0026] For the prediction boxes inside and outside the candidate set, different positive and negative sample weights are calculated, including:

[0027] The predicted boxes in the positive sample candidate set, whether positive sample weight or negative sample weight, should be positively correlated with IoU and negatively correlated with the classification score;

[0028] All prediction boxes outside the candidate set are regarded as negative samples, with only negative sample weights and no positive sample weights;

[0029] The weight of negative samples should be negatively correlated with IoU and negatively correlated with the classification score;

[0030] Prediction box in the positive sample candidate set: For positive and negative sample weights t pos , t neg , define a consistency measure respectively, where t pos =(1-s) λ *IoU μ , t neg =s λ *IoU μ, s is the classification score, λ and μ are used to balance the two conditions of IoU and classification score; λ is generally set to 1, and μ is set to 5 for better results;

[0031] Adding an exponential modulation factor promotes the weight distance and obtains the weights of positive and negative samples in the candidate set respectively: v is a hyperparameter that controls the relative gap between different weights, and a value of 5 or 6 works well.

[0032] Prediction box outside the candidate set: set a negative sample probability P neg , get the weight of negative samples outside the candidate set: w neg2 =P neg *s ξ ,ξ is a factor of how much priority should be given to negative samples, and a value of 2 is better.

[0033] Get the classification loss:

[0034] Loss within the candidate set:

[0035] Loss outside the candidate set:

[0036] The final classification loss L cls It is formed by adding the losses inside and outside the candidate set:

[0037] N is the number of samples in the positive sample candidate set, and M is the number of negative samples outside the candidate set.

[0038] Add the regression loss and classification loss: Final loss: L = L cls +μL box .

[0039] Regression loss L box

[0040] μ is a measure of consistency between positive and negative samples defined by t pos , t neg The same balance factor is used, and 5 is the best.

[0041] Beneficial effect: The present invention proposes to optimize the diagonal angle γ of the minimum rectangle that encloses the real box and the predicted box, which is equal to the diagonal angle θ corresponding to the real box. On the basis of adding single angle information to SIoU, the model convergence is further accelerated. At the same time, on the basis of SIoU optimization of the width and height of the predicted box and the real box, the aspect ratio information of the two boxes is added again, which can accelerate convergence and improve the performance of the target detection model.

[0042] The present invention proposes that the prediction boxes in the positive sample candidate set, both the positive sample weight and the negative sample weight should be positively correlated with the IoU and negatively correlated with the classification score, to solve the problem that the dual-weighted label assignment method does not optimize the negative sample component of the candidate set when the IoU is large, and optimizes the positive sample component of the candidate set slowly when the positive sample score is small; for the prediction boxes outside the candidate set, all are regarded as negative samples, with only negative sample weights but no positive sample weights. The negative sample weights are negatively correlated with the IoU and negatively correlated with the classification score, which accelerates model convergence and can improve the performance of the target detection model.

[0043] This invention is a development of target detection technology in the field of computer vision, and has good application prospects. It optimizes the model to make the model performance better and accelerate convergence, reduces the model training time and algorithm development time under big data deep learning, and puts the algorithm into practical application more quickly. It has good applications in the fields of face detection, license plate detection, industrial defect detection, etc. The detection frame positioning is more precise and the classification results are more accurate. It has good performance in dense target scenes and scenes with close targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Optimize the target detection model of the present invention.

[0045] Figure 2 Various parameters that need to be optimized when training target detection models and optimizing prediction box regression;

[0046] Figure 3 The loss function descent curve for using different IoU as regression loss when training the target detection model. DETAILED DESCRIPTION

[0047] The present invention is an optimization method for a target detection model, which is mainly used for distinguishing positive and negative samples of model output results, calculating loss functions, and back-propagating the optimization model during target detection model training.

[0048] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0049] See Figure 1 As shown, in the embodiment of the present invention, the process of the target detection model optimization process is as follows:

[0050] During training, a set of positive candidate samples is created for each ground-truth bounding box. Based on the large amount of predicted bounding boxes output by the detection model, the grid in which the center of the ground-truth bounding box falls is calculated. The three grids closest to the center point are then found. These four grids forming a square are considered the predicted ground-truth bounding box.

[0051] The mSIoU (mean squared inter-domain over union) is calculated for all predicted boxes in the grid and the ground-truth boxes. Those with an IoU greater than a certain threshold are added to the positive candidate set. If all boxes are less than the threshold, the predicted box with the largest IoU is selected to ensure that each ground-truth box is assigned at least one predicted box for prediction. All boxes outside the candidate set are considered negative. Using mSIoU for IoU, only samples in the positive candidate set need to be calculated for IoU-loss; negative samples outside the candidate set only need to be calculated for classification loss.

[0052] For the large amount of predicted boxes output by the detection model, the center of the ground-truth box is calculated in which grid cell. The three grid cells closest to the center cell are then identified. These four grid cells, forming a square, are considered the predicted ground-truth box. The mean square intersection over (MIoU) is calculated for all predicted boxes in the grid cell and the ground-truth box. Those with a mean square intersection over (MIoU) greater than a certain threshold are added to the candidate set of positive samples. If all predicted boxes have a mean square intersection over (MIoU) less than the threshold, the predicted box with the largest IoU is selected to ensure that each ground-truth box is assigned at least one predicted box. All boxes outside the candidate set are considered negative samples.

[0053] mSIoU is still composed of 4 cost functions: angle, distance, shape, and IoU. Figure 2 , α and β are the predicted box B and the real box B respectively gt The angle between the center point and the x-axis and y-axis is, if α≤π / 4, that is, α≤β, when convergence, first optimize α, otherwise minimize β. At the same time, γ is the angle between B and B. gt The diagonal angle of the minimum rectangle, θ is B gt The corresponding diagonal angle, optimizing γ and θ to be equal, speeds up convergence, and the angle cost is

[0054] c w 、c h is the predicted box B and the real box B gt The distance between the center point in the x and y directions, d w d h It wraps around B, B gt Minimum rectangle width and height, optimize c w with d w The ratio is 0, optimizing c h with d h The ratio is 0, and the distance cost is

[0055] w、h、w gt 、h gt They are the predicted box B and the real box B gt Optimize the width and height of w and w gt The difference between the two and the larger of max{w, w gt} is 0, optimize h and h gt The difference between the two and the larger one max{h, h gt} is 0, optimizing the predicted box B and the real box B gt The aspect ratio is equal. The shape cost is The value of η is unique and controls the degree of attention paid to shape cost.

[0056] The final mSIoU is mSIoU = IoU-(Δ+Ω) / 2, and the prediction box regression loss, i.e., IoU-Loss, is L box =1-IoU+ξΔ+Ω) / 2. Figure 3 , the detection model is retrained on a new dataset with the same category. The two curves are the loss decline curves when mSIoU and SIoU are used as regression losses. mSIoU-Loss decreases faster than SIoU-Loss, and the model converges faster.

[0057] In order to prevent negative samples with small IoU from still having high classification scores, it is necessary to improve the consistency of prediction box regression and classification. The prediction boxes in the positive sample candidate set, both the positive sample weight and the negative sample weight should be positively correlated with IoU. The larger the IoU, the closer the classification score of the true box of the prediction box should be to 1, and the scores of other classifications should be closer to 0. The weights of positive and negative samples should be negatively correlated with the classification score. The smaller the classification score of the true box, the larger the positive sample weight, the larger the scores of other classifications, and the larger the negative sample weight. This can speed up the convergence of the model and solve the problem of not optimizing the negative sample component of the candidate set when the IoU is large. For the weights of positive and negative samples, a consistency measure t is defined, t pos =(1-s) λ *IoU μ , t neg =s λ *IoU μ , λ is generally set to 1, and μ is set to 5 for better results; adding an exponential modulation factor promotes the weights to widen the distance, and the weights of positive and negative samples in the candidate set are obtained respectively: ξ is a factor that determines how much priority should be given to negative samples, and a value of 2 is better.

[0058] Calculate the classification loss for the samples in the positive sample candidate set, and use the positive sample weight w for the category to which the real box belongs pos , use negative sample weights w for the remaining categories neg1 ; Calculate the classification loss for negative samples outside the candidate set, and use the negative sample weight w for all classifications neg2 .

[0059] The classification loss and regression loss are added together to obtain the final loss function. The optimizer Adam (Adaptive Moment Estimation) or SGD (Stochastic Gradient Descent) is used to calculate the gradient backpropagation to optimize the target detection model.

[0060] All prediction boxes outside the candidate set are considered negative samples, with only negative sample weights and no positive sample weights. The negative sample weights should be negatively correlated with the IoU. The smaller the IoU, the closer all classification scores are to 0. The negative sample weights should be negatively correlated with the classification scores. The larger the classification scores, the larger the negative sample weights, which speeds up the model convergence. Set a negative sample probability P neg The larger the IoU, the smaller the probability, and the smaller the IoU, the greater the probability. Weight of negative samples outside the candidate set: w neg2 =P neg *s ξ ,ξ is a factor of how much priority should be given to negative samples, and a value of 2 is better.

[0061] The final classification loss: N is the number of samples in the candidate set of positive samples, M is the number of negative samples outside the candidate set, and the final loss is: L = L cls +μL box , μ is a measure of consistency with the definition of positive and negative samples t pos , t neg The same balance factor is used, and 5 is the best.

Claims

1. A target detection model optimization method, characterized in that: The steps include: 1) Find the grid of predicted real boxes during target detection training; 2) Calculate the prediction box regression loss mSIoU; 3) Assign positive and negative samples according to the intersection-over-union (IoU) threshold and establish a candidate set of positive samples; For the prediction boxes inside and outside the positive sample candidate set, different positive and negative sample weights are calculated, and finally the classification loss is obtained; 4) Add the regression loss and classification loss to get the final loss, and use the optimizer to backpropagate the optimization model; Specifically: In step 1), find the grid of the predicted true box: for the large amount of predicted box information output by the detection model, calculate which grid the center point of the true box falls in, and then find the other three grids closest to the center point. These four grids that form a square are considered to be the predicted true box; In step 2), the prediction box regression loss function mSIoU is calculated: mSIoU consists of four cost functions, which optimize angle, distance, shape and intersection over union (IoU) respectively; The optimization angle: optimize the predicted box B and the real box B gt The smaller angle between the center point line and the x-axis or y-axis is min{α,β} to 0; optimize the area around B and B gt The angle between the diagonal of the minimum rectangle γ and B gt The corresponding diagonal angles θ are equal; The angle cost is: Optimize distance: optimize the predicted box B and the real box B gt The distance c from the center point in the x direction w and wrap B, B gt The minimum rectangular width d w The ratio is 0; Optimize the predicted box B and the real box B gt The distance c from the center point in the y direction h and wrap B, B gt The minimum rectangular height d h The ratio is 0; The distance cost is: Optimize shape: Optimize the width w of the predicted box B and the real box B gt Width w gt The difference between the two is the larger of max{w,w gt } is 0; optimize the height h of the predicted box B and the real box B gt High h gt The difference between the two is the larger of max{h,h gt } is 0; optimize the predicted box B and the real box B gt The aspect ratio is equal; Shape cost is: mSIoU consists of four cost functions: mSIoU = IoU-(Δ+Ω) / 2, and the prediction box regression loss, i.e., IoU-Loss, is L box =1-IoU+(Δ+Ω) / 2.

2. The target detection model optimization method according to claim 1, characterized in that: in: In step 3), positive and negative samples are assigned to establish a candidate set of positive samples: For all predicted boxes and real boxes in the grid, mSIoU is calculated again, and those greater than a certain threshold are added to the positive sample candidate set; If both are less than the threshold, select the prediction box with the largest IoU to ensure that each true box is assigned at least one prediction box; All samples outside the candidate set are considered negative samples; For the prediction boxes inside and outside the candidate set, different positive and negative sample weights are calculated, including: The predicted boxes in the positive sample candidate set, whether positive sample weight or negative sample weight, should be positively correlated with IoU and negatively correlated with the classification score; All prediction boxes outside the candidate set are regarded as negative samples, with only negative sample weights and no positive sample weights; The weight of negative samples is negatively correlated with IoU and negatively correlated with the classification score; Prediction box in the positive sample candidate set: For positive and negative sample weights t pos , t neg , define a consistency measure respectively, where t pos =(1-s) λ *IoU μ ,t neg =s λ *IoU μ , s is the classification score, λ and μ are used to balance the two conditions of IoU and classification score; Adding an exponential modulation factor promotes the weight distance and obtains the weights of positive and negative samples in the candidate set respectively: ν is a hyperparameter that controls the relative gap between different weights; Prediction box outside the candidate set: set a negative sample probability P neg , get the weight of negative samples outside the candidate set: w neg2 =P neg *s ξ ,ξ is a factor of how much priority should be given to negative samples; Get the classification loss: Loss within the candidate set: Loss outside the candidate set: The final classification loss L cls It is formed by adding the losses inside and outside the candidate set: N is the number of samples in the positive sample candidate set, and M is the number of negative samples outside the candidate set; Add the regression loss and classification loss: Final loss: L = L cls +μL box ; regression loss is L box ; μ is a measure of consistency between positive and negative samples defined by t pos , t neg The same balance factor.

3. The target detection model optimization method according to claim 2, characterized in that: λ is set to 1.

4. The target detection model optimization method according to claim 2, wherein: ξ is a factor that gives priority to negative samples and its value is 2.

5. The target detection model optimization method according to claim 2, wherein: μ is a measure of consistency between positive and negative samples defined by t pos , t neg The same balance factor is used when , and both are 5; ν is a hyperparameter that controls the relative gap between different weights and is 5 or 6.

Images