Loss function acquisition method for small target detection

Abstract

The invention relates to a loss function acquisition method for small target detection, which comprises the following steps of: 1, performing mathematical modeling on a prediction frame and a real frame according to coordinate positions of the prediction frame and the real frame to generate a corresponding probability distribution model; 2, generating a probability density matrix corresponding to the corresponding discrete probability distribution based on the probability distribution model; 3, calculating the distance between the two matrixes in the step 2 based on a Sinkhorn algorithm; 4, normalizing the distance obtained through calculation in the step 3, and taking the normalized distance as an evaluation index of the prediction frame and the real frame; and 5, converting the evaluation index in the step 4 into a loss function for a target detection model. In order to solve the technical problem that an IOU-based neural network cannot be optimized when an intersection-to-union ratio-based IOU cannot calculate overlapping and no intersection in the existing IOU-based target detection technology, the invention provides a loss function acquisition method for small target detection.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, in particular to computer vision technology, and specifically to a method for obtaining a loss function for small target detection. Background Art

[0002] Object detection is the main research direction in the field of computer vision. With the development of convolutional neural networks over the past decade, the effect of object detection has been getting better and better. However, the detection of small objects still has performance problems in the fields of assisted driving, defect detection and remote sensing.

[0003] Most detectors will choose the IOU loss function and its variants when optimizing the prediction box. Traditional CVV uses L2Loss for pre-regression of bounding boxes, but since L2 loss directly optimizes the coordinates of the bounding box (in the form of xt, xb, xl, xr), it does not take into account that the coordinate parameters are highly correlated. UnitBox: An Advanced Object Detection Network introduces the Intersection over Union (IOU) loss function for bounding box prediction, which regresses the four boundaries of the prediction box as a whole unit. In the non-maximum suppression module, choosing the appropriate threshold of the IOU of positive and negative samples can enable the classifier to achieve better results in the classification and regression process. Generalized Intersection over Union: AMetric and A Loss for Bounding Box Regression was proposed to solve the problem that the gradient of IOU cannot be optimized when it is non-overlapping and the situation that IOU cannot represent overlapping bounding boxes. Faster and better learning for bounding box regression further considers the overlapping area, center point distance and aspect ratio in IOU and regularization terms. In this way, better convergence speed and detection performance can be obtained.

[0004] "Enhancing geometric factors in model learning and inference for object detection and instance segmentation" believes that the aspect ratio in CIOU is not appropriate as a separate regularization term, and the length and width should be optimized independently, so the aspect ratio loss is changed to a separate length loss and width loss. "A family of power intersection over union losses for bounding box regression" is based on the generalization of IOU Loss to a new Power IOU series Loss, which has a Power IOU term and an additional Power regularization term with a single Power parameter α. This new loss series is called αIOU Loss. By adjusting α, the detector has greater flexibility in achieving different levels of bbox regression accuracy. And α-IOU is more robust to small data sets and noise.

[0005] When optimizing with the Euclidean distance for regularization, the convergence speed is always unsatisfactory. SIOU loss: More powerful learning for bounding box regression chooses to introduce the angle regularization loss to quickly converge the distance between the predicted box and the formal box.

[0006] The IOU-based methods in the prior art can be unified into the following formula:

[0007]

[0008] Where R g and R p is the real box and the predicted box, R(R g ,R p ) is an additional penalty term. These formulas are all based on IOU, which inevitably introduces the shortcomings of IOU:

[0009] 1) The IOU based on intersection-over-union ratio cannot calculate the cases of overlap and no intersection. The former will make the IOU become a constant, and the latter will make the IOU 0. These two cases will make it impossible to optimize the neural network based on IOU;

[0010] 2) When calculating the intersection and union, and the intersection of two bounding boxes is constant, the larger the size of the bounding box, the smaller the proportion of the intersection to the bounding box itself, so the smaller the IOU. That is, the IOU is greatly affected by the size of the bounding box. Summary of the invention

[0011] The purpose of the present invention is to solve the technical problem that the existing IOU-based target detection technology cannot calculate overlap and no intersection when the IOU based on intersection-over-union ratio cannot calculate overlap and no intersection, which makes the IOU-based neural network unable to optimize, and proposes a method for obtaining the loss function for small target detection.

[0012] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0013] A method for obtaining a loss function for small target detection comprises the following steps:

[0014] Step 1: Generate a probability distribution model of the predicted box and the true box based on the coordinates of the predicted box and the true box;

[0015] Step 2: Generate the probability density matrix of the predicted box and the true box according to the probability distribution model of the predicted box and the true box;

[0016] Step 3: Calculate the distance between the two matrices in step 2 according to the Sinkhorn algorithm;

[0017] Step 4: Normalize the distance between the matrices calculated in step 3, and use the normalized distance as the evaluation index between the predicted box and the true box;

[0018] Step 5: Convert the evaluation index in step 4 into the loss function in the neural network.

[0019] In step 1, the area where the target box and the predicted box are located is regarded as part of a two-dimensional Gaussian distribution, and then a discrete Gaussian distribution is used to mathematically model the two boxes.

[0020] The specific steps include:

[0021] Step 1-1: For the bounding box in the image, we use the principle of mathematical probability to consider the possibility that each pixel in the image belongs to the target object. Then, based on the probability distribution of the pixels inside the bounding box belonging to the target, we model the horizontal and vertical dimensions of the image separately, thereby obtaining the probability distribution of the pixels belonging to the target in different dimensions.

[0022] Step 1-2: Analyze the probability distribution of pixels belonging to the target in different dimensions, so as to obtain the joint distribution of the probability of these pixels belonging to the target in each dimension.

[0023] In step 1-1, the general steps of pixels belonging to the target in each dimension are as follows:

[0024] (1) For any bounding box in an image, R = (c x ,cy ,w,h); where (c x ,c y ) represents the coordinates of the center point of the bounding box, and (w,h) represents the width and length of the bounding box;

[0025] (2) Calculate the Gaussian distribution corresponding to the bounding box from the horizontal dimension and the vertical dimension to obtain the horizontal Gaussian distribution X~N X (μ X ,∑ X ), longitudinal Gaussian distribution Y~N Y (μ Y ,∑ Y ), where μ X =x, μ Y =y,

[0026] In step 1-2, calculate the lateral Gaussian distribution X~N X (μ X ,Σ X ), longitudinal Gaussian distribution Y~N Y (μ Y ,∑ Y ), find the joint probability distribution R~N corresponding to the bounding box R in the image X,Y (μ (x,y) ,∑ (x,y) ), where μ (x,y) =[μ X ,μ Y ] T ,

[0027] In step 2, specifically:

[0028] 2-1: For any bounding box in an image, R = (c x ,c y ,w,h), intercept the matrix based on the center point of the bounding box:

[0029]

[0030] 2-2: According to the formula R~N X,Y (μ (x,y) ,∑ (x,y) ), bring each element in M(2w×2h) into R~N X,Y (μ (x,y) ,∑ (x,y) ), generate the corresponding probability density matrix C(2w×2h).

[0031] In step 3, when obtaining the difference distance between the probability density matrices obtained in step 2, the following steps are taken:

[0032] Step 3-1: Get the real box R in the picture gt =(cx gt ,cy gt ,w gt ,h gt ), prediction box

[0033] Step 3-2: Generate the probability density matrix of the predicted box according to step 2 The probability density matrix C of the real box gt (2w×2h);

[0034] Step 3-3: According to the Sinkhorn algorithm and formula Will With C gt Substituting (2w×2h), we get is the Sinkhorn distance between the two matrices; λ is the adjustment coefficient.

[0035] In step 4, the difference distance between the predicted box and the true box is normalized, including the following steps:

[0036] Step 4-1: Get the difference distance

[0037] Step 4-2: Get the average size C of all objects in the target dataset, where: Where M represents the number of images in a dataset, N i represents the number of bounding boxes marked in the i-th image, (w ij ,h ij ) represents the width and height of the jth bounding box in the ith image;

[0038] Step 4-3: Normalize using exponential form Get a new measure of normalized Sinkhorn distance in

[0039] In step 5, the distance measure obtained in step 4 is Will The indicator is designed as a loss function:

[0040] Compared with the prior art, the present invention has the following technical effects:

[0041] 1) The present invention first uses Gaussian probability distribution to perform a refined mathematical modeling of the bounding box of the target object in the image. The Sinkhorn distance is introduced as a calculation tool, and the obtained loss function can alleviate the rapid decline of IOU loss caused by target size during target detection;

[0042] 2) The invention is based on the flexibility and powerful expression ability of Gaussian distribution, which can capture the uncertainty of the position and size of the bounding box, replacing the 0-1 distribution. It can reduce the impact of excessive changes in the value of the loss function when the IOU series is offset for small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0044] Figure 1 It is the overall flow chart of the present invention;

[0045] Figure 2 Schematic diagram of IoU-deviation curve;

[0046] Figure 3 Schematic diagram of NSD-deviation curve. DETAILED DESCRIPTION

[0047] like Figure 1 As shown, a method for obtaining a loss function for small target detection includes the following steps:

[0048] Step 1: Generate a probability distribution model of the predicted box and the true box based on the coordinates of the predicted box and the true box;

[0049] Step 2: Generate the probability density matrix of the predicted box and the true box according to the probability distribution model of the predicted box and the true box;

[0050] Step 3: Calculate the distance between the two matrices in step 2 according to the Sinkhorn algorithm;

[0051] Step 4: Normalize the distance between the matrices calculated in step 3, and use the normalized distance as the evaluation index between the predicted box and the true box;

[0052] Step 5: Convert the evaluation index in step 4 into the loss function in the neural network.

[0053] In step 1, the area where the target box and the predicted box are located is regarded as part of a two-dimensional Gaussian distribution, and then a discrete Gaussian distribution is used to mathematically model the two boxes;

[0054] The specific steps include:

[0055] Step 1-1: Improve the distribution of the probability that the pixels in the bounding box belong to the target in the image, model the bounding box from the horizontal dimension and the vertical dimension respectively, and obtain the distribution of the probability that the pixels belong to the target in different dimensions in the image;

[0056] Step 1-2: Analyze the probability distribution of pixels belonging to the target in different dimensions to obtain the joint distribution of the probability of pixels belonging to the target in different dimensions.

[0057] Figure 2 The IoU-deviation curve is shown; wherein the horizontal axis coordinate value represents the number of pixel deviations between the center points of two bounding boxes of the same size, and the vertical axis value represents the corresponding metric value;

[0058] Depend on Figure 2 It can be seen that IoU is not suitable for small object detection. Figure 2 The IoU curves for different bounding box sizes are shown, each curve is drawn by keeping bounding box A fixed and moving bounding box B in the horizontal direction; it can be seen that the smaller the scale of the bounding box, the faster the curve drops. This feature means that a small change in the positional relationship between bounding boxes may cause a large change in the IoU value of a tiny object, and the use of IoU will reduce the performance of tiny object detection.

[0059] Figure 3 The NSD-deviation curve is shown; wherein the horizontal axis coordinate value represents the number of pixel deviations between the center points of two bounding boxes of the same size, and the vertical axis value represents the corresponding metric value;

[0060] Depend on Figure 3 The NSD curves for different bounding box sizes are shown. It can be seen that when bounding box B is far away from bounding box A, the NSD value decreases more slowly than the IoU, especially when the object size is very small. Therefore, using NSD in object detection can improve the detection quality.

[0061] In step 1-1, the joint distribution of the probability that a pixel belongs to the target in different dimensions is obtained; specifically, as follows:

[0062] (1) For any bounding box in an image, R = (c x ,c y ,w,h); where (c x ,c y ) represents the coordinates of the center point of the bounding box, and (w,h) represents the width and length of the bounding box;

[0063] (2) Calculate the Gaussian distribution corresponding to the bounding box from the horizontal dimension and the vertical dimension to obtain the horizontal Gaussian distribution X~N X (μ X ,∑ X), longitudinal Gaussian distribution Y~N Y (μ Y ,∑ Y ), where μ X =c x , μ Y =c y ,

[0064] In step 1-2, calculate the lateral Gaussian distribution X~N X (μ X ,∑ X ), longitudinal Gaussian distribution Y~N Y (μ Y ,∑ Y ), find the joint probability distribution R~N corresponding to the bounding box R in the image X,Y (μ (x,y) ,∑ (x,y) ), where μ (x,y) =[μ X ,μ Y ] T ,

[0065] In step 2, specifically:

[0066] 2-1: For any bounding box in an image, R = (c x ,c y ,w,h), intercept the matrix based on the center point of the bounding box:

[0067]

[0068] 2-2: According to the formula R~N X,Y (μ (x,y) ,∑ (x,y) ), bring the subscript of each element in M(2w×2h) into R~N X,Y (μ (x,y) ,∑ (x,y) ), generate the corresponding probability density matrix C(2w×2h).

[0069] In step 3, when obtaining the difference distance between the probability density matrices obtained in step 2, the following steps are taken:

[0070] Step 3-1: Get the real box R in the picture gt =(cx gt ,cy gt ,w gt ,h gt ), prediction box

[0071] Step 3-2: Generate the probability density matrix of the predicted box according to step 2 The probability density matrix C of the real box gt (2w×2h);

[0072] Step 3-3: According to the Sinkhorn algorithm and formula Will With C gt Substituting (2w · 2h) into the equation, we get is the Sinkhorn distance between the two matrices; λ is the adjustment coefficient.

[0073] In step 4, the difference distance between the predicted box and the true box is normalized, including the following steps:

[0074] Step 4-1: Get the difference distance

[0075] Step 4-2: Get the average size C of all objects in the target dataset, where:

[0076] Where M represents the number of images in a dataset, N i represents the number of bounding boxes marked in the i-th image, (w ij ,h ij ) represents the width and height of the jth bounding box in the ith image;

[0077] Step 4-3: Normalize using exponential form Get a new measure of normalized Sinkhorn distance

[0078] in

[0079] In step 5, the distance measure obtained in step 4 is Will The indicator is designed as a loss function:

[0080] Table 1 shows the NSD and IoU series of comparative experimental indicators, as follows:

[0081] Metric AP <![CDATA[AP 50 ]]> APvt APt APs DIOU 3.8 7.8 0.0 2.6 13.7 CIOU 4.0 8.2 0.0 2.3 12.6 GIOU 6.0 14.7 0.0 2.3 12.6 IOU 7.8 18.0 0.0 5.2 16.9 NSD 11.5 29.6 6.3 15.1 19.7

[0082] Table 1 NSD and IoU series comparison experimental indicators

[0083] From Table 1, we can see that the values ​​of IOU, GIOU, CIOU, and DIOU on APvt are 0, which means that in the scenario of detecting extremely small targets, the effect of using the traditional bounding box-based loss function for detection is almost zero;

[0084] The method proposed in this invention has achieved a significant improvement in APvt index compared with the IOU series.

[0085] At the same time, it can be found from Table 1 that the improvement of the three indicators Apt, APS, and APM is getting smaller and smaller; this shows that NSD has a huge advantage over the IOU series indicators in small target detection.

[0086] Table 2 shows the comparison data between NSD and main target detection methods, as follows:

[0087] Method Backbone AP <![CDATA[AP 50 ]]> <![CDATA[AP 75 ]]> <![CDATA[AP vt ]]> <![CDATA[AP t ]]> <![CDATA[AP s ]]> <![CDATA[AP m ]]> SSD-512 ResNet-50 7.1 21.82 2.6 1.0 4.7 11.5 13.5 TridentNet ResNet-50 7.6 20.8 3.4 1.0 5.6 12.5 14.2 FoveaBox ResNet-50 8.1 19.6 5.2. 2.7 9.1 12.9 14.4 PepPonits ResNet-50 9.2 23.6 5.2 2.6 9.2 13.1 14.5 FCOS ResNet-50 9.8 24.2 5.8 1.3 8.1 15.0 17.3 M-CenterNet ResNet-50 14.3 40.6 6.4 6.1 15.0 19.4 20.4 RetinaNet ResNet-50 4.7 13.6 2.01 2.0 5.3 6.4 7.6 ATSS ResNet-50 12.9 30.5 8.4 2.0 11.6 19.5 29.0 Fster RCNN ResNet-50 11.1 26.3 7.6 0.0 7.2 23.3 33.6 Csacade RCNN ResNet-50 13.8 30.8 10.4 0.0 10.5 25.5 36.5 DetectoRS ResNet-50 14.3 32.6 14.3 0.0 10.5 28.1 38.1 DCFL ResNet-50 25.4 58.9 17.9 10.5 24.3 35.4 40.3 SAHI ResNet-50 23.6 55.6 16.3 9.5 22.5 33.7 39.6 RetinaNet+NSD ResNet-50 8.2 24.9 5.0 3.3 9.8 13.4 16.8 ATSS+NSD ResNet-50 13.7 33.5 8.9 2.3 11.0 21.2 31.5 Fster RCNN+NSD ResNet-50 17.6 43.2 1.0 2.9 17.6 26.5 34.5 Csacade RCNN+NSD ResNet-50 18.6 44.3 13.0 3.5 17.6 26.9 25.8 DetectoRS+NSD ResNet-50 20.6 49.0 14.5 6.5 19.5 29.6 38.3 DCFL+NSD ResNet-50 27.9 60.5 25.6 11.2 26.4 38.5 39.6 SAHI+NSD ResNet-50 28.6 55.9 26.4 10.6 26.5 41.5 47.5

[0088] Table 2 Experimental indicators of NSD and main target detection methods

[0089] As can be seen from Table 2, FasterR-CNN, CascadeR-CNN, and DetectoRS have an APVT index of 0, but when IOU is replaced by GDM, the APvt index has increased significantly, which shows that NSD can improve the detection performance of some networks for small targets. DCFL and SAHI are currently the most advanced small target detection networks. After adding NSD, the evaluation index is still improved; in the experiment, DCFL obtains the optimal value of 60.5 in the AP0.5 index after adding NSD; DetectoRS, DCFL, and SAHI have the largest improvement in the three networks; DCFL's AP and AP0.75 have the largest improvement, which are 7.5 and 7.7; DetectoRS has the largest improvement in AP0.5, APVT, and APT, which are 16.2, 6.5, and 9; SAHI has the largest improvement in APS and APM, which are 7.8 and 7.9.

Claims

1. A method for obtaining a loss function for small target detection, characterized in that: It includes the following steps: Step 1: Generate a probability distribution model of the predicted box and the true box based on the coordinates of the predicted box and the true box; Step 2: Generate the probability density matrix of the predicted box and the true box according to the probability distribution model of the predicted box and the true box; Step 3: Calculate the distance between the two matrices in step 2 according to the Sinkhorn algorithm; Step 4: Normalize the distance between the matrices calculated in step 3, and use the normalized distance as the evaluation index between the predicted box and the true box; Step 5: Convert the evaluation index in step 4 into the loss function in the neural network.

2. The method according to claim 1, characterized in that In step 1, the area where the target box and the predicted box are located is regarded as part of a two-dimensional Gaussian distribution, and then a discrete Gaussian distribution is used to mathematically model the two boxes; The specific steps include: Step 1-1: Optimize the distribution probability of the target pixel in the bounding box of the image, model the bounding box from the horizontal dimension and the vertical dimension respectively, and obtain the distribution of the probability of the pixel belonging to the target in different dimensions in the image; Step 1-2: Analyze the probability distribution of pixels belonging to the target in different dimensions to obtain the joint distribution of the probability of pixels belonging to the target in different dimensions.

3. The method according to claim 2, characterized in that In step 1-1, the joint distribution of the probability that a pixel belongs to the target in different dimensions is obtained; specifically, as follows: (1) For any bounding box in an image, R = (c x ,c y ,w,h); where (c x ,c y ) represents the coordinates of the center point of the bounding box, and (w,h) represents the width and length of the bounding box; (2) Calculate the Gaussian distribution corresponding to the bounding box from the horizontal dimension and the vertical dimension to obtain the horizontal Gaussian distribution X~N X (μ X ,∑ X ), longitudinal Gaussian distribution Y~N Y (μ Y ,∑ Y ), where μ X =c x , μ Y =c y , 4. The method according to claim 2, characterized in that: In step 1-2, calculate the lateral Gaussian distribution X~N X (μ X ,∑ X ), longitudinal Gaussian distribution Y~N Y (μ Y ,∑ Y ), find the joint probability distribution R~N corresponding to the bounding box R in the image X,Y (μ (x,y) ,v (x,y) ), where μ (x,y) =[μ X ,μ Y ] T , 5. The method according to claim 1, characterized in that: In step 2, specifically: 2-1: For any bounding box in an image, R = (c x ,c y ,w,h), intercept the matrix based on the center point of the bounding box: 2-2: According to the formula R~N X,Y (μ (x,y) ,∑ (x,y) ), bring the subscript of each element in M(2w×2h) into R~N X,Y (μ (x,y) ,∑ (x,y) ), generate the corresponding probability density matrix C(2w×2h).

6. The method according to any one of claims 1 to 5, characterized in that: In step 3, when obtaining the difference distance between the probability density matrices obtained in step 2, the following steps are taken: Step 3-1: Get the real box R in the picture gt =(cx gt ,cy gt ,w gt ,h gt ), prediction box Step 3-2: Generate the probability density matrix of the predicted box according to step 2 The probability density matrix C of the real box gt (2w×2h); Step 3-3: According to the Sinkhorn algorithm and formula Will With C gt Substituting (2w×2h), we get is the Sinkhorn distance between the two matrices; λ is the adjustment coefficient.

7. The method according to claim 6, characterized in that In step 4, the difference distance between the predicted box and the true box is normalized, including the following steps: Step 4-1: Get the difference distance Step 4-2: Get the average size C of all objects in the target dataset, where: Where M represents the number of images in a dataset, N i represents the number of bounding boxes marked in the i-th image, (w ij ,h ij ) represents the width and height of the jth bounding box in the ith image; Step 4-3: Normalize using exponential form Get a new measure of normalized Sinkhorn distance in 8. The method according to claim 7, characterized in that In step 5, the distance measure obtained in step 4 is Will The indicator is designed as a loss function:

Images