A single-stage target detection regression position loss algorithm based on intersection over union

Abstract

A single-stage target detection regression position loss algorithm based on intersection over union is used to calculate the regression position loss between the predicted frame and the true value frame used in the training of the single-stage target detection algorithm, the intersection over union IoU of the predicted frame and the true value frame is first obtained, then the intersection over union is subtracted from the position relationship penalty term of the predicted frame and the true value frame, then the non-overlapping area penalty term of the predicted frame and the true value frame is subtracted, that is, the intersection over union RPIoU based on the regression position relationship sensitivity is obtained, and finally 1 is subtracted from RPIoU to obtain the regression position loss LOSS RPIOU The present application constructs the penalty term from the distance of the corner points of the predicted frame and the true value frame and the size of the non-overlapping area of the two frames, distinguishes the different containing relationship of the predicted frame and the true value frame, and finally obtains the single-stage target detection regression position loss based on the intersection over union, which overcomes the defect that the existing loss construction algorithm based on IoU cannot distinguish the different containing relationship of the predicted frame and the true value frame, and can accurately guide the position of the frame regression.

Description

Technical Field

[0001] The present invention relates to the field of single-stage target detection based on deep learning, and in particular to a single-stage target detection regression position loss algorithm based on intersection-over-union (IoU) ratio. Background Art

[0002] Deep learning-based object detection algorithms are mainly divided into two categories: two-stage detection algorithms and one-stage detection algorithms. Two-stage detection algorithms divide the detection problem into two stages: first generating candidate regions and then performing regression and classification on the candidate regions. Typical examples of two-stage detection algorithms are the R-CNN series of algorithms. While highly accurate, these algorithms suffer from high computational complexity, slow operation speed, and inability to meet real-time requirements. One-stage detection algorithms do not generate candidate regions. Instead, they directly input data into a convolutional neural network, predict the bounding box information of the object, and then discriminate these candidate bounding boxes, completing classification and regression in a single step. This results in faster detection speeds. Typical one-stage object detection algorithms include the YOLO algorithm and the SSD algorithm.

[0003] In single-stage object detection algorithms, the total loss function typically consists of three parts: classification loss, position loss, and confidence loss. Position loss plays a key role in optimizing speed and accuracy. Position loss typically includes mean absolute error loss (also known as L1-norm loss) and mean squared error loss (also known as L2-norm loss). However, L1 and L2-norm losses only consider the Euclidean distance between the four corners of the predicted and true boxes, without taking into account the positional relationship between the four corners of the two boxes.

[0004] The Intersection over Union (IoU) between the predicted and true bounding box is a practical metric for evaluating bounding box localization performance. When L1 and L2 are the same, the IoU between the two boxes can differ significantly. Therefore, IoU is used as the bounding box regression loss function to optimize the detection algorithm. However, when the true and predicted boxes have no intersection, IoU degenerates to zero, making training impossible. Therefore, improvements based on IoU are proposed, such as GIoU (Generalized IoU), DIoU (Distance IoU), CIoU (Complete IoU), and EIoU (Efficient IoU).

[0005] The above IoU loss construction only considers the cases where the predicted box and the true value box intersect, are completely separated, and partially overlap. However, inclusion situations often occur in target detection, that is, the predicted box contains the true value box or the true box contains the predicted box. The existing IoU-based loss construction algorithm will produce the incorrect result that the position loss is equal when the true box and the predicted box are in different inclusion relationships. As a result, it cannot be applied to the case where the true box and the predicted box have an inclusion relationship, and cannot guide the regression position of single-stage target detection. Summary of the Invention

[0006] The purpose of this invention is to provide a single-stage object detection regression position loss algorithm based on intersection over union (IoU) to improve the sensitivity of the bounding box regression loss based on IoU to changes in the position relationship of the bounding boxes, and accurately distinguish the position relationship between the predicted box and the true value box.

[0007] The technical solution adopted by the present invention to solve the above technical problems is: a single-stage target detection regression position loss algorithm based on intersection-over-union (IoU) is used to calculate the regression position loss between the prediction box and the true value box used in the training of the single-stage target detection algorithm. The IoU of the prediction box and the true value box is first obtained, and then the IoU is subtracted from the positional relationship penalty term of the prediction box and the true value box. Then, the non-overlapping area penalty term of the predicted box and the true value box is subtracted to obtain the intersection-over-union (RPIoU) based on the sensitivity of the regression position relationship. Finally, the regression position loss LOSS is obtained by subtracting RPIoU from 1. RPIOU , the formula is:

[0008] (1);

[0009] (2);

[0010] In formula (2), is the prediction box, is the true value box;

[0011] (2);

[0012] In formula (3), d is the penalty parameter of the non-overlapping area, r is the judgment parameter, when the predicted box and the true value box are in the included state, r=1, and in other states r=0;

[0013] (4);

[0014] (5);

[0015] (6);

[0016] In formula (5) and (6), is the coordinate of the upper left corner of the prediction box, is the coordinate of the upper left corner of the true value box, is the coordinate of the lower right corner of the prediction box, is the coordinate of the lower right corner of the true value box, and c is the diagonal length of the minimum bounding rectangle that covers both the prediction box and the true value box;

[0017] (7);

[0018] In formula (7), is the area of ​​the truth box, is the area of ​​the prediction box.

[0019] According to the above technical solution, the beneficial effects of the present invention are:

[0020] The present invention constructs penalty terms from two aspects: the distance between the corner points of the predicted box and the true value box and the area size of the non-overlapping area of ​​the two boxes. The intersection-over-union ratio of the predicted box and the true value box is first subtracted from the position relationship penalty term to characterize the position relationship between the two boxes and guide the corner points of the two boxes to be infinitely close; then, when the predicted box and the true value box are in an inclusion state, that is, r=1, the non-overlapping area penalty term is further subtracted, thereby distinguishing the different inclusion relationships between the predicted box and the true value box, calculating the intersection-over-union ratio based on the regression position relationship sensitivity and finally obtaining the regression position loss, overcoming the problem that the existing regression position loss construction algorithm based on IoU is not applicable to the situation where the true value box and the predicted box have an inclusion relationship, and can more accurately guide the position of the frame regression. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a series calculation of IoU of different and same center points;

[0022] Figure 2 It is a schematic diagram of predictive stepwise regression;

[0023] Figure 3 This is a comparison chart of the detection results of different algorithms on the VisDrone dataset;

[0024] Figure 4 This is a comparison chart of the detection results of different algorithms in the infrared aircraft dataset. DETAILED DESCRIPTION

[0025] With reference to the accompanying drawings, the specific implementation is as follows:

[0026] A single-stage object detection regression position loss algorithm based on intersection-over-union (IoU) is used to calculate the regression position loss between the predicted box and the true value box used in the training of the single-stage object detection algorithm.

[0027] First, we calculate the Intersection over Union (IoU) of the predicted box and the true value box, and then we subtract the positional relationship penalty term between the predicted box and the true value box from the IoU. Then, the non-overlapping area penalty term of the predicted box and the true value box is subtracted to obtain the intersection-over-union (RPIoU) based on the regression position relationship sensitivity. Finally, the regression position loss LOSS is obtained by subtracting RPIoU from 1. RPIOU .

[0028] Existing classic losses based on IoU include GIoU (Generalized IoU), DIoU (Distance IoU), CIoU (Complete IoU) and EIoU (Efficient IoU). The above IoU loss construction only considers the intersection, separation and partial simple overlap of the predicted box and the true value box. However, in object detection, especially small object detection, inclusion often occurs, that is, the predicted box contains the true value box or the true box contains the predicted box.

[0029] like Figure 1 As shown, the solid line is the prediction box and the dotted line is the true value box. Figure 1 The left side of (a) and (b) are the inner envelope of the predicted frame completely wrapped by the true value frame. Figure 1 The right sides of (a) and (b) are the outer envelopes where the predicted box completely wraps the true value box. Figure 1 (a) The area of ​​the left truth box is GTarea=1, and the area of ​​the prediction box is BBarea=0.8. Figure 1 The area of ​​the true value box on the right side of (a) is GTarea=1, and the area of ​​the predicted box is BBarea=1.25, so Figure 1 The area difference between the two boxes on the left side of (a) is 0.2, and the area difference between the two boxes on the right side is 0.25. Figure 1 (b) The area of ​​the left truth box is GTarea=2.25, and the area of ​​the predicted box is BBarea=1. Figure 1 The area of ​​the true value box on the right side of (b) is GTarea=2.25, and the area of ​​the predicted box is BBarea=5.0625, so Figure 1 The area difference between the two boxes on the left side of (b) is 1.25, and the area difference between the two boxes on the right side is 2.8125.

[0030] From this we can see that in Figure 1 In the two cases (a) and (b), the area difference on the left and the area difference on the right are different, so Figure 1 (a) left side compared to right side, and Figure 1The regression effect of the left side of (b) is different from that of the right side. However, after calculating IoU, GIoU, DIoU, CIoU and EIoU, IoU, GIoU, DIoU, CIoU and EIoU are different. Figure 1 The results of the left and right sides of (a) are the same. Figure 1 The calculation results on the left and right sides of (b) are the same, but only the Regression Position Relationship Sensitivity IoU (RPIoU) based on the regression position relationship sensitivity given by the present invention can produce different results, thus Figure 1 The different inclusion relationships between the prediction boxes and the true value boxes on the left and right sides in (a) and (b) are used to distinguish them.

[0031] RPIoU is calculated using equations (2) to (7):

[0032] (2);

[0033] In formula (2), is the prediction box, is the true value box.

[0034] (2);

[0035] In formula (3), d is the penalty parameter of the non-overlapping area, r is the decision parameter, when the predicted box and the true value box are in the included state, r = 1, and r = 0 in other states.

[0036] (4);

[0037] (5);

[0038] (6);

[0039] In formula (5) and (6), is the coordinate of the upper left corner of the prediction box, is the coordinate of the upper left corner of the true value box, is the coordinate of the lower right corner of the prediction box, is the coordinate of the lower right corner of the true value box, and c is the diagonal length of the minimum bounding rectangle that covers both the prediction box and the true value box.

[0040] (7);

[0041] In formula (7), is the area of ​​the truth box, is the area of ​​the prediction box.

[0042] After calculating RPIoU, LOSS is obtained by formula (1) RPIOU :

[0043] (1).

[0044] When the predicted box and the true value box have no inclusion relationship, the penalty term for the area difference of the non-overlapping area of ​​the two boxes in RPIoU is no longer called, but it still exists The penalty is imposed by using the relationship between the corner points of the two boxes, that is, the corner points of the predicted box are closer to the corner points of the true value box. The greater the difference between the regression positions of the predicted box and the true value box, The larger the value, the smaller the RPIoU, the greater the loss value, and the faster the convergence speed. In the above construction, when the center points of the two boxes coincide but the two boxes have not yet overlapped, It can help distinguish the current regression state and make the loss converge better.

[0045] The target detection effect of the present invention is verified through simulation experiments:

[0046] The experimental hardware platform was Windows 10, with an Intel(R) Core(TM) i7-10700 CPU at 2.90 GHz and 32 GB of RAM. The GPU was an NVIDIA 3090 with 8 GB of video memory and 24 GB of RAM. The software platform used the PyTorch deep learning framework, Python 3.8, and CUDA 11.3.1 and CUDNN 8.2.1 for GPU acceleration.

[0047] Experimental validation was conducted using two datasets: a publicly available visible light dataset and a custom infrared dataset. The visible light dataset was obtained by mixing and regrouping the publicly available VisDrone2019 and VisDrone2020 datasets. The dataset contains 14,038 static images collected by drones of various models in various scenarios and under various weather and lighting conditions, encompassing over 2.6 million urban traffic ground objects, such as pedestrians, cars, bicycles, and tricycles.

[0048] The experimental data for the self-made infrared dataset is derived partly from infrared aircraft images and partly from augmented images using rotation and translation operations, resulting in a total of 5,582 frames. A 10% random partition of the infrared aircraft dataset was used as the test set. The remaining data was divided into training and validation sets using a cross-hatch method with a ratio of 8:2. The infrared aircraft dataset is annotated according to the following categories, resulting in a total of six: Back Attitude Fuselage (BAF), Back Attitude Tailflame (BAT), Lateral Attitude Fuselage (LAF), Lateral Attitude Tailflame (LAT), Backward Attitude Fuselage (BWF), and Backward Attitude Tailflame (BWT).

[0049] The experiment uses YOLOv5 as the benchmark network to verify whether the loss function proposed in this invention has better effects in actual detection. The specific experimental design is as follows. First, only the effectiveness of RPIoU is verified: the IoU part in YOLOv5 is replaced by GIoU, DIoU, CIoU, EIoU and RPIoU respectively, and the other network structures remain unchanged, and are all run in the same experimental environment; then the experimental results are compared with other classic algorithms. In order to verify the universality of the method proposed in this invention, the above experimental verification is carried out on the visible light dataset and the infrared dataset respectively. The image size is uniformly set to 640×640, the batchsize is set to 16, the entire network uses the SGD optimizer, the weight decay is set to 0.0005, the learning rate is 0.01, and a total of 100 generations of training are performed.

[0050] 1. Visible Light Dataset Experiment

[0051] To verify the effectiveness of RPIoU, we conducted comparative experiments on the VisDrone dataset using YOLOv5 as the base algorithm, using GIoU, DIoU, CIoU, EIoU, and RPIoU. Table 1 shows the mAP of each algorithm at IoU thresholds of 0.5 and 0.5:0.95. Table 2 shows the AP of each algorithm in each category at an IoU threshold of 0.5.

[0052] Table 1 Experimental results of VisDrone dataset

[0053]

[0054] Table 2 Detection accuracy of each category in the VisDrone dataset

[0055]

[0056] From Table 1 and Table 2, we can see that:

[0057] (1) When the IoU threshold is 0.5, the mAP using RPIoU is the highest, which is 2.6%, 1.9%, 1.9%, and 2.4% higher than IoU, DIoU, CIoU, and EIoU, respectively.

[0058] (2) When the IoU threshold is 0.5:0.95, RPIoU achieves the best performance among the comparison algorithms, with mAP reaching 24.6%, which is 0.8%, 0.7%, 0.7%, and 1.3% higher than GIoU, DIoU, CIoU, and EIoU.

[0059] In Table 1, RPIoU achieves the highest accuracy at both an mAP threshold of 0.5 and an average value between 0.5 and 0.95, achieving optimal performance. Table 2 shows that RPIoU outperforms other IoU detection methods across all categories, with particularly significant improvements for van, truck, and bus detection. This demonstrates that improvements to RPIoU enable more effective position regression and improve detection accuracy.

[0060] Figure 2 This is a diagram of predictive stepwise regression. The bounding box with the word "truck" is the predicted bounding box, and the other is the ground-truth bounding box. As can be seen, RPIoU achieves better convergence at the same number of iterations. After 10 iterations, the algorithm using RPIoU detects the object and establishes a containment relationship with the ground-truth bounding box, while the algorithm using CIoU fails to detect it. After 10 iterations, the algorithm using CIoU detects the object with a confidence level of only approximately 0.3, and establishes a containment relationship with the ground-truth bounding box. In contrast, the predicted bounding box using the RPIoU algorithm approaches approximately 0.7. At the 80th iteration, the algorithm using RPIoU has essentially converged, reaching a confidence level of 0.84, while the confidence level using CIoU is only 0.6. Compared to CIoU, the improved RPIoU approach allows for better convergence and accurately determines the positional relationship between the predicted and ground-truth bounding boxes.

[0061] The improved loss proposed in this paper was introduced into YOLOv5 and compared with the following algorithms: Faster-RCNN, SSD, Centernet, EfficientDet, and YOLOv3. The experimental results are shown in Table 3.

[0062] Table 3 Experimental results of different target detection algorithms on the VisDrone dataset

[0063]

[0064] As shown in Table 3, the improved YOLOv5 based on our invention performs best among all the compared detection algorithms, reaching a maximum accuracy of 42.9%, significantly higher than the other detection algorithms. Compared to the original YOLOv5 algorithm, our algorithm improves detection accuracy by 1.9% and also achieves the highest FPS compared to other algorithms, meeting real-time requirements.

[0065] The YOLOv5 target detection algorithm with improved loss proposed in this paper is compared with some classic detection algorithms on the VisDrone dataset. Some of the visual detection results are shown in Figure 2. Figure 2 As shown in the figure. Comparing the first set of dense crowd images, Centernet, SSD, and EfficientDet detect fewer people. Observation reveals that Centernet and SSD detect mostly objects in the center of the image, while EfficientDet detects mostly objects at the edges. In contrast, the YOLO series detects the most objects in the image, while the improved YOLOv5 with the present invention achieves the highest accuracy, detecting all but the crowded crowd in the upper left corner. Comparing the second set of ground vehicle images in a darker environment, SSD fails to detect any objects, and EfficientDet only detects one. While Centernet detects many objects, it's far fewer than the YOLO series. The improved detection results for densely populated areas with small objects outperform YOLOv3, reducing missed detections. Comparing the third set of ground vehicle images in a brighter environment, Centernet and EfficientDet only detect the largest vehicle and the small car at the top. SSD detects an additional bicycle, but both methods detect fewer objects than the YOLO series. However, compared to the present invention, some missed detections still occur, such as the vehicle in the lower right corner.

[0066] 2. Infrared Dataset Experiment

[0067] Based on the YOLOv5 algorithm, GIoU, DIoU, CIoU, EIoU and RPIoU were used to conduct comparative experiments on the infrared aircraft dataset. The results are shown in Table 4.

[0068] Table 4 Experimental results of infrared aircraft dataset

[0069]

[0070] From Table 4 we can see that:

[0071] (1) When the IoU threshold is 0.5, the mAP using RPIoU is 1.0%, 0.7%, 0.3%, and 1.0% higher than that of GIoU, DIoU, CIoU, and EIoU, respectively.

[0072] (2) When the IoU threshold is 0.5:0.95, RPIoU achieves the best mAP among the compared algorithms, reaching 60.6%, which is consistent with DIoU and CIoU, and higher than GIoU and EIoU.

[0073] Therefore, the RPIoU proposed in this invention achieves the highest detection accuracy at the mAP threshold of 0.5 and the average value of 0.5 to 0.95, achieving the best performance effect.

[0074] The improved loss proposed in this invention was introduced into YOLOv5 and compared with the following classic algorithms: Faster-RCNN, SSD, Centernet, EfficientDet, and YOLOv3. The experimental results are shown in Table 5. It can be seen that the improvement proposed in this invention performs best among all the compared detection algorithms, reaching the highest 96.6%, significantly higher than the other detection algorithms. In terms of real-time performance, it can be seen that the FPS is also the highest compared to other algorithms, which can meet the real-time requirements. The above experiments prove that the present invention is still feasible on infrared datasets. This loss function can meet the needs of different scenarios and has high accuracy and robustness.

[0075] Table 5 Experimental results of different target detection algorithms on infrared aircraft dataset

[0076]

[0077] The improved loss proposed in this paper is introduced into YOLOv5 and compared with some classic detection algorithms on the infrared dataset. Some of the visual detection results are shown in the figure. Figure 4 The first set of comparison images shows aircraft with tail flames facing away from the sky, obscured by clouds. All algorithms can detect the targets. SSD and Centernet have a confidence level of less than 0.8, while the confidence levels of the targets detected by the other algorithms are all above 0.8, with the improved algorithm of the present invention having the highest confidence level. The second set of images shows a group of aircraft with tail flames facing away from the sky, with neither EfficientDet nor Centernet detecting any targets. SSD only detects one target, while the YOLO series detects all targets. The improved algorithm of the present invention has the highest confidence level, exceeding 0.8. The third set of images shows a group of aircraft with sideways fuselages, with Centernet having the worst detection effect, detecting only one target. SSD and EfficientDet detect most targets, but their confidence levels are all below 0.8, while the YOLO series detects most targets with confidence levels exceeding 0.8.

[0078] In summary, this paper constructs a penalty term based on the distance between the corner points of the predicted and true boxes, as well as the size of the non-overlapping regions between the two boxes. The proposed regression positional relationship sensitivity loss based on the intersection-over-union ratio can better guide bounding box regression. This paper has been extensively experimentally validated on both visible light and infrared datasets, demonstrating its general applicability.

Claims

1. A single-stage object detection regression position loss algorithm based on intersection-over-union (IoU) is used to calculate the regression position loss between the predicted box and the true value box used in the training of the single-stage object detection algorithm. The characteristics are: First, calculate the intersection over union (IoU) of the predicted box and the true value box, and then subtract the positional relationship penalty term between the predicted box and the true value box from the IoU. Then, the non-overlapping area penalty term of the predicted box and the true value box is subtracted to obtain the intersection-over-union (RPIoU) based on the sensitivity of the regression position relationship. Finally, the regression position loss LOSS is obtained by subtracting RPIoU from 1. RPIOU , the formula is: （1）； （2）； In formula (2), is the prediction box, is the true value box; （2）； In formula (3), d is the penalty parameter of the non-overlapping area, r is the judgment parameter, when the predicted box and the true value box are in the included state, r=1, and in other states r=0; （4）； （5）； （6）； In formula (5) and (6), is the coordinate of the upper left corner of the prediction box, is the coordinate of the upper left corner of the true value box, is the coordinate of the lower right corner of the prediction box, is the coordinate of the lower right corner of the true value box, and c is the diagonal length of the minimum bounding rectangle that covers both the prediction box and the true value box; （7）； In formula (7), is the area of ​​the truth box, is the area of ​​the prediction box.

Images