A single-stage target detection classification loss algorithm with moderate attention to easy negative samples

Abstract

The application provides a single-stage target detection classification loss algorithm for moderately paying attention to easy negative samples, wherein a modulation parameter is added to a classical Focal Loss to control the weight of difficult negative sample loss and easy negative sample loss in total loss, that is, in the training process of the detection algorithm, while paying attention to difficult negative samples, the weight of easy negative samples in the total loss is increased, so that the detection algorithm can learn the feature information of the target from the easy negative samples.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a single-stage target detection classification loss algorithm that appropriately focuses on negatively oriented samples. Background Technology

[0002] Deep learning-based object detection algorithms are mainly divided into two categories: two-stage and one-stage algorithms. In one-stage object detection algorithms, the total loss function typically consists of three parts: classification loss, location loss, and confidence loss. The classification loss is the sum of probabilities for each class in the bounding box, the location loss is calculated using corner coordinates and area information, and the confidence loss determines whether an object exists in the bounding box. Object detection often suffers from imbalanced positive and negative samples and problems with easy and difficult samples. In classification loss, the majority of the loss is for easy negative samples, which has almost no effect on the convergence of the detection algorithm. A common solution is to use Focal Loss. It focuses the training sample attention of the detection model on difficult samples, reducing the weight of easy samples and increasing the weight of difficult samples, because these easy samples are mostly related to background information. Focal Loss and its improved methods mainly focus on reducing the weight of easy samples and increasing the weight of difficult samples, but the detection model still focuses the vast majority of its attention on difficult samples.

[0003] Under existing positive and negative sample partitioning strategies, the number of positive samples is far less than the number of negative samples, and the number of easy samples is far greater than the number of hard samples. Different easy negative samples may contain different target information, and these easy negative samples containing target features can provide the detection algorithm with more learnable target features. However, Focal Loss focuses too much on hard negative samples, neglecting the learning of easy negative samples containing target information, resulting in insufficient extraction of target features by the detection algorithm. Therefore, it is necessary to appropriately increase the attention given to easy negative samples during training to more fully learn target features from the samples. YOLOv5 is used as an example.

[0004] Example 1: Based on infrared aircraft images, visualize the positive and negative samples and easy and difficult samples during model training, such as... Figure 1 As shown, Figure 1 The image on the left shows the target's coordinates in the original image. Figure 1 The image on the right is a magnified view of the target region in the original image, mapped from the predicted bounding boxes of the target grid in each output layer of YOLOv5. (For example...) Figure 2 As shown, Figure 2 It is a local magnified view of the predicted bounding boxes of the target mesh in the low-resolution two-layer structure mapped to the high-resolution output layer. From Figure 1 , 2As can be observed, there are 3 positive samples and 6 negative samples at the target grid. Furthermore, the prediction confidence of all negative samples is less than 0.5, indicating that these 6 negative samples are easily negative samples containing information from different parts of the target. Therefore, paying appropriate attention to easily negative samples containing target features can enable the detection model to learn more target information.

[0005] Example 2: Based on visible light aerial images, visualize the positive and negative samples and easy and difficult samples of a single pedestrian target during model training, such as... Figure 3 As shown. Figure 3 The image on the left shows the target's coordinates in the original image. Figure 3 The image on the right is a magnified view of the target region in the original image, mapped from the predicted bounding boxes of the target grid in each output layer of YOLOv5. Figure 4 It is a local magnified view of the predicted bounding boxes of the target mesh in the low-resolution two-layer structure mapped to the high-resolution output layer. From Figure 3 , 4 It can be observed that there are 3 positive samples and 6 negative samples at the target grid. At the same time, the prediction confidence of the negative samples is less than 0.5. All 6 negative samples are easy negative samples containing target information. Therefore, we can draw a similar conclusion to that of infrared images, that is, we need to pay appropriate attention to easy negative samples containing target features. Summary of the Invention

[0006] The purpose of this invention is to provide a single-stage target detection classification loss algorithm that moderately focuses on negatively susceptible samples. By adding moduloable parameters to the classic Focal Loss, the weights of the loss from difficult-to-suppress samples and the loss from negatively susceptible samples in the total loss are controlled. That is, during the training process of the detection algorithm, while focusing on difficult-to-suppress samples, the weight of negatively susceptible samples in the total loss is increased, so that the detection algorithm can learn the feature information of the target from the negatively susceptible samples.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: a single-stage target detection classification loss algorithm that moderately focuses on negatively oriented samples, comprising the following steps;

[0008] Step 1: Collect static images to obtain the training image dataset;

[0009] Step 2: Introduce modulation parameters to construct a classification loss function and apply it to the single-stage object detection algorithm model. The classification loss function is:

[0010]

[0011] Where y = 1 represents the true label, p represents the predicted probability of label y, θ is modulation parameter 1, and γ is modulation parameter 2;

[0012] Step 3: Input the training image dataset into the single-stage object detection algorithm model for training. Use the SGD stochastic gradient descent algorithm as the model optimizer and use the genetic algorithm to find the optimal solution of hyperparameters, modulation parameter 1θ and modulation parameter 2γ.

[0013] Step 4: Use the trained single-stage object detection algorithm model to perform object detection on the real-time input image.

[0014] Preferably, step 3, which uses a genetic algorithm to find the optimal hyperparameter modulation parameters, includes:

[0015] Step 3.1: Initialize the hyperparameters and encode them in decimal, and set the number of evolution iterations n;

[0016] Step 3.2: Calculate the fitness value of individuals in the i-th generation population. The fitness function is:

[0017] f(x1,x2)=0.9×x a +0.1×x b

[0018] In the formula: x a Indicates mAP: 0.5, x b This indicates mAP: 0.5-0.95;

[0019] Step 3.3: If the number of iterations is less than the maximum number of generations, continue the genetic iteration. If the number of iterations equals the maximum number of generations, obtain the individual corresponding to the optimal fitness value in the current population and end the genetic iteration.

[0020] Step 3.4: Sort all individuals in the current population in ascending order, select the individual with the lowest fitness value for mutation, and calculate the fitness value of the mutated individual. The mutated individual and other individuals are directly inherited into the next generation of the population.

[0021] Step 3.5, repeat steps 3.3 and 3.4 until the termination condition of step 3.3 is met, then end the genetic iteration.

[0022] Beneficial effects: This invention adjusts the proportion of hard-to-fail sample loss and easy-to-fail sample loss in the total loss by adding a modulation factor to Focal Loss. While focusing on learning hard-to-fail samples, it makes full use of the target characteristics contained in some easy-to-fail samples, helping the feature extraction network learn more meaningful target features. Attached Figure Description

[0023] Figure 1 This is a magnified view of the target coordinates and the target area in the original image of Example 1 of the present invention.

[0024] Figure 2This is a magnified view of a portion of Example 1 of the present invention mapped onto the high-resolution output layer.

[0025] Figure 3 This is a magnified view of the target coordinates and target area in the original image of Example 2 of the present invention.

[0026] Figure 4 This is a magnified view of a portion of Example 2 of the present invention mapped onto the high-resolution output layer.

[0027] Figure 5 This is a graph showing the trend of f3 when θ = 0.2 and different values ​​of γ.

[0028] Figure 6 This is a graph showing the trend of f4 when θ = 0.2 and different values ​​of γ.

[0029] Figure 7 This is a graph showing the trend of f4 when γ = 2 and different values ​​of θ.

[0030] Figure 8 This is a visualization of the CE test results.

[0031] Figure 9 This is a visualization of the Focal Loss detection results.

[0032] Figure 10 This is a visualization of the QFocal Loss detection results.

[0033] Figure 11 This is a visualization of the Poly Loss detection results.

[0034] Figure 12 This is a visualization of the Proposed Loss detection results. Detailed Implementation

[0035] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0036] The structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0037] It should be noted that any parts not described in detail in this application are prior art.

[0038] In Focal Loss, for negative samples, when the prediction probability p1∈[0,0.5] for easy samples and the prediction probability p2∈(0.5,1] for hard samples, the ratio of the modulation factor of easy negative samples to the modulation factor of hard negative samples in Focal Loss is f1, as shown in formula (3):

[0039]

[0040] To increase the weight of easily negative samples in the total loss, this invention introduces a parameter θ to construct a new modulation factor, thereby improving the target detection algorithm's focus on easily negative samples. After introducing the modulation parameter, the ratio of the modulation factor of easily negative samples to the modulation factor of hard-negative samples in Focal Loss becomes f2, as shown in formula (4):

[0041]

[0042] The difference between f2 and f1 is shown in formula (5):

[0043]

[0044] In equation (5), since p2 is always greater than p1, f2 is always greater than f1. Therefore, introducing the parameter θ can increase the weight of negatively sensitive samples in the total loss, thereby increasing the attention of the target detection algorithm to negatively sensitive samples.

[0045] Therefore, this invention introduces a new modulation parameter θ to construct a new classification loss function, as shown in formula (6):

[0046]

[0047] The modulation parameter θ in the formula can increase the proportion of the loss of the negative sample in the total loss.

[0048] To prove that the modulation parameter γ can increase the proportion of hard sample loss in the total loss, assume that p1+p2=1, p1∈[0,0.5] and θ remains unchanged, then the ratios of the modulation factor of easy samples to the modulation factor of hard samples in positive and negative samples are f3 and f4, respectively, as shown in formulas (7) and (8):

[0049]

[0050]

[0051] To visually observe the changing trends of f3 and f4 under different γ values, a graph was drawn as follows: Figure 5The graph shown in Figure 6 illustrates this. It can be seen that as γ increases, the values ​​of f3 and f4 both decrease. This indicates that the proportion of hard sample loss in the total loss is increasing. For example, ① when γ = 2, the prediction probability of hard positive samples... The predicted probability of the positive sample When γ = 2 and θ = 0.2, the value of f3 is approximately 0.18; when γ = 2 and θ = 0.2, the predicted probability of negatively susceptible samples is... Predicted probability of hard-to-bear samples When γ = 3, the value of f4 is 0.308. ② When γ = 3, the prediction probability of the difficult positive sample... The predicted probability of the positive sample When γ = 3 and θ = 0.2, f3 is 0.078; when γ = 3 and θ = 0.2, the predicted probability of negatively susceptible samples is... Predicted probability of hard-to-bear samples At that time, the value of f4 was 0.17.

[0052] To visually observe the changing trend of the ratio of the modulation factor of the easy-to-negative sample to that of the hard-to-negative sample under different θ values, assuming p1+p2=1, p1∈[0,0.5] and γ remains constant, the ratio of the modulation factor of the easy-to-negative sample to that of the hard-to-negative sample is f4, as shown in formula (8). The ratio curve of the modulation factor of the easy-to-negative sample to that of the hard-to-negative sample is shown in... Figure 7 As shown.

[0053] from Figure 7 It can be seen that when γ is constant, the ratio f4 of the modulation factor of easy-to-fail samples to that of hard-to-fail samples increases with the increase of θ. For example, when p = 0.2, γ = 2, and θ = 0.2, the value of f4 is 0.16; when the values ​​of p and γ remain unchanged, and θ = 0.4, the value of f4 is 0.25. This indicates that θ can narrow the gap between the proportion of easy-to-fail sample and hard-to-fail sample losses in the total loss, allowing easy-to-fail samples containing target information to contribute more to the target detection algorithm. Meanwhile, the prediction probability of easy-to-fail samples is always less than that of hard-to-fail samples, so the ratio f4 of the modulation factor of easy-to-fail samples to that of hard-to-fail samples is always less than or equal to 1. Therefore, the modulation factor of hard-to-fail samples is greater than that of easy-to-fail samples, indicating that the detection algorithm still focuses more on hard samples.

[0054] Therefore, controlling θ can adjust the weight of hard negative sample loss and easy negative sample loss in the total sample loss. That is, while emphasizing training with hard negative samples, we can appropriately increase the attention to easy negative samples to better adjust the influence of hard negative samples and easy negative samples on the loss function, and help the network learn more meaningful features from negative samples containing target features.

[0055] This invention is used as the classification loss in the single-stage object detection algorithm YOLOv5, and a genetic algorithm is used to optimize the two hyperparameters θ and γ in the designed classification loss. The steps are as follows:

[0056] (1) Initialize the hyperparameters and encode them in decimal, and set the number of evolution iterations n.

[0057] (2) Calculate the fitness value of individuals in the i-th generation population. The fitness function is shown in equation (9):

[0058] f(x1,x2)=0.9×x a +0.1×x b (9)

[0059] In the formula: x a Indicates mAP: 0.5, x b This indicates mAP: 0.5-0.95.

[0060] (3) If the number of iterations is less than the maximum number of generations, continue the genetic iteration. If the number of iterations is equal to the maximum number of generations, obtain the individual corresponding to the best fitness value in the current population and end the genetic iteration.

[0061] (4) Sort all individuals in the current population in ascending order, select the individual with the lowest fitness value for mutation, and calculate the fitness value of the mutated individual. The mutated individual and other individuals are directly inherited into the next generation of the population.

[0062] (5) Repeat (3) and (4) until the termination condition of step (3) is met, and end the genetic iteration.

[0063] The target detection effect of the present invention was verified through simulation experiments:

[0064] Experimental hardware platform: Windows 10 operating system, Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz CPU, 32GB RAM. GPU: NVIDIA 3090 with 8GB VRAM, 24GB RAM; Software platform: PyTorch deep learning framework, Python 3.8 programming language, with CUDA 11.3.1 and CUDNN 8.2.1 used to accelerate the GPU.

[0065] 1. Experimental dataset and parameters

[0066] The visible light dataset was obtained by mixing and regrouping the VisDrone 2019 and VisDrone 2020 public datasets collected by the AISKYEYE team at Tianjin University. The dataset contains 14,038 still images, collected by various drone models in different scenes and under various weather and lighting conditions, including over 2.6 million urban traffic ground targets (such as pedestrians, vans, buses, trucks, tricycles, etc.). 10% of the dataset was randomly partitioned into a test set, and the remaining data was divided into training and validation sets in an 8:2 ratio using a cross-multiplication method.

[0067] During model training, the SGD stochastic gradient descent algorithm was used as the model optimizer, with an initial learning rate of 0.01, momentum factor of 0.937, and weight decay of 0.0005. When using a genetic algorithm to find the optimal hyperparameters, the hyperparameters for improving the classification loss were initialized to θ = 0.5, γ = 1, and the number of iterations was 20. The optimal hyperparameter solution found by the genetic algorithm was θ = 0.9, γ = 2.

[0068] 2. Comparison of different classification loss algorithms

[0069] (1) Cross Entropy Loss

[0070] In object detection, cross-entropy loss is a commonly used loss function to measure the difference between the model's prediction and the true label. In classification tasks, cross-entropy loss is used to measure the difference between the model's prediction for each object category and the true label. For each object, the model generates a confidence score, representing the probability that it belongs to each possible category. Cross-entropy loss compares the probability distribution of the model's output with the probability distribution of the true label and calculates the difference between them. This allows the model to correctly assign high confidence scores to the correct categories. By minimizing cross-entropy loss, an object detection model can be trained to accurately classify object categories in an image, as shown in Equation (10):

[0071]

[0072] In the formula, y = 1 represents the true label, and p represents the predicted probability of label y.

[0073] (2) Focal Loss

[0074] In single-stage object detection algorithms, the imbalance between positive and negative samples is a prominent issue, with a large proportion of easy samples contributing to the total loss. The strategy for dividing easy and difficult samples is as follows: for positive samples, a predicted bounding box with a confidence score less than 0.5 is considered a difficult positive sample; a predicted bounding box with a confidence score greater than or equal to 0.5 is considered an easy positive sample. For negative samples, a predicted bounding box with a confidence score less than 0.5 is considered an easy negative sample; a predicted bounding box with a confidence score greater than or equal to 0.5 is considered a difficult negative sample.

[0075] Since the number of easy samples far exceeds the number of difficult samples in training, the easy sample loss dominates the model's classification loss training, which leads to low training efficiency and model degradation. Focal Loss was proposed to solve the problem of imbalanced sample classes and to make the model pay more attention to difficult-classified samples during training, thereby achieving the accuracy of a two-stage detection algorithm without affecting the original speed of a single-object detection algorithm. The calculation formula of Focal Loss is shown in formula (12):

[0076]

[0077] In the formula, y = 1 represents the true label, p represents the predicted probability of label y, and the modulation parameter γ is used to adjust the contribution of easy and difficult samples to the loss function.

[0078] (3) Quality Focal Loss

[0079] Since Focal Loss only supports discrete category labels such as 0 and 1, it is powerless for smooth labels (0~1). Therefore, Quality Focal Loss (QFL) is derived, as shown in formula (12):

[0080] QFL(p,y)=|yp| γ *[(1-y)log(1-p)+ylog(p)] (12)

[0081] In the formula, y represents the label (0~1) after smoothing, p represents the predicted probability of label y, and the modulation parameter γ is used to adjust the contribution of easy and difficult samples to the loss function.

[0082] (4) Poly Loss

[0083] Cross-entropy loss and Focal loss are the most common choices when training deep neural networks to handle classification problems. However, in general, a good loss function can take a more flexible form and should be customized for different tasks and datasets. Inspired by the Taylor expansion approximation function, Poly Loss treats and designs the loss function as a linear combination of polynomial functions. Poly Loss allows for easy adjustment of the importance of different polynomial bases according to the target task and dataset, while treating cross-entropy loss and focus loss as special cases. The formula for calculating Poly Loss for Focal Loss is shown in Equation (13):

[0084]

[0085] In the formula, y = 1 represents the true label, p represents the predicted probability of label y, the modulation parameter γ is used to adjust the contribution of easy and difficult samples to the loss function, N represents the number of polynomials, and ε represents the correlation coefficient of the corresponding polynomials.

[0086] The algorithm of this invention was applied to the single-stage object detection algorithm YOLOv5 and compared with the above four classification loss algorithms. The detection results of different algorithms on VisDrone are shown in Table 1.

[0087] Table 1. Detection accuracy and mean accuracy for each category in the VisDrone dataset.

[0088]

[0089] As can be seen from Table 1:

[0090] (1) The classification loss algorithm of this invention achieves 41.5% on the evaluation index mAP:0.5, which is 0.8%, 9.6%, 15.6% and 10.6% higher than CE, FocalLoss, QFocal Loss and Poly Loss, respectively.

[0091] (2) The classification loss algorithm of this invention has lower detection accuracy than CE in the categories of pedestrian and tricycle, but the highest detection accuracy in the remaining categories.

[0092] Visualized detection results and magnified local plots of different classification loss algorithms are shown below. Figure 8 , 9 As shown in figures 10, 11, and 12, it can be seen that:

[0093] (1) In Figure 8In (a) to 12(a), when the truck target in the upper left corner is occluded, only the algorithm of this invention and the PolyLoss classification loss algorithm can correctly detect the target position, while the others cannot correctly detect the target position.

[0094] (2) In Figure 8 In (b) to 12(b), when the target is in a narrow space, only the algorithm of this invention can correctly classify bicycles and electric vehicles in the image.

[0095] (3) In Figure 8 Of the algorithms in (c) to 12(c), only the algorithm of this invention can correctly detect the type and location of cyclists at traffic intersections. Other algorithms can locate the target, but cannot correctly classify it.

[0096] In summary, this invention, while focusing on training with difficult-to-fail samples, appropriately increases the attention given to easily-failed samples. Applying the invented classification loss to a single-stage object detection algorithm enables the algorithm to learn more target features from easily-failed samples, thus improving the algorithm's classification ability. This invention has been thoroughly experimentally verified on a visible light aerial photography dataset.

[0097] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. A single-stage object detection classification loss algorithm that appropriately considers negatively susceptible samples, characterized in that: Includes the following steps; Step 1: Collect static images to obtain the training image dataset; Step 2: Introduce modulation parameters to construct a classification loss function and apply it to the single-stage object detection algorithm model. The classification loss function is: Where y = 1 represents the true label, p represents the predicted probability of label y, θ is modulation parameter 1, and γ is modulation parameter 2; Step 3: Input the training image dataset into the single-stage object detection algorithm model for training. Use the SGD stochastic gradient descent algorithm as the model optimizer and use the genetic algorithm to find the optimal solution of hyperparameters, modulation parameter 1θ and modulation parameter 2γ. Step 4: Use the trained single-stage object detection algorithm model to perform object detection on the real-time input image.

2. The single-stage target detection classification loss algorithm with moderate attention to negatively oriented samples as described in claim 1, characterized in that: Step 3, which uses a genetic algorithm to find the optimal solution for the hyperparameter modulation parameters, includes the following steps: Step 3.1: Initialize the hyperparameters and encode them in decimal, and set the number of evolution iterations n; Step 3.2: Calculate the fitness value of individuals in the i-th generation population. The fitness function is: f(x1,x2)=0.9×x a +0.1×x b In the formula: x a Indicates mAP: 0.5, x b This indicates mAP: 0.5-0.95; Step 3.3: If the number of iterations is less than the maximum number of generations, continue the genetic iteration; if the number of iterations is equal to the maximum number of generations, obtain the individual corresponding to the optimal fitness value in the current population and end the genetic iteration. Step 3.4: Sort all individuals in the current population in ascending order, select the individual with the smallest fitness value for mutation, and calculate the fitness value of the mutated individual. The mutated individual and other individuals are directly inherited into the next generation of the population. Step 3.5, repeat steps 3.3 and 3.4 until the termination condition of step 3.3 is met, then end the genetic iteration.

Images