Anchor-free object detection method using quadratic IoU loss function

Abstract

The present invention provides an anchor-free target detection method using a quadratic IoU loss function, which is characterized by comprising: preprocessing an image to be detected and then inputting the pre-trained target detection model for inference to obtain a corresponding image detection result, wherein the target detection model is pre-trained by: obtaining a training image dataset and supervision information; constructing an initial target detection model, and inputting the training image dataset and supervision information into the model; using a quadratic IoU loss function to calculate positioning loss, and using a Focal Loss loss function to calculate classification loss; respectively using the positioning loss and the classification loss to derive model parameters of the initial target detection model, and then using back propagation to update the model parameters; judging whether the updated model parameters meet the termination condition, and if so, entering the next step, otherwise entering the second step of the training process; saving the updated model parameters and loading them to obtain the target detection model.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to an anchor-free target detection method using a quadratic IoU loss function. Background Art

[0002] With the rapid development of the manufacturing and electronic information industries, smart devices have rapidly grown, and various electronic devices have become part of people's lives. The widespread use of electronic devices has generated a massive amount of image data, which contains a wealth of useful information. Manually processing this vast amount of image data and filtering out the useful information is extremely time-consuming and laborious. Therefore, humans are designing algorithms that machines can understand and execute to filter out interesting information, such as face detection algorithms that can detect faces in images. Designing efficient and high-performance algorithms for processing image-related data is a key research topic in the field of computer vision.

[0003] Object detection algorithms, the foundation of many advanced computer vision algorithms, have been successfully applied to all aspects of our lives, such as face detection and recognition, text recognition, and security. Before 2018, mainstream object detection algorithms were almost all anchor-frame-based. The deployment of commercial applications required extensive code for anchor frame operations, such as anchor frame generation, matching, encoding, and decoding, which increased the difficulty of deployment. Today, anchor-frame-free object detection algorithms are gaining popularity, accelerating their implementation and making a growing number of everyday scenarios more convenient with the help of these algorithms.

[0004] In recent years, anchor-free object detection algorithms have made tremendous progress. Researchers have invented various types of anchor-free object detection algorithms and created many new methods to improve object detection performance. Object detection tasks require both classifying and localizing the categories of interest in an image. Researchers have made many improvements to the classification and localization loss functions to improve the detection performance of the models.

[0005] After years of research, the main positioning loss functions for object detection include Smooth L1 Loss, IoULoss, Linear IoU Loss, GIoU Loss, DIoU Loss, and CIoU Loss. These loss functions are used in different object detection algorithms, and the IoU Loss series is primarily used in anchor-free object detection, achieving excellent performance. Typically, the positioning quality of object detection is expressed using IoU, which is the intersection of the areas of two rectangular boxes divided by the union of their areas. Because Smooth L1 Loss optimizes the transformations of the center point coordinates x, y, and the length, width, and h of the rectangular box, the authors of IoU Loss believe that Smooth L1 Loss breaks the relationship between the four variables of the rectangular box. Therefore, IoU Loss uses the negative logarithm of IoU to measure the positioning loss, with positioning quality as the optimization objective. Linear IoU Loss uses a linear IoU loss function with a constant derivative to avoid the problem of undefined derivatives when IoU is 0, which can cause the model to fail to optimize. GIoU Loss introduces the concept of closure regions, allowing the model to optimize the target based on the intersection of the predicted and target boxes, improving the model's positioning performance. DIoU Loss and CIoU Loss use the length, width, and center distance of the predicted and target boxes as regularization terms to accelerate model convergence and improve model performance.

[0006] Although GIoU Loss, DIoU Loss, and CIoU Loss have achieved excellent results on a few object detection algorithm models, they only improve mAP by 0.2% to 0.3% on anchor-free object detection algorithms such as FCOS. Because these IoU Loss methods do not consider the overall positioning quality (IoU) during parameter updates to adjust the parameter update step size, the model cannot fully learn the location of the object during training, resulting in a lack of significant improvement in detection results. Summary of the Invention

[0007] To solve the above problems, a method for anchor-free object detection using a quadratic IoU loss function is provided. The present invention adopts the following technical solutions:

[0008] The present invention provides a method for anchor-free target detection using a quadratic IoU loss function, which is characterized by comprising: step S1-1, preprocessing an image to be detected to obtain a preprocessed image; step S1-2, inputting the preprocessed image into a pre-trained target detection model for inference to obtain a corresponding image detection result, wherein the target detection model is pre-trained by the following steps: step S2-1, obtaining a training image dataset and corresponding supervision information, the supervision information including positioning supervision information and category supervision information; step S2-2, constructing an initial target detection model, and inputting the training image dataset and the supervision information into the initial target detection model, the initial target model having a feature extraction module, a positioning module and a classification module, the feature extraction module is used to extract a fixed-scale feature map according to the training image dataset as an output feature map, the positioning module is used to process the output feature map to obtain the predicted box position information of each point on the output feature map, the classification module is used to process the output feature map to obtain the confidence of each point on the output feature map corresponding to all categories, step S2-3, using the quadratic IoU loss function to calculate the positioning loss according to the predicted box position information and the positioning supervision information, and using Focal The Loss loss function calculates the classification loss based on the confidence and category supervision information; in step S2-4, the model parameters of the initial target detection model are derived using the positioning loss and the classification loss respectively, and then the model parameters are updated using back propagation to obtain the updated model parameters; in step S2-5, it is determined whether the updated model parameters meet the termination conditions. If so, step S2-6 is entered, otherwise step S2-2 is entered; in step S2-6, the updated model parameters are saved and loaded into the initial target detection model to form a target detection model.

[0009] The present invention provides a method for anchor-free object detection using a quadratic IoU loss function, which may also have the following technical features:

[0010] Loss loc =λ-αIoU-βIoU 2

[0011] Where λ, α, and β are balance factors, which are constants and β≠0. IoU is the intersection over union (IoU) of the predicted box and the supervised target box, and its value range is [0,1].

[0012] The present invention provides a method for anchor-free target detection using a quadratic IoU loss function, which may also have a technical feature in that when the model parameters of the initial target detection model are derived using the positioning loss and the classification loss, the derivative function of the positioning loss includes an IoU term, and the form of the derivative function is as follows:

[0013] Loss′loc =-α-2βIoU

[0014] Where α and β are balance factors, which are constants and β≠0. IoU is the intersection over union (IoU) of the predicted box and the supervised target box, and its value range is [0,1].

[0015] The present invention provides an anchor-free target detection method using a quadratic IoU loss function, which may also have a technical feature in which the feature extraction module includes at least one or more combinations of ResNet, DenseNet, and ResNet+FPN.

[0016] The present invention provides an anchor-free target detection method using a quadratic IoU loss function, which may also have a technical feature in which the termination condition is that the target detection model trains all training data sets for 12 rounds during the training process.

[0017] The present invention provides an anchor-free object detection method using a quadratic IoU loss function, which may also have the following technical features:

[0018]

[0019] Where p t is the category probability output by the classification module, Label information for the category, is the positive sample class, For background class, a t is a parameter factor that balances the imbalance of positive and negative samples. The balance factor is a constant. γ is a parameter factor that balances the imbalance of difficult and easy samples. The balance factor is a constant.

[0020] Functions and effects of the invention

[0021] The present invention discloses an anchor-free object detection method using a quadratic IoU loss function. Since the initial object model comprises a feature extraction module, a localization module, and a classification module, a quadratic IoU loss function is used to calculate the localization loss based on the predicted box position information and supervisory information. The localization loss is then used to differentiate the model parameters of the initial object detection model to obtain a primary IoU function. Therefore, during the training phase, the gradient update step size is determined based on the IoU between the predicted box and the target box, thereby capturing small changes in position. Furthermore, during the parameter update process, different gradients are obtained based on the IoU between the predicted box and the target box to control the different optimization effects on the object detection model parameters for objects of different sizes. This is because small objects are extremely sensitive to position. Slight changes in the position and scale of the predicted box of a small object will have a relatively large impact on its IoU with the target box, which in turn has a greater impact on parameter updates. Therefore, the detection performance of the predicted box of a small object in the object detection model is more significantly improved than that of medium and large objects. This adaptive adjustment of the gradient plays a crucial role in optimizing the object detection model during training, enabling more comprehensive learning and enhancing the detection performance of the object detection model. This method adjusts the step size of parameter update according to the IoU between the predicted box and the target box, and adds IoU as a whole to the parameter update process, thereby improving the positioning quality of the target detection model, thereby improving the detection effect of the target detection model and improving the current target detection algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 4 is a flowchart of a method for anchor-free object detection using a quadratic IoU loss function in an embodiment of the present invention;

[0023] Figure 2 is a flowchart of applying the quadratic IoU loss function in the object detection task in an embodiment of the present invention;

[0024] Figure 3 is a function image of the quadratic IoU loss function when λ=1.5, α=1, and β=0.5 in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following is a detailed description of the present invention involving an anchor-free target detection method using a quadratic IoU loss function in combination with embodiments and drawings.

[0026] <Example>

[0027] Figure 1 4 is a flowchart of an anchor-free target detection method using a quadratic IoU loss function in an embodiment of the present invention.

[0028] like Figure 1As shown, in this embodiment, the quadratic IoU loss function is used to replace the original positioning loss function to train the target detection model, and then the performance of the target detection model is tested, which specifically includes steps S1-1 to S1-2.

[0029] Step S1-1: preprocess the image to be detected to obtain a preprocessed image.

[0030] In step S1-2, the pre-processed image is input into a pre-trained target detection model for inference to obtain the corresponding image detection result.

[0031] In this embodiment, the target detection model needs to be trained in advance through a training process, specifically:

[0032] Figure 2 4 is a flowchart of applying the quadratic IoU loss function in the target detection task in an embodiment of the present invention.

[0033] like Figure 2 As shown, the training of the target detection model in this embodiment includes steps S2-1 to S2-6.

[0034] Step S2-1: Obtain a training image dataset and corresponding supervision information, where the supervision information includes positioning supervision information and category supervision information.

[0035] The training image dataset and the corresponding supervisory information need to be obtained by preprocessing the image dataset and the annotation information. In this embodiment, the preprocessing specifically includes labeling processing and data enhancement processing.

[0036] The marking process is as follows: the position and size of all objects of interest in the image are represented by rectangular boxes, and the categories corresponding to all objects are represented by rectangular boxes.

[0037] The data augmentation process is as follows: subtract the mean [102.9801, 115.9465, 122.7717] of the three RGB channels in the image, and normalize the image pixel values ​​to between -1 and 1; scale the image size proportionally to a minimum of 800 pixels on the shortest side and a maximum of 1333 pixels on the longest side.

[0038] Step S2-2: construct an initial target detection model, and input the training image dataset and supervision information into the initial target detection model. The initial target model has a feature extraction module, a positioning module, and a classification module.

[0039] The feature extraction module is used to extract a fixed-scale feature map as an output feature map based on the training image dataset.

[0040] The positioning module is used to process the output feature map to obtain the predicted box position information of each point on the output feature map.

[0041] The classification module is used to process the output feature map to obtain the confidence level of each point on the output feature map corresponding to all categories.

[0042] In step S2-3, the quadratic IoU loss function is used to calculate the positioning loss based on the predicted box position information and positioning supervision information, and the Focal Loss loss function is used to calculate the classification loss based on the confidence and category supervision information.

[0043] The optimizer used for training here is SGD, and the classification loss function is Focal Loss, which is as follows:

[0044]

[0045] Where p t is the category probability output by the classification module, Label information for the category, is the positive sample class, For background class, a t is a parameter factor that balances the imbalance of positive and negative samples. The balance factor is a constant. γ is a parameter factor that balances the imbalance of difficult and easy samples. The balance factor is a constant.

[0046] The positioning loss function is a quadratic IoU loss function, which is as follows:

[0047] Loss loc =λ-αIoU-βIoU 2

[0048] Where λ, α, and β are balance factors, which are constants and β≠0. IoU is the intersection over union (IoU) of the predicted box and the supervised target box, and its value range is [0,1].

[0049] Figure 3 is a function image of the quadratic IoU loss function when λ=1.5, α=1, and β=0.5 in an embodiment of the present invention.

[0050] like Figure 3 As shown in Figure 2, the quadratic IoU loss function at this time is the function image when λ=1.5, α=1, and β=0.5.

[0051] In addition, a grid search method is used to search for the hyperparameters α and β of the quadratic IoU loss function. That is, different combinations of α and β are used as the parameters of the quadratic IoU loss function to train the initial object detection model. After multiple training sessions, different object detection models with different effects are obtained. The α and β of the object detection model with the best effect are taken as the final grid search result.

[0052] In step S2-4, the model parameters of the initial target detection model are derived using the positioning loss and the classification loss respectively, and then the model parameters are updated using back propagation to obtain the updated model parameters.

[0053] Among them, when the model parameters of the initial target detection model are derived using the positioning loss and the classification loss respectively, the derivative function of the positioning loss includes the IoU term, and the form of the derivative function is as follows:

[0054] Loss′ loc =-α-2βIoU

[0055] Where α and β are balance factors, which are constants and β≠0. IoU is the intersection over union (IoU) of the predicted box and the supervised target box, and its value range is [0,1].

[0056] Step S2-5, judging whether the updated model parameters have reached the termination condition, if so, proceeding to step S2-6, otherwise proceeding to step S2-2.

[0057] Among them, the termination condition is that the target detection model is trained for 12 rounds on all training data sets during the training process.

[0058] Step S2-6, save the updated model parameters and load them into the initial target detection model to form a target detection model.

[0059] The above process completes the object detection model training, thus achieving object detection based on the object detection model. Next, the trained object detection model is tested using a pre-prepared test set and the output image detection results are evaluated.

[0060] Compared to other IoU regression loss functions, the quadratic IoU loss function used in this example can capture position sensitivity and improve positioning. The experimental results are shown in Table 1. mAP is used as the evaluation metric, and mAP is calculated for objects of different scales. Table 1 shows the experimental results of an object detection model using ResNet50 as the backbone network. The improvement is most significant for position-sensitive small-scale objects, followed by medium-scale objects. The improvement is smallest for large-scale objects, as large-scale objects are insensitive to small changes in position.

[0061] Table 1 Comparison of model effects (the bold value is the loss function proposed in this example)

[0062]

[0063] Example Function and Effect

[0064] According to this embodiment, a method for anchor-free object detection using a quadratic IoU loss function is provided. Since the initial object model includes a feature extraction module, a localization module, and a classification module, a quadratic IoU loss function is used to calculate the localization loss based on the predicted box position information and supervision information. The localization loss is then used to differentiate the model parameters of the initial object detection model to obtain a primary IoU function. Therefore, during the training phase, the gradient update step size is determined based on the IoU between the predicted box and the target box to capture subtle changes in position. Furthermore, during the parameter update process, different gradients are obtained based on the IoU between the predicted box and the target box to control the different optimization effects on the object detection model parameters for objects of different sizes. This is because small objects are extremely sensitive to position. Slight changes in the position and scale of the predicted box of a small object will have a relatively large impact on its IoU with the target box, which in turn has a greater impact on parameter updates. Therefore, the detection performance of the predicted box of a small object in the object detection model is more significantly improved than that of medium and large objects. This adaptive adjustment of the gradient plays a crucial role in optimizing the object detection model during training, enabling more comprehensive learning and enhancing the detection performance of the object detection model. This method adjusts the step size of parameter update according to the IoU between the predicted box and the target box, and adds IoU as a whole to the parameter update process, thereby improving the positioning quality of the target detection model, thereby improving the detection effect of the target detection model and improving the current target detection algorithm.

[0065] In the embodiment, by using the quadratic IoU loss function proposed in the present invention to replace the positioning loss function in the original target detection framework, the position information of the prediction box is added during the model parameter update process, thereby improving the detection effect of the target detection model.

[0066] In an embodiment, the quadratic IoU loss function proposed in the present invention is only used in the training stage. After the training is completed, the image to be detected is preprocessed to obtain a preprocessed image, and the preprocessed image is further input into a pre-trained target detection model for inference to obtain the corresponding image detection result, thereby improving the detection effect of the target detection model on the preprocessed image.

[0067] The above embodiments are only used to illustrate specific implementations of the present invention, and the present invention is not limited to the description scope of the above embodiments.

Claims

1. A method for anchor-free object detection using a quadratic IoU loss function, characterized in that: The steps include: Step S1-1, preprocessing the image to be detected to obtain a preprocessed image; Step S1-2: Input the pre-processed image into the pre-trained target detection model for inference to obtain the corresponding image detection result. The target detection model is pre-trained through the following steps: Step S2-1, obtaining a training image dataset and corresponding supervision information, wherein the supervision information includes positioning supervision information and category supervision information; Step S2-2: construct an initial target detection model, and input the training image dataset and the supervision information into the initial target detection model, wherein the initial target model has a feature extraction module, a positioning module, and a classification module. The feature extraction module is used to extract a fixed-scale feature map as an output feature map based on the training image dataset. The positioning module is used to process the output feature map to obtain the predicted box position information of each point on the output feature map, The classification module is used to process the output feature map to obtain the confidence of each point on the output feature map corresponding to all categories, Step S2-3, using the quadratic IoU loss function to calculate the positioning loss based on the predicted box position information and the positioning supervision information, and using the Focal Loss loss function to calculate the classification loss based on the confidence and the category supervision information; Step S2-4, respectively deriving the model parameters of the initial target detection model using the positioning loss and the classification loss, and then updating the model parameters using back propagation to obtain updated model parameters; Step S2-5, determining whether the updated model parameters meet the termination condition, if so, proceeding to step S2-6, otherwise proceeding to step S2-2; Step S2-6, saving the updated model parameters and loading them into the initial target detection model to form the target detection model.

2. The anchor-free object detection method using a quadratic IoU loss function according to claim 1, characterized in that: in, The quadratic IoU loss function is: Loss loc =λ-αIoU-βIoU 2 Where λ, α, and β are balance factors, which are constants and β≠0. IoU is the intersection over union (IoU) of the predicted box and the supervised target box, and its value range is [0, 1].

3. The anchor-free object detection method using a quadratic IoU loss function according to claim 2, characterized in that: in, When the model parameters of the initial target detection model are derived using the positioning loss and the classification loss respectively, the derivative function of the positioning loss includes the IoU term, and the form of the derivative function is as follows: Loss' loc =-α-2βIoU Where α and β are balance factors, which are constants and β≠0. IoU is the intersection over union (IoU) of the predicted box and the supervised target box, and its value range is [0, 1].

4. The anchor-free object detection method using a quadratic IoU loss function according to claim 1, wherein: in, The feature extraction module includes at least one or a combination of ResNet, DenseNet, and ResNet+FPN.

5. The anchor-free object detection method using a quadratic IoU loss function according to claim 1, wherein: in, The termination condition is that the target detection model is trained for 12 rounds on all the training data sets during the training process.

6. The anchor-free object detection method using a quadratic IoU loss function according to claim 1, characterized in that: in, The Focal Loss loss function is: Where p t is the category probability output by the classification module, Label information for the category, is the positive sample class, For background class, a t is a parameter factor that balances the imbalance of positive and negative samples. The balance factor is a constant. γ is a parameter factor that balances the imbalance of difficult and easy samples. The balance factor is a constant.

Images