A long-tail object detection method for adaptively eliminating negative gradient of classification

Abstract

The application discloses a long-tail target detection method for adaptively eliminating classification negative gradient, comprising the following steps: 1) constructing a long-tail target detection model, obtaining a long-tail data set, inputting a training image into the model, calculating a classification loss and a positioning loss for a prediction result, obtaining the classification loss by a loss function for adaptively eliminating negative gradient of a tail class, and obtaining a long-tail learning total loss value by weighted summation of the classification loss and the positioning loss; 2) performing gradient back propagation and parameter updating by using the long-tail learning total loss value, saving model parameters with optimal performance after completing all rounds of training, and obtaining an optimal long-tail target detection model; and 3) inputting a to-be-detected image in a test set into the optimal model to obtain a prediction result of object category and position in the to-be-detected image. The application can adaptively eliminate negative gradient of a tail class according to output probability, is helpful to solve false positive problems caused by insufficient learning of the tail class, and improves the performance of a target detection model.

Description

Technical Field

[0001] This invention relates to the technical field of long-tailed object detection, and in particular to an adaptive method for eliminating negative gradients in classification for long-tailed object detection. Background Technology

[0002] Object detection, a fundamental task in computer vision, has made significant progress. Existing object detection algorithms have achieved good performance on balanced datasets. However, in reality, data from different categories is often imbalanced, with a small number of categories comprising the majority of samples (called head classes) and most categories having only a small number of samples (called tail classes). This imbalanced data distribution appears as a long-tailed distribution in graphs. This extreme imbalance in data distribution leads to poor performance of existing object detection models, often resulting in very low detection accuracy on the tail classes with their limited sample size.

[0003] Current methods for detecting long-tailed targets often employ resampling, which balances the dataset by undersampling the head class and oversampling the tail class. However, undersampling the head class degrades the model's detection performance for that class, while undersampling the tail class leads to overfitting. Another common approach is reweighting, which assigns greater weight to the tail class in the loss function; however, this method often yields suboptimal results. Many existing techniques focus on mitigating the extreme imbalance of positive and negative gradients to the tail class by eliminating negative gradients from all other classes, thereby improving the model's detection performance for that class. However, blindly eliminating negative gradients from all other classes for the tail class can result in inadequate suppression of the tail class, leading to false positives. Summary of the Invention

[0004] The purpose of this invention is to address the false positive problem caused by eliminating all negative gradients in current mainstream methods. It proposes an adaptive method for eliminating negative gradients in classification for long-tailed target detection, which can adaptively eliminate negative gradients for tail classes based on the output probability. This helps to solve the false positive problem caused by insufficient learning of tail classes and improves the performance of target detection models.

[0005] To achieve the above objectives, the technical solution provided by this invention is: an adaptive method for eliminating negative gradients in classification for long-tailed target detection, comprising the following steps:

[0006] 1) Construct a long-tailed object detection model consisting of a backbone network, a region generation network, and a detection head; obtain a long-tailed dataset and divide it into a training set, a validation set, and a test set; sample training images and labels from the training set, input the images into the long-tailed object detection model, extract feature maps from the images by the backbone network, and generate proposed regions that may contain objects by the region generation network. After the proposed regions are mapped onto the feature maps, they are scaled to a uniform size, and finally the detection head predicts the proposed regions; calculate the classification loss and localization loss for the prediction results, where the classification loss is calculated by an adaptive loss function that eliminates the negative gradient of the tail class. The classification loss and localization loss are weighted and summed to obtain the total long-tail learning loss value.

[0007] 2) Use the obtained long-tail learning total loss value to perform gradient backpropagation and parameter update on the long-tail object detection model; after each round of training, use the validation set to evaluate the model's performance; after completing all rounds of training, save the parameters of the best-performing model to obtain the optimal long-tail object detection model.

[0008] 3) Input the images to be detected from the test set into the trained long-tailed object detection model to obtain the prediction results of the object category and location in the images to be detected.

[0009] Furthermore, in step 1), long-tail datasets are used for training to detect long-tailed distributed targets.

[0010] Furthermore, in step 1), the classification loss for each proposed region is calculated as follows:

[0011]

[0012] w j =1-E(r)T λ (f j )T ξ (p j (1-y) j )

[0013] In the formula, L AEL The classification loss represents a single proposed region; C represents the number of classes; r is the first letter of "region," representing the proposed region; p j Let be the probability value of the j-th class; This represents the predicted value for class j. If the proposed region belongs to class j, then... equals p j Otherwise, it equals 1-p j ;y j The label representing the proposed region is 1 if the proposed region belongs to class j, and 0 otherwise; w j f represents the weight coefficient of the j-th class; j Let N be the sample frequency of class j, given by Nj / N is calculated, where N j Let E(r) be the number of samples in class j, and N be the total number of samples; T λ (f j ) and T ξ (p j The numbers () represent the foreground / background discriminant function, the frequency discriminant function, and the probability discriminant function, respectively, and their formulas are expressed as follows:

[0014]

[0015]

[0016]

[0017] In the formula, λ represents the frequency threshold; ξ represents the probability threshold, and its value is between [0,1].

[0018] The negative gradient elimination operation is as follows: During the training of the long-tail object detection model, after feature extraction, region generation, and classification localization, several proposed regions are obtained with probability values ​​belonging to each class. If the following conditions are met simultaneously: the proposed region is foreground, the j-th class is the tail class (i.e., the sample frequency is less than the threshold λ), the proposed region does not belong to the j-th class, and its class output probability is less than the threshold ξ, then the classification loss value of the j-th class item is set to 0, thereby eliminating the negative gradient to the j-th class.

[0019] Furthermore, in step 1), the negative gradient of the tail class is adaptively eliminated according to the output probability. If a false positive is caused by the high output probability of the tail class, the negative gradient of the tail class is not eliminated; otherwise, it is eliminated.

[0020] Furthermore, in step 2), the detection performance metric used to evaluate the model's performance is AP, which stands for Average Precision, a performance metric that combines recall and completeness.

[0021] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0022] 1. The method of the present invention alleviates the extremely unbalanced positive and negative gradients experienced by the tail class by eliminating some of the negative gradients of other classes on the tail class.

[0023] 2. The output probability used in the method of this invention reflects the learning state of the model to a certain extent. Therefore, the output probability is introduced as a criterion in the classification loss function, which can adaptively eliminate the tail class negative gradient and help the model to be trained better.

[0024] 3. The method of this invention addresses the issue of excessively high output probabilities for the tail class of incorrect tail class foreground objects by retaining the negative gradient for the tail class. This effectively solves the false positive problem caused by insufficient tail class learning and improves the performance of the target detection model.

[0025] 4. The method of the present invention is modular and universal, plug and play, and can be extended to application fields with long-tail data characteristics, such as wildlife identification and traffic condition identification. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the Faster RCNN object detection model structure using the method of this invention. Detailed Implementation

[0027] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0028] This invention proposes an adaptive method for eliminating negative gradients in classification for detecting long-tailed targets. This method can be directly applied to various target detection models with simple configuration modifications. The following section uses Faster R-CNN as an example to provide a detailed explanation of this method.

[0029] First, construct as follows Figure 1 The Faster RCNN model shown is a long-tail object detection model, which includes the backbone network ResNet50, the region generation network RPN, the detection head RCNN, and ROI Align for uniformly proposing region scales.

[0030] Taking the long-tailed dataset LVIS v1.0 as an example, LVIS v1.0 is a typical long-tailed dataset, consisting of 1203 categories. Each category contains a maximum of 2.2k instances and a minimum of only 1 instance. The LVIS v1.0 training set contains approximately 100k images and 1.3M instances; the validation set contains approximately 20k images and 245k instances; and the test set contains approximately 20k images.

[0031] The Faster R-CNN is trained using the LVIS v1.0 training set images, and the process includes the following steps:

[0032] 1) The input image is first used for feature extraction by the backbone network ResNet50. As the network deepens, five feature maps of varying scales can be obtained.

[0033] 2) Input the last feature map into the Region Generation Network (RPN). The RPN filters a large number of pre-generated anchor boxes to obtain 128 anchor boxes that are most likely to include objects. Then, it randomly selects 128 background anchor boxes that do not contain objects to obtain 256 proposed regions.

[0034] 3) After the proposed region is mapped onto the feature map, it is then scaled to the specified size using ROI Align;

[0035] 4) Finally, the detection head predicts the category and location of each proposed region. Specifically, after passing through the fully connected layer, the localization branch determines the object location, and the classification branch determines the object category.

[0036] 5) Calculate the classification loss and weighted average for the prediction results of each proposed region, and then sum them to obtain the total long-tail learning loss. Specifically, for the classification result of each proposed region, the classification loss is calculated by the loss function that adaptively eliminates the tail-like negative gradients:

[0037]

[0038] w j =1-E(r)T λ (f j )T ξ (p j (1-y) j )

[0039] In the formula, L AEL The classification loss represents a single proposed region; C represents the number of classes; r is the first letter of "region," representing the proposed region; p j Let be the probability value of the j-th class; This represents the predicted value for class j. If the proposed region belongs to class j, then... equals p j Otherwise, it equals 1-p j ;y j The label representing the proposed region is 1 if the proposed region belongs to class j, and 0 otherwise; w j f represents the weight coefficient of the j-th class; j Let N be the sample frequency of class j, given by N j / N is calculated, where N j Let E(r) be the number of samples in class j, and N be the total number of samples. λ (f j ) and T ξ (p j The numbers ) represent the foreground / background discriminant function, the frequency discriminant function, and the probability discriminant function, respectively, and their formulas are expressed as follows:

[0040]

[0041]

[0042]

[0043] In the formula, λ represents the frequency threshold; ξ represents the probability threshold, both of which take values ​​between [0,1].

[0044] The negative gradient elimination operation is as follows: During the training of the long-tail object detection model, after feature extraction, region generation, and classification localization, several proposed regions are obtained with probability values ​​belonging to each class. If the following conditions are met simultaneously: the proposed region is foreground, the j-th class is the tail class (i.e., the sample frequency is less than the threshold λ), the proposed region does not belong to the j-th class, and its class output probability is less than the threshold ξ, then the classification loss value of the j-th class item is set to 0, thereby eliminating the negative gradient to the j-th class.

[0045] 6) Use the long-tail learning total loss value calculated in step 5) to perform gradient backpropagation and parameter update;

[0046] 7) After each complete training cycle, calculate the AP value of the model on the validation set;

[0047] 8) After completing all training rounds, save the parameters of the model with the best performance to obtain the optimal long-tail object detection model.

[0048] By inputting the images to be detected from the test set into the trained long-tailed object detection model, the predicted categories and locations of objects in the images can be obtained.

[0049] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for long-tail object detection that adaptively eliminates the negative gradient of classification, characterized in that, The method comprises the following steps: 1) constructing a long-tail object detection model composed of a backbone network, a region generation network and a detection head; obtaining a long-tail dataset and dividing it into a training set, a validation set and a test set; sampling training images and labels from the training set, inputting the images into the long-tail object detection model, extracting feature maps of the images by the backbone network, generating proposal regions possibly containing objects by the region generation network, scaling the proposal regions to a uniform size after mapping the proposal regions to the feature maps, and finally predicting results of the proposal regions by the detection head; calculating classification loss and positioning loss for the predicted results, wherein the classification loss is calculated by a loss function that adaptively eliminates negative gradients of tail classes, and the classification loss and the positioning loss are weighted and summed to obtain a long-tail learning total loss value; The classification loss of each proposal region is calculated as: ； ； where, is the classification loss of a single proposal region; is the number of classes; is the first letter of region, representing the proposal region; is the probability value of the j-th class; is the prediction value of the j-th class, when the proposal region belongs to the j-th class, then is equal to , otherwise is equal to ; is the label of the proposal region, which is 1 when the proposal region belongs to the j-th class, otherwise is 0; is the weight coefficient of the j-th class; is the sample frequency of the j-th class, which is calculated by , where is the number of samples of the j-th class, is the total number of samples; , and represent the foreground / background discriminant function, the frequency discriminant function and the probability discriminant function respectively, and their formula expressions are respectively: ； ； ； wherein represents a frequency threshold value; represents a probability threshold value, both of which are in the range [0, 1]; The negative gradient elimination operation is: during training of the long tail object detection model, after feature extraction, region generation and classification positioning, probability values of a plurality of proposal regions belonging to each class are obtained, if the following conditions are met simultaneously: the proposal region is foreground, the jth class is a tail class, i.e., the sample frequency is less than a threshold , the proposal region does not belong to the jth class, and the class output probability is less than a threshold , the classification loss value of the jth class item is set to 0, so as to eliminate the negative gradient of the jth class. According to the output probability, the negative gradient of the tail class is adaptively eliminated. If false positives are caused due to high output probability of the tail class, the negative gradient of the tail class is not eliminated, otherwise the negative gradient of the tail class is eliminated 2) using the obtained long-tail learning total loss value to perform gradient backpropagation and parameter update on the long-tail object detection model; after completing a complete round of training, the performance of the model is evaluated using the validation set; After completing all rounds of training, the model parameters with the best performance are saved to obtain an optimal long-tail object detection model; 3) inputting a to-be-detected image in the test set into the trained long-tail object detection model to obtain the prediction results of the object class and position in the to-be-detected image.

2. The method of claim 1, wherein, In step 1), the long-tail dataset is used for training to detect long-tail distributed objects.

3. The method of claim 1, wherein the method further comprises: In step 2), the detection performance index used for performance evaluation of the model is AP, which is the average precision, a performance index that comprehensively considers recall rate and recall rate.

Images