Long-tail feature learning method based on class prior distribution and feature disturbance guidance

Abstract

The invention discloses a long-tail feature learning method based on class prior distribution and feature disturbance guidance. A batch of input long-tail distribution image samples are copied into three groups, and basic data enhancement operation is applied to the three groups respectively and then the three groups are input into a network model. Respectively carrying out feature disturbance on the comparison features of the first group and the second group through Gaussian disturbance and a class prior distribution coefficient; and for the third group of classification features, class center features are generated through full connection layer transformation, and class feature samples of each batch are supplemented to participate in training based on Gaussian disturbance frequency in negative correlation with the number of class samples. And finally, updating model parameters by calculating loss. Through the method of the invention, the model can learn the feature distribution with higher discrimination ability on the long-tail distribution data set, thereby remarkably improving the accuracy of long-tail classification. The method is also suitable for visual identification application scenes with long-tail distribution, such as automatic driving scene identification and medical image fine-grained classification.

Description

Technical Field

[0001] The present invention relates to image recognition technology under long-tail distribution in deep learning, and mainly to a method for guiding a model to perform feature learning on long-tail images by applying Gaussian perturbation to a class prior distribution. Background Art

[0002] In recent years, with the development of deep learning technology, deep neural networks have achieved remarkable results in computer vision tasks, especially in areas such as image classification and object detection. Thanks to the support of large-scale data sets (such as ImageNet), the performance of deep neural network models has been continuously improved. However, in actual application scenarios, data usually presents the characteristics of long-tail distribution, that is, some common "head" categories have a large number of samples, while many uncommon or rare "tail" categories have only very few samples. Due to the insufficient number of tail category samples, it is difficult for the model to learn enough feature information, which brings challenges to some high-precision recognition tasks, such as medical image analysis, autonomous driving and other fields. In these applications, it is crucial to improve the recognition accuracy of tail categories.

[0003] To address this problem, traditional methods mainly focus on two aspects: resampling and reweighting. Resampling methods balance the category distribution in the dataset by oversampling the tail categories or undersampling the head categories. Oversampling techniques increase the number of samples in the tail categories by generating new samples, while undersampling methods reduce the number of samples in the head categories. However, resampling methods may cause the model to overfit the samples in the tail categories or lose important information in the head categories. Another type of method is the reweighting strategy, which gives a greater weight to the tail categories in the loss function, so that the model pays more attention to the samples in the tail categories during training. Although these methods have improved the model's recognition ability for the tail categories to a certain extent, they have certain limitations because they cannot fundamentally solve the imbalance problem in feature learning. In recent years, contrastive learning technology, as an emerging feature learning method, has demonstrated powerful capabilities. In particular, the supervised contrastive learning (SCL) technology introduces label information to make samples of the same type closer in the feature space, while samples of different types are far away from each other, thereby improving intra-class consistency and inter-class separation. However, there are still two major problems when directly applying SCL to long-tail distribution data. First, since there are more samples in the head category, they contribute more to the SCL loss function, which will cause the distribution in the feature space to be biased towards the head category, thus affecting the feature learning effect of the tail category. Second, supervised contrastive learning relies on the existence of "difficult samples" (i.e., feature samples that are difficult to learn in the feature space) to improve the separability of features, but in long-tail distribution data, the samples of the tail category are relatively small, and it is difficult to provide enough "difficult samples", which will further reduce the classification performance of the tail category. Summary of the invention

[0004] Purpose of the invention: In view of the problems existing in the above-mentioned background technology, the present invention starts from learning discriminative long-tail features and provides a long-tail feature learning method based on class prior distribution and feature perturbation guidance. During the training process, our method applies a perturbation amplitude that is negatively correlated with the number of categories to the samples of each category, thereby generating more "difficult samples" and balancing the number of samples in each batch by perturbing the class center. In the process of contrastive learning, these perturbed features and the generated additional features can be close to their respective category centers in the feature space, thereby improving intra-class consistency and inter-class separability. In addition, we introduced a logarithmic compensated cross entropy loss to further reduce the bias of the classifier and improve the classification performance under long-tail distribution data.

[0005] Technical solution: A long-tail feature learning method based on class prior distribution and feature perturbation guidance, including the following steps:

[0006] Step S1: Input the original long-tail dataset S, perform basic geometric transformation and color transformation on each batch of images I∈S, so as to achieve data enhancement. Then, input the enhanced image data into the pre-initialized convolutional neural network model to extract image features;

[0007] Step S2, applying a Gaussian perturbation with an amplitude negatively correlated with the number of categories to the features extracted by the model, generating a feature set containing the perturbation, which is used as a comparison feature set;

[0008] Step S3, the features output by the model are calculated through a fully connected layer to obtain the proxy center features of the category, and based on the Gaussian perturbation frequency that is negatively correlated with the number of category samples, Gaussian perturbation is applied to the proxy center features to form a class supplementary feature set in batch training, thereby achieving category balance locally;

[0009] Step S4: Combine the contrast feature set generated in step S2, the class supplement feature set generated in step S3, and the label set of the sample to calculate the supervised contrast learning loss. At the same time, calculate the classification loss based on supervised learning. Then, add the weighted sum of the two losses to obtain the joint loss. Finally, update the parameters of the convolutional neural network model and the classifier through back propagation;

[0010] Step S5: Repeat steps S1 to S4 to perform B rounds of iterative training. At the same time, use the validation set to evaluate the classification and recognition accuracy of the model and save the optimal model weights.

[0011] Step S6: Input the long-tail image test set that has not been trained by the model into the network model trained in the above steps to generate a classification prediction result for each test sample.

[0012] Preferably, in step 2, the amplitude that is negatively correlated with the number of categories, that is, the disturbance coefficient of category t is defined as λ t :

[0013]

[0014] In a long-tail image dataset S with C categories, the total number of samples of a certain category t is n t , sort all classes in descending order according to the number of samples, that is, n1>n2>…>n C ; The convolutional network model extracts features from sample i to obtain feature f i , through the category perturbation coefficient λ t , obtain the contrast feature p after disturbance i , defined as follows:

[0015] p i =f i +λ t *∈

[0016] where ∈ represents a normalized perturbation parameter randomly sampled from a Gaussian distribution, and its magnitude is determined by the perturbation coefficient λ t Control, while λ t It is negatively correlated with the number of categories. That is, the tail classes (classes with few samples) in the long-tail dataset will receive a wider disturbance in the feature space, so that the model can have more robust recognition and parameter optimization for the tail classes during training.

[0017] Preferably, in step 3, the class supplementary feature solving process is as follows:

[0018] Step S3.1: Trainable temporary proxy center features o te Defined as:

[0019] o te =w t .A

[0020] Among them, w t represents the tth row of the classifier parameter matrix W, that is, the parameter vector of category t, with a size of 1×d, and A represents the weight matrix of the fully connected layer, with a size of d×b. t Represents the agent center feature of category t, with size 1×b.

[0021] Step S3.2: In each batch, we need to calculate the average feature M of each class in the current batch o , update the proxy center feature in a weighted manner to obtain o t , calculated as follows:

[0022] o t =θ*o te +(1-θ)*o te

[0023] Among them, θ∈[0,1] represents the weighting factor.

[0024] Step S3.3: Obtain the perturbed class supplementary features by applying a random Gaussian perturbation of a fixed amplitude, which is defined as follows:

[0025] g t =o t +η*∈

[0026] Where η∈[-1, 1] represents a fixed disturbance amplitude.

[0027] Step S3.4: The perturbation frequency described in step 3, which is negatively correlated with the number of categories, is calculated as follows: First, a batch class balance cardinality N is defined u , for the current batch, count the number of times each class appears and record it as n tBatch , let γ=(Nu -n tBatch ). When γ>0, repeat the Bernoulli experiment for γ times to obtain the result test=Bernoulli(λ t ), if test=1, through the class center o t Perform a random Gaussian perturbation to obtain a class supplementary feature sample g of class t t .

[0028] Preferably, step S4 optimizes the parameters of the model by calculating the supervised contrast loss and the classification loss, wherein for the sample feature i output by the model, its supervised classification loss is calculated. The results are as follows:

[0029]

[0030] Where |B| represents the number of sample feature sets B involved in calculating the classification loss in the current batch. Y represents the label set of long-tail data. y Represents the feature currently output by the convolutional network model, whose label is y, and y' and y are different labels. π y’ It represents the normalized proportion of the number of training sample sets with label y' in the number of training sample sets of all classes. For the supervised contrast loss mentioned in step S4, it is defined as:

[0031]

[0032] Among them B y is a subset of the comparison feature set and class supplementary feature set in the current batch, and its label is y. B represents the current training batch feature set. i and p j Represents two samples of the same class. k Represents other feature samples of the current batch, with label k. The supervised contrast loss and the supervised classification loss are jointly calculated through weighting coefficients α and β to obtain the joint loss L.

[0033] Beneficial effects:

[0034] This method introduces a category-based feature perturbation mechanism, which generates more challenging difficult samples through different degrees of perturbation, prompting the model to learn more discriminative features, thereby improving the generalization ability of the model. Through category-aware feature generation, the method dynamically balances the data distribution during training, alleviates the problem of insufficient number of long-tail category samples, and makes the model more effective in learning long-tail categories. The supervised contrast loss can bring similar samples closer and push heterogeneous samples away during training. At the same time, the proposed feature perturbation and feature class supplementation are combined to achieve a balanced and robust contrastive learning process, thereby improving the model's discriminability and robustness when extracting long-tail features, and achieving higher-precision classification and recognition capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a long-tail feature learning method guided by class prior distribution and feature disturbance provided by the present invention;

[0036] Figure 2 This is a framework diagram of a long-tail feature learning algorithm provided by the present invention guided by class prior distribution and feature perturbation. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The present invention provides a long-tail feature learning method based on class prior distribution and feature perturbation guidance. The specific process is as follows: Figure 1 As shown, the following steps are included:

[0039] Step S1: Input the original long-tail dataset S, perform basic geometric transformation and color transformation on each batch of images I∈S, so as to achieve data enhancement. Then, input the enhanced image data into the pre-initialized convolutional neural network model to extract image features;

[0040] Specifically, Figure 2 , each image needs to be copied into 3 copies, and basic data enhancements are performed on each copy, such as random image flipping and color jittering. The first and second data-enhanced images are used as comparison image pairs. The third image is used as the classification image. The three images are respectively passed through the same initialized ResNet-50 convolutional neural network for feature extraction.

[0041] Step S2, applying a Gaussian perturbation with an amplitude negatively correlated with the number of categories to the features extracted by the model, generating a feature set containing the perturbation, which is used as a comparison feature set;

[0042] Specifically, Figure 2 The feature perturbation module applies feature perturbations based on class prior distribution to the contrast feature pair f obtained in step S1 in the following ways to obtain the perturbed feature p i , the calculation formula is as follows:

[0043] p i =f i +λ t *∈

[0044] where ∈ represents a normalized perturbation parameter randomly sampled from a Gaussian distribution, and its magnitude is determined by the perturbation coefficient λ t Control, while λ t It is negatively correlated with the number of categories. That is, the tail classes (classes with few samples) in the long-tail dataset will receive a wider disturbance in the feature space, so that the model can have more robust recognition and parameter optimization for the tail classes during training.

[0045] Step S3, the classification features output by the model are calculated through a fully connected layer to obtain the proxy center features of the category, and based on the Gaussian perturbation frequency that is negatively correlated with the number of category samples, Gaussian perturbation is applied to the proxy center features to form a class supplementary feature set in batch training, thereby achieving category balance locally;

[0046] Specifically, in step 3, the class supplementary feature solving process is as follows:

[0047] Step S3.1: Trainable temporary proxy center features o te Defined as:

[0048] o te =w t .A

[0049] Among them, w t represents the tth row of the classifier parameter matrix W, that is, the parameter vector of category t, with a size of 1×d, and A represents the weight matrix of the fully connected layer, with a size of d×b. t Represents the agent center feature of category t, with size 1×b.

[0050] Step S3.2: In each batch, we need to calculate the average feature M of each class in the current batch o , update the proxy center feature in a weighted manner to obtain o t , calculated as follows:

[0051] o t =θ*o te +(1-θ)*o te

[0052] Among them, θ∈[0,1] represents the weighting factor.

[0053] Step S3.3: Obtain the perturbed class supplementary features by applying a random Gaussian perturbation of a fixed amplitude, which is defined as follows:

[0054] g t =o t +η*∈

[0055] Where η∈[-1, 1] represents a fixed disturbance amplitude.

[0056] Step S3.4: The perturbation frequency described in step 3, which is negatively correlated with the number of categories, is calculated as follows: First, a batch class balance cardinality N is defined u , for the current batch, count the number of times each class appears and record it as n tBatch , let γ=(N u -n tBatch ). When γ>0, repeat the Bernoulli experiment for γ times to obtain the result test=Bernoulli(λ t ), if test=1, through the class center o t Perform a random Gaussian perturbation to obtain a class supplementary feature sample g of class t t .

[0057] Step S4: Combine the contrast feature set generated in step S2, the class supplement feature set generated in step S3, and the label set of the sample to calculate the supervised contrast learning loss. At the same time, calculate the classification loss based on supervised learning.

[0058] Specifically, for the sample feature i output by the model, its supervised classification loss is calculated The results are as follows:

[0059]

[0060] Where |B| represents the number of sample feature sets B involved in calculating the classification loss in the current batch. Y represents the label set of long-tail data. y Represents the feature currently output by the convolutional network model, whose label is y, and y' and y are different labels. π y’ It represents the normalized proportion of the number of training sample sets with label y' in the number of training sample sets of all classes. For the supervised contrast loss mentioned in step S4, it is defined as:

[0061]

[0062] Among them B y is a subset of the comparison feature set and class supplementary feature set in the current batch, and its label is y. B represents the current training batch feature set. iand p j Represents two samples of the same class. k Represents other feature samples of the current batch, with label k. The supervised contrast loss and the supervised classification loss are jointly calculated through weighted coefficients α and β to obtain the joint loss L. The parameters of the convolutional neural network model and classifier are updated through back propagation;

[0063] Step S5: Repeat steps S1 to S4 to perform B rounds of iterative training. At the same time, use the validation set to evaluate the classification and recognition accuracy of the model and save the optimal model weights.

[0064] Step S6: Input the long-tail image test set that has not been trained by the model into the network model trained in the above steps to generate a classification prediction result for each test sample.

[0065] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A long-tail feature learning method based on class prior distribution and feature perturbation guidance, characterized in that: The steps of the method for long-tail data training and learning are as follows: Step S1: Input the original long-tail dataset S, perform basic geometric transformation and color transformation on each batch of images I∈S, so as to achieve data enhancement. Then, input the enhanced image data into the pre-initialized convolutional neural network model to extract image features; Step S2, applying a Gaussian perturbation with an amplitude negatively correlated with the number of categories to the features extracted by the model, generating a feature set containing the perturbation, which is used as a comparison feature set; Step S3, the features output by the model are calculated through a fully connected layer to obtain the proxy center features of the category, and based on the Gaussian perturbation frequency that is negatively correlated with the number of category samples, Gaussian perturbation is applied to the proxy center features to form a class supplementary feature set in batch training, thereby achieving category balance locally; Step S4: Combine the contrast feature set generated in step S2, the class supplement feature set generated in step S3, and the label set of the sample to calculate the supervised contrast learning loss. At the same time, calculate the classification loss based on supervised learning. Then, add the weighted sum of the two losses to obtain the joint loss. Finally, update the parameters of the convolutional neural network model and the classifier through back propagation; Step S5: Repeat steps S1 to S4 to perform B rounds of iterative training. At the same time, use the validation set to evaluate the classification and recognition accuracy of the model and save the optimal model weights. Step S6: Input the long-tail image test set that has not been trained by the model into the network model trained in the above steps to generate a classification prediction result for each test sample.

2. The long-tail feature learning method based on class prior distribution and feature perturbation guidance according to claim 1, characterized in that: In step 2, the amplitude that is negatively correlated with the number of categories, that is, the perturbation coefficient of category t is defined as λ t : In a long-tail image dataset S with C categories, the total number of samples of a certain category t is n t , sort all classes in descending order according to the number of samples, that is, n1>n2>…>n C ; The convolutional network model extracts features from sample i to obtain feature f i , through the category perturbation coefficient λ t , obtain the contrast feature p after disturbance i , defined as follows: p i =f i +λ t *e where ε represents a normalized perturbation parameter randomly sampled from a Gaussian distribution, and its magnitude is determined by the perturbation coefficient λ t Control, while λ t It is negatively correlated with the number of categories. That is, the tail classes (classes with few samples) in the long-tail dataset will receive a wider disturbance in the feature space, so that the model can have more robust recognition and parameter optimization for the tail classes during training.

3. The long-tail feature learning method based on class prior distribution and feature perturbation guidance according to claim 2, characterized in that: In step 3, the class supplementary feature solution process is as follows: Step S3.1: Trainable temporary proxy center features o te Defined as: o te =w t .A Among them, w t represents the tth row of the classifier parameter matrix W, that is, the parameter vector of category t, with a size of 1×d, and A represents the weight matrix of the fully connected layer, with a size of d×b. t Represents the agent center feature of category t, with size 1×b. Step S3.2: In each batch, we need to calculate the average feature M of each class in the current batch o , update the proxy center feature in a weighted manner to obtain 0 t , calculated as follows: the t =θ*o te +(1-θ)*o te Among them, θ∈[0,1] represents the weighting factor. Step S3.3: Obtain the perturbed class supplementary features by applying a random Gaussian perturbation of a fixed amplitude, which is defined as follows: g t =o t +h*e Where η∈[-1, 1] represents a fixed disturbance amplitude. Step S3.4: The perturbation frequency described in step 3, which is negatively correlated with the number of categories, is calculated as follows: First, a batch class balance cardinality N is defined u ,For the current batch, count the number of times each class appears and record it as n tBatch , let γ=(N u -n tBatch ). When γ>0, repeat the Bernoulli experiment for γ times to obtain the result test=Bernoulli(λ t ), if test=1, through the class center o t Perform a random Gaussian perturbation to obtain a class supplementary feature sample g of class t t .

4. The long-tail feature learning method based on class prior distribution and feature perturbation guidance according to claim 3, characterized in that: Step S4 optimizes the model parameters by calculating the supervised contrast loss and the classification loss. For the sample feature i output by the model, its supervised classification loss is calculated. The results are as follows: Where |B| represents the number of sample feature sets B involved in calculating the classification loss in the current batch. Y represents the label set of long-tail data. y Represents the feature currently output by the convolutional network model, whose label is y, and y' and y are different labels. π y’ It represents the normalized proportion of the number of training sample sets with label y' in the number of training sample sets of all classes. For the supervised contrast loss mentioned in step S4, it is defined as: Among them B y is a subset of the comparison feature set and class supplementary feature set in the current batch, and its label is y. B represents the current training batch feature set. i and p j Represents two samples of the same class. k Represents other feature samples of the current batch, with label k. Through the weighting coefficients α and , the supervised contrast loss and the supervised classification loss are jointly calculated to obtain the joint loss L.

Images